Unix/Linux Go Back    


Linux 2.6 - man page for erl_scan (linux section 3erl)

Linux & Unix Commands - Search Man Pages
Man Page or Keyword Search:   man
Select Man Page Set:       apropos Keyword Search (sections above)


erl_scan(3erl)			     Erlang Module Definition			   erl_scan(3erl)

NAME
       erl_scan - The Erlang Token Scanner

DESCRIPTION
       This module contains functions for tokenizing characters into Erlang tokens.

DATA TYPES
       category() = atom()
       column() = integer() > 0
       line() = integer()
       location() = line() | {line(), column()}
       reserved_word_fun() -> fun(atom()) -> bool()
       set_attribute_fun() -> fun(term()) -> term()
       symbol() = atom() | float() | integer() | string()
       token() = {category(), attributes()} | {category(), attributes(), symbol()}
       attributes() = line() | list() | tuple()

EXPORTS
       string(String) -> Return
       string(String, StartLocation) -> Return
       string(String, StartLocation, Options) -> Return

	      Types  String = string()
		     Return = {ok, Tokens, EndLocation} | Error
		     Tokens = [token()]
		     Error = {error, ErrorInfo, EndLocation}
		     StartLocation = EndLocation = location()
		     Options = Option | [Option]
		     Option   =   {reserved_word_fun,reserved_word_fun()}   |  return_comments	|
		     return_white_spaces | return | text

	      Takes the list of characters String and tries to scan (tokenize) them. Returns {ok,
	      Tokens, EndLocation} , where Tokens are the Erlang tokens from String . EndLocation
	      is the first location after the last token.

	      {error, ErrorInfo, EndLocation} is returned if an error occurs. EndLocation is  the
	      first location after the erroneous token.

	      string(String)  is  equivalent to string(String, 1) , and string(String, StartLoca-
	      tion) is equivalent to string(String, StartLocation, []) .

	      StartLocation indicates the initial location when scanning starts. If StartLocation
	      is  a  line attributes() as well as EndLocation and ErrorLocation will be lines. If
	      StartLocation is a pair of a line and a column attributes() takes the  form  of  an
	      opaque  compound	data  type,  and EndLocation and ErrorLocation will be pairs of a
	      line and a column. The token attributes contain information about  the  column  and
	      the  line  where	the  token  begins, as well as the text of the token (if the text
	      option is given), all of	which  can  be	accessed  by  calling  token_info/1,2  or
	      attributes_info/1,2 .

	      A  token	is  a  tuple  containing  information about syntactic category, the token
	      attributes, and the actual terminal symbol. For punctuation characters (e.g. ; ,	|
	      ) and reserved words, the category and the symbol coincide, and the token is repre-
	      sented by a two-tuple. Three-tuples have one of the following forms:  {atom,  Info,
	      atom()}  ,  {char,  Info,  integer()}  ,	{comment, Info, string()} , {float, Info,
	      float()} , {integer, Info, integer()} , {var, Info,  atom()}  ,  and  {white_space,
	      Info, string()} .

	      The valid options are:

		{reserved_word_fun, reserved_word_fun()} :
		  A callback function that is called when the scanner has found an unquoted atom.
		  If the function returns true , the unquoted atom itself will be the category of
		  the  token;  if  the	function returns false , atom will be the category of the
		  unquoted atom.

		return_comments :
		  Return comment tokens.

		return_white_spaces :
		  Return white space tokens. By convention, if there is a newline  character,  it
		  is  always  the first character of the text (there cannot be more than one new-
		  line in a white space token).

		return :
		  Short for [return_comments, return_white_spaces] .

		text :
		  Include the token's text in the token attributes. The text is the part  of  the
		  input corresponding to the token.

       tokens(Continuation, CharSpec, StartLocation) -> Return
       tokens(Continuation, CharSpec, StartLocation, Options) -> Return

	      Types  Continuation = [] | Continuation1
		     Return = {done, Result, LeftOverChars} | {more, Continuation1}
		     LeftOverChars = CharSpec
		     CharSpec = string() | eof
		     Continuation1 = tuple()
		     Result = {ok, Tokens, EndLocation} | {eof, EndLocation} | Error
		     Tokens = [token()]
		     Error = {error, ErrorInfo, EndLocation}
		     StartLocation = EndLocation = location()
		     Options = Option | [Option]
		     Option   =   {reserved_word_fun,reserved_word_fun()}   |  return_comments	|
		     return_white_spaces | return | text

	      This is the re-entrant scanner which scans characters until a dot ('.' followed  by
	      a white space) or eof has been reached. It returns:

		{done, Result, LeftOverChars} :
		  This	return	indicates  that  there	is sufficient input data to get a result.
		  Result is:

		  {ok, Tokens, EndLocation} :
		    The scanning was successful. Tokens is the list of tokens including dot .

		  {eof, EndLocation} :
		    End of file was encountered before any more tokens.

		  {error, ErrorInfo, EndLocation} :
		    An error occurred. LeftOverChars is the remaining  characters  of  the  input
		    data, starting from EndLocation .

		{more, Continuation1} :
		  More	data  is  required for building a term. Continuation1 must be passed in a
		  new call to tokens/3,4 when more data is available.

	      The CharSpec eof signals end of file. LeftOverChars will then take the value eof as
	      well.

	      tokens(Continuation, CharSpec, StartLocation) is equivalent to tokens(Continuation,
	      CharSpec, StartLocation, []) .

	      See string/3 for a description of the various options.

       reserved_word(Atom) -> bool()

	      Types  Atom = atom()

	      Returns true if Atom is an Erlang reserved word, otherwise false .

       token_info(Token) -> TokenInfo

	      Types  Token = token()
		     TokenInfo = [TokenInfoTuple]
		     TokenInfoTuple = {TokenItem, Info}
		     TokenItem = atom()
		     Info = term()

	      Returns a list containing information about the token Token  .  The  order  of  the
	      TokenInfoTuple s is not defined. The following TokenItem s are returned: category ,
	      column , length , line , symbol , and text . See token_info/2 for information about
	      specific TokenInfoTuple s.

	      Note  that  if token_info(Token, TokenItem) returns undefined for some TokenItem in
	      the list above, the item is not included in TokenInfo .

       token_info(Token, TokenItemSpec) -> TokenInfo

	      Types  Token = token()
		     TokenItemSpec = TokenItem | [TokenItem]
		     TokenInfo = TokenInfoTuple | undefined | [TokenInfoTuple]
		     TokenInfoTuple = {TokenItem, Info}
		     TokenItem = atom()
		     Info = term()

	      Returns a list containing information about the token Token . If TokenItemSpec is a
	      single  TokenItem  ,  the  returned  value is the corresponding TokenInfoTuple , or
	      undefined if the TokenItem has no value. If TokenItemSpec is a list of TokenItem	,
	      the  result is a list of TokenInfoTuple . The TokenInfoTuple s will appear with the
	      corresponding TokenItem s in the same order as the TokenItem s appeared in the list
	      of  TokenItem s. TokenItem s with no value are not included in the list of TokenIn-
	      foTuple .

	      The following TokenInfoTuple s with corresponding TokenItem s are valid:

		{category, category()} :
		  The category of the token.

		{column, column()} :
		  The column where the token begins.

		{length, integer() > 0} :
		  The length of the token's text.

		{line, line()} :
		  The line where the token begins.

		{location, location()} :
		  The line and column where the token begins, or just  the  line  if  the  column
		  unknown.

		{symbol, symbol()} :
		  The token's symbol.

		{text, string()} :
		  The token's text.

       attributes_info(Attributes) -> AttributesInfo

	      Types  Attributes = attributes()
		     AttributesInfo = [AttributeInfoTuple]
		     AttributeInfoTuple = {AttributeItem, Info}
		     AttributeItem = atom()
		     Info = term()

	      Returns  a  list containing information about the token attributes Attributes . The
	      order of the AttributeInfoTuple s is not defined. The following AttributeItem s are
	      returned: column , length , line , and text . See attributes_info/2 for information
	      about specific AttributeInfoTuple s.

	      Note that if  attributes_info(Token,  AttributeItem)  returns  undefined	for  some
	      AttributeItem in the list above, the item is not included in AttributesInfo .

       attributes_info(Attributes, AttributeItemSpec) -> AttributesInfo

	      Types  Attributes = attributes()
		     AttributeItemSpec = AttributeItem | [AttributeItem]
		     AttributesInfo = AttributeInfoTuple | undefined | [AttributeInfoTuple]
		     AttributeInfoTuple = {AttributeItem, Info}
		     AttributeItem = atom()
		     Info = term()

	      Returns  a  list	containing information about the token attributes Attributes . If
	      AttributeItemSpec is a single AttributeItem , the returned value is the correspond-
	      ing  AttributeInfoTuple  ,  or  undefined  if  the  AttributeItem  has no value. If
	      AttributeItemSpec is a list of AttributeItem , the result is a list of AttributeIn-
	      foTuple . The AttributeInfoTuple s will appear with the corresponding AttributeItem
	      s in the same order as the AttributeItem s appeared in the list of AttributeItem s.
	      AttributeItem s with no value are not included in the list of AttributeInfoTuple .

	      The following AttributeInfoTuple s with corresponding AttributeItem s are valid:

		{column, column()} :
		  The column where the token begins.

		{length, integer() > 0} :
		  The length of the token's text.

		{line, line()} :
		  The line where the token begins.

		{location, location()} :
		  The  line  and  column  where  the token begins, or just the line if the column
		  unknown.

		{text, string()} :
		  The token's text.

       set_attribute(AttributeItem, Attributes, SetAttributeFun) -> AttributesInfo

	      Types  AttributeItem = line
		     Attributes = attributes()
		     SetAttributeFun = set_attribute_fun()

	      Sets the value of the line attribute of the token attributes Attributes .

	      The SetAttributeFun is called with the value of  the  line  attribute,  and  is  to
	      return the new value of the line attribute.

       format_error(ErrorDescriptor) -> string()

	      Types  ErrorDescriptor = errordesc()

	      Takes an ErrorDescriptor and returns a string which describes the error or warning.
	      This function is usually called implicitly when processing an  ErrorInfo	structure
	      (see below).

ERROR INFORMATION
       The  ErrorInfo  mentioned above is the standard ErrorInfo structure which is returned from
       all IO modules. It has the following format:

       {ErrorLocation, Module, ErrorDescriptor}

       A string which describes the error is obtained with the following call:

       Module:format_error(ErrorDescriptor)

NOTES
       The continuation of the first call to the re-entrant input functions must be [] . Refer to
       Armstrong,  Virding  and  Williams,  'Concurrent Programming in Erlang', Chapter 13, for a
       complete description of how the re-entrant input scheme works.

SEE ALSO
       io(3erl) , erl_parse(3erl)

Ericsson AB				  stdlib 1.17.3 			   erl_scan(3erl)
Unix & Linux Commands & Man Pages : ©2000 - 2018 Unix and Linux Forums


All times are GMT -4. The time now is 09:53 AM.