17.5.2Class Parser

Main parsing class.

Class Parser

A generic parser is a parsing system based on token recognition and callback invocation. This class provides general token recognition and calls adequate callbacks.

In strick terms, this class acts more like a lexer which drives a callback-driven parser provided by the user.

The Parser class works by using a set of named states (in the class PState), which corespond to different ruleset which are used in different parsing steps.

Rules, that is, user-provided callbacks, can return a string representing a state name; the coresponding satate is then pushed on a stack, and can be popped later by returning the special state "#pop".

Each PState provides a set of four elements:

• separators: a string containing characters considered "blanks" which hasn't any grammar meaning except that of separating tokens and keywords. In example, " t" would recognize spaces and tab characters as separators.
• tokens: A list of Rule instances which match recognized tokens. Tokens are recognized when it can be demonstrated that no other token can be matched. In example, if the parser has "+" and "+=" tokens, the proper action (either the one associated with + or +=) will be called when the ambiguity can be resolved.
• keywords: A list of Rule instances which are recognized only if surrounded by separators or tokens. In example, a keyword "fun" would be recognized only if preceded and followed by any of the characters in the seperators string, or by any recognized token.
• onElement: it's a callback which gets called when identifying a sequence between separators and/or tokens which is not listed in the keywords.

In example, the following parser regognizes and saves strings by pushing a different state when they are found:

   load parser

   // base state
   base = PState(
      separators|" \t\n\r",
      tokens| [ Rule( '"', onQuote ) ],
      onElement|ignoring )

   // state when parsing a string
   pstring = PState(
      separators|"", // spaces are relevant
      tokens|[
         Rule( "\\\"", onEscapeQuote ),
         Rule( "\"", onStringClosed ) ],
      onElement| addToString )

   content = ""
   function onQuote( qt )
      global content
      // starting string mode
      content = ""
      return "string"
   end

   function ignoring( item )
      > "Ignoring ", item
   end

   function addToString( data )
      global content
      content += data
   end

   function onEscapeQuote( qt )
      global content
      content += "\""
   end

   function onStringClosed( qt )
      global content
      > "Found string: >>", content, "<<"
      return "#pop"
   end

   parser = Parser()
   parser.addState( "base", base )
   parser.addState( "string", pstring )

   stream = StringStream( 'unrecognized text, "an \"interesting\" string", other text')
   parser.parse( stream, "base" )

The parser ensures that all the tokens, keywords and unrecognized elements are notified via callbacks in the same order in which they are found in the parsed file.

Other than returning a state name, the rule callbacks may also return a new instance of PState; this would push the state on the stack and make it the currently active state. This makes possible to create new parser states on the fly, in case some of the parsing conditions are not known in advance.

It is also possible to modify existing states by i.e. adding new keywords or tokens as they become defined while parsing the source code.

Properties
row
states
trace
Methods
initParser
parse	Performs complete parsing.
parseLine
parseStream
popState	Pops current state.
pushState	Pushes a given state making it the active state.
terminate	Request parsing termination.
traceMsg
traceRule
traceStates
traceText

Properties

row

states

trace

Methods

initParser

initParser( initState )

parse

Performs complete parsing.

parse( stream, initState, string, [initRow] )

stream	An input stream where the input data is read.
initState	The name of the initial state.
initRow	If given, set the initial row to this number
Raise	ParseError if the parser has not been correctly initialized.

parseLine

parseLine( line, ctx )

parseStream

parseStream( stream, initState )

popState

Pops current state.

popState()

Raise

ParseError if the state backlist is empty.

pushState

Pushes a given state making it the active state.

pushState( name )

name	The name of the state that should be pushed.
Raise	ParseError if the state name is not known.

terminate

Request parsing termination.

terminate()

A rule callback may call this method via sender.terminate() to request immediate exit from the parser sequence.

A rule may also quit the parser by returning the special "#quit" state.

traceMsg

traceMsg( msg )

traceRule

traceRule( r, context )

traceStates

traceStates()

traceText

traceText( line )