Dealing with existing Recognizers

A recognizer is a word to which you pass a string. If the recognizer recognizes the string, it typically returns some data and the xt of a word for processing the data; this word is called the translator. If the recognizer does not recognize the string, it returns the xt of notfound.

All recognizers have the stack effect ( c-addr u – ... xt ).

Recognizers take a string and return some data and a translator for interpreting that data. Gforth implements that translator as xt (executing it will perform the appropriate action to handle the token in the current state), but other Forth systems may implement it as actual table, with three xts inside. The first xt is the interpretation/run-time xt, it performs the interpretation semantics on the data (usually, this means it just leaves the data on the stack). The second xt performs the compilation semantics, it gets the data and the run-time semantics xt. The third xt perfoms the postpone semantics, it also gets the data and the run-time semantics xt. You can use >postpone to postpone the run-time xt.

Recognizers are organized as stack, so you can arrange the sequence of recognizers in the same way as the vocabulary stack. Recognizer stacks are themselves recognizers, i.e. they are executable, take a string and return a translator.

notfound ( state –  ) gforth-experimental “notfound”

If a recognizer fails, it returns notfound

rec-nt ( addr u – nt translate-nt | notfound  ) gforth-experimental “rec-nt”

recognize a name token

rec-num ( addr u – n/d table | notfound  ) gforth-experimental “rec-num”

converts a number to a single/double integer

rec-float ( addr u – r translate-float | notfound  ) gforth-experimental “rec-float”

recognize floating point numbers

rec-string ( addr u – addr u’ r:string | rectype-null  ) gforth-experimental “rec-string”

Convert strings enclosed in double quotes into string literals, escapes are treated as in S\".

rec-to ( addr u – xt n r:to | rectype-null  ) gforth-experimental “rec-to”

words prefixed with -> are treated as if preceeded by TO or IS, with +> as +TO, with '> as ADDR, and with @> as ACTION-OF.

rec-tick ( addr u – xt rectype-num | rectype-null  ) gforth-experimental “rec-tick”

words prefixed with ` return their xt. Example: `dup gives the xt of dup

rec-dtick ( addr u – nt rectype-num | rectype-null  ) gforth-experimental “rec-dtick”

words prefixed with `` return their nt. Example: ``S" gives the nt of S"

rec-body ( addr u – xt translate-tick | translate-null  ) gforth-experimental “rec-body”

words bracketed with '<' '>' return their body. Example: <dup> gives the body of dup

get-recognizers ( – xt1 .. xtn n  ) gforth-experimental “get-recognizers”

push the content on the recognizer stack

set-recognizers ( xt1 .. xtn n –  ) gforth-experimental “set-recognizers”

set the recognizer stack from content on the stack

recognize ( addr u rec-addr – ... rectype  ) gforth-experimental “recognize”

apply a recognizer stack to a string, delivering a token

recognizer-sequence: ( xt1 .. xtn n "name" –  ) gforth-experimental “recognizer-sequence:”

concatenate a stack of recognizers to one recognizer with the name "name". xtn is tried first, xt1 last, just like on the recognizer stack

forth-recognize ( c-addr u – ... translate-xt  ) recognizer “forth-recognize”

The system recognizer

forth-recognizer ( – xt  ) gforth-experimental “forth-recognizer”

backward compatible to Matthias Trute recognizer API. This construct turns a deferred word into a value-like word.

set-forth-recognize ( xt –  ) recognizer “set-forth-recognize”

Change the system recognizer

translate: ( int-xt comp-xt post-xt "name" –  ) gforth-experimental “translate:”

create a new recognizer table. Items are in order of STATE value, which are 0 or negative. Up to 7 slots are available for extensions.

translate-nt ( i*x nt – j*x  ) gforth-experimental “translate-nt”

translate a name token

translate-num ( x – | x  ) gforth-experimental “translate-num”

translate a number

translate-dnum ( dx – | dx  ) gforth-experimental “translate-dnum”

translate a double number


try-recognize ( addr u xt – results | false  ) gforth-experimental “try-recognize”

For nested recognizers: try to recognize addr u, and execute xt to check if the result is desired. If xt returns false, clean up all side effects of the recognizer, and return false. Otherwise return the results of the call to xt, of which the topmost is non-zero.

>interpret ( translator –  ) gforth-experimental “>interpret”

perform interpreter action of translator

>compile ( translator –  ) gforth-experimental “>compile”

perform compile action of translator

>postpone ( translator –  ) gforth-experimental “>postpone”

perform postpone action of translator

translate-method: ( "name" –  ) gforth-experimental “translate-method:”

create a new translate method, extending the translator table. You can assign an xt to an existing rectype by using xt rectype to translator.

translate-state ( xt –  ) gforth-experimental “translate-state”

change the current state of the system so that executing a translator matches the translate-method passsed as xt