scannertool documentation
- Usage
scannertool filename [outputfile]
Where filename is the name of the input file, and outputfile is name of the output file. If no output file is named, the name "scanner.l" will be used.
The scanner produced will be in the format of flex. This file can then be converted to C by flex and compiled by gcc.
Input file format:
The format of the input file file is exactly like flex (see man flex), with following extentions:
TOKEN(name, precedence)
BEGIN_BLOCK(name, precedence, nesting [, token_name])
END_BLOCK(name [, token_name])
These can only be used in flex actions, they are described in more detail below.
- Tokens
Tokens are segments of text that recongnized by scanner. Tokens are used to partition text, and to recognize special parts of a language. In gtkeditor we use tokens primarily to highlight parts of a text file. You can specify tokens to gtkeditor in the folowing manner:
TOKEN(name, precedence)
This function specifies a token to the editor. The name can later be referenced in highlighting functions and plugins. Precedence allows this pattern to be highlighted inside a block, if the token is in a block has a higher precedence than the block it is conatined in than it will be highlighted it's color and not the blocks color.
- Example
In this example we will recognize some of the C keywords.
(if|else|for|while|do) TOKEN("keyword", 0);
- Blocks
Blocks are used to highlight sections of text of the form:
begin_pattern ... end_patern
That is everything between and including begin_pattern and end_pattern will be highlighted the same color.
BEGIN_BLOCK(name, precedence, nesting [, token_name])
This function tells the editor to start highlighting as a block. This means that all text that comes after this will be highlighted the color of the block until END_BLOCK is called or the end of the file is reached. There is one exception to this however, a pattern with higher precedence, be it a nested block or a token, will be highlighted acording to the highlighting data asociated with these patterns.
Precedence, is similar to precedence for tokens. If a block has greater or equal precedence to the block it is inside, it will be highlighted using it's own properties. Otherwise it gets the properties of the block that contains it.
Nesting, specifys if the block allows nesting or not. If true other blocks can nested inside the block otherwise nesting is not allowed.
The token_name is an optional paramerter, this allows you to ascociate a token with the pattern that begins a block. It is useful in the case you want highlight the begining of the block diferently from the inside of the block.
END_BLOCK(name [, token_name])
End block terminates highlighting of the block "name". As with BEGIN_BLOCK, you can also specify a token in combination with the end of block, in case you want to highlight the end of the block diferently from the inside of the block.
- Example
Here is example of C comments implemented using blocks.
"/" BEGIN_BLOCK("comment", 0, FALSE);
"/" END_BLOCK("comment");
Things to avoid using in Flex:
One such feature to avoid is the flex stack. Since editing can take place
anywhere in the buffer, there is no way of insuring the stack is properly set
up without keeping track of the flex stack in the editor. It is posiible in the
future we might support the flex stack, however right now stick to the BEGIN
macro to achieve stack like behavior.