Token Reference
Token definitions used by the tokenizer.
Literals
tkInteger - decimal integers (10, 42).tkDecimal - floating-point literals (3.14159265358979).tkString - double-quoted strings.tkCharacter - character literals.tkIdentifier - identifiers and names.tkEOF - end-of-file sentinel.
Operators
Includes all operator tokens such as ::, ->, .[, .#, .*, .@, and the
compound assignments (+=, etc.).
Keywords
import, using, struct, enum, fn, return, let, def, if, else,
for, while, do, loop, match, switch, defer, errdefer, true,
false, null, typename, this, _ (default pattern), and logical aliases
or, and.
Notes
- Keyword recognition uses a trie to ensure single-pass tokenization.
- Multi-character operators (
::, .[, .#, .*, .@, >>=) rely on
lookahead logic.
- Generic parsing may split
>> into > + > to disambiguate template closers.