3 Commits

Author SHA1 Message Date
b99f3586dc
chore(license): Added NOTICE header to all source files
Signed-off-by: erick-alcachofa <erick@artichoke.dev>
2025-12-25 13:12:41 -06:00
f9051e1c21
fix: Minor fixes
Fixed some minor mistakes (wrong messages/errors) due to copy/pasting
code.

Fixed that digits weren't allowed in identifiers before.

Also minor improvements in some functions/code parts.
2025-06-30 00:31:10 -06:00
85a34bdd65
feat: Added Token, Tokenizer, Generator, and some utilities
Initial version of Tokenizer and Token
Generator template for coroutines (used in tokenizer)
Utilities like string related functions, TrieMap, and error handling

TODO: Add tests for Tokenizer
TODO: Add tests for Generator
2025-03-10 01:20:23 -06:00