3 Commits

Author SHA1 Message Date
85a34bdd65
feat: Added Token, Tokenizer, Generator, and some utilities
Initial version of Tokenizer and Token
Generator template for coroutines (used in tokenizer)
Utilities like string related functions, TrieMap, and error handling

TODO: Add tests for Tokenizer
TODO: Add tests for Generator
2025-03-10 01:20:23 -06:00
0f4474821d
chore: Added CMake project setup
Added CMake files to set up project build, also added the file tree
structure of the project and clangd related settings
2025-03-04 12:50:53 -06:00
0f68b149da
chore: Initial commit
Initial commit for repo, added .gitignore, LICENSE, NOTICE and README.md
files
2025-03-01 01:27:46 -06:00