Signed-off-by: erick-alcachofa <erick@artichoke.dev>
This commit refactors the AST printing functionality by moving the
human-readable `toString` implementation into its own file
(`lib/src/Parser/AST/toString.cpp`) and introducing a new `toDot`
function in `lib/src/Parser/AST/toDot.cpp` for generating Graphviz DOT
format output.
The `AST.hpp` header is updated to declare both the new `toDot` function
and the modified `toString` function, which now uses an optional
`prefix` parameter for prettier tree output. The `Token.hpp`/`Token.cpp`
files are also adjusted to have `toString(const TokenV &)` return a
`std::string_view`, and `toString(const Token &)` provides a cleaner
string representation using only the token's value.
Signed-off-by: erick-alcachofa <erick@artichoke.dev>
This commit lays the foundational groundwork for the artichoke language
parser by introducing the formal language grammar specification.
The tokenizer was updated to include new operators and keywords, also
added the posibility to handle comments.
Key Additions:
- Implemented support for C-style block comments (`/* ... */`),
including error handling for unclosed comments.
- Added all necessary tokens for missing keywords (e.g., `module`,
`export`, `using`, `match`, `loop`) and operators (e.g., `+=`, `:=`,
`.#`, `.*`, `.@`).
- The `Token` enum has been expanded to reflect the full language
feature set.
Documentation:
- Added `docs/grammar.ebnf` which contains the official, well-structured
EBNF grammar for the language.
- Added `docs/readme.md` providing a detailed technical overview of the
language's features, syntax, and semantics.
BREAKING CHANGE: The `kwVariant` and `kwMut` tokens have been removed to
align with the updated language design defined in the new grammar.
Initial version of Tokenizer and Token
Generator template for coroutines (used in tokenizer)
Utilities like string related functions, TrieMap, and error handling
TODO: Add tests for Tokenizer
TODO: Add tests for Generator
