erick-alcachofa 146a48aef9
fix(tokenizer): harden move semantics and operator parsing
Signed-off-by: erick-alcachofa <erick@artichoke.dev>

This commit addresses subtle state-management issues uncovered while
exercising tokenizer move semantics and operator parsing. Moved-from
instances were retaining stale iterators and metadata that could
manifest as incorrect token offsets when the objects were reused.
Additionally, token location bookkeeping during comment skipping and
operator backtracking drifted from the iterator, producing inaccurate
column reports.

- Refines the move constructor by exchanging `line`, `column`, `iter`,
  `tokensGenerator`, `tokensBuffer`, and `source`, ensuring moved-from
  tokenizers reset to neutral defaults without dangling references.
- Aligns the move assignment operator with the constructor by using
  `std::exchange` across all transferred members, preventing stale state
  when a tokenizer is reassigned.
- Adjusts column advancement for single-line comments so the cursor
  increments by one when skipping the comments sequence, matching the
  iterator progression.
- Promotes the operator trie to a function-local static cache, avoiding
  repeated construction each time an operator token is read.
- Restores accurate token metadata by assigning the original line and
  column (`cLine`, `cColumn`) when emitting operator tokens and rolling
  back the column alongside the iterator during longest-valid
  backtracking.
2026-01-01 07:55:09 +00:00
2025-03-04 12:50:53 -06:00
2025-05-10 21:07:49 -06:00
2025-03-01 01:27:46 -06:00
2025-03-01 01:27:46 -06:00

⚠️ WIP Highly Experimental Project

  • The language, compiler, and tools are under active development and may be very unstable.
  • There will likely be breaking changes and periods where no work is done on the project.
  • Expect breaking changes as the compiler progresses through semantic analysis and code generation.

The artichoke Programming Language

artichoke is a modern, statically-typed programming language designed to satisfy my personal preferences and requirements for programming, combining the low-level control and powerful modern features like a robust type system, generics, integrated error handling, and a clean, ergonomic syntax.

The goal of artichoke is to provide a language that is simple, safe, and productive for programming, eliminating common pitfalls without sacrificing performance or control.

For a detailed guide to the language, grammar specification, and syntax features, please see the project wiki

Core Philosophy & Features

artichoke is built around a few core principles to create a safer, more productive programming experience:

  • Explicitness: Type conversions and error handling are explicit.
  • Safety: Non-nullable pointers, a robust type system, and deterministic resource management are prioritized.
  • Unambiguous Design: A grammar designed for fast, single-pass parsing and clear error reporting.
  • Modern Ergonomics: Features like generics, defer, and a clean module system reduce boilerplate and improve readability.

The language includes a powerful generic type system, first-class error handling, a full suite of control flow statements (including match), a true module system, and compile-time reflection.

Project Status

artichoke is currently in the early implementation phase. The front-end infrastructure is not yet defined but contains a simple program for printing and visualizing the generated AST, development has shifted now toward semantic validation.

  • Lexical Analysis: Full tokenizer implementation.
  • Syntactic Analysis: Handwritten Recursive Descent + Pratt Expression Parser.
  • AST Infrastructure: Complete Abstract Syntax Tree with Graphviz and String-Graph based visualization support.
  • Semantic Analysis (In Progress): Multi-pass symbol table generation and type checking.
  • Backend: Code generation and optimization.

Building from Source

# Get the source code
git clone https://git.artichoke.dev/me/artichoke-lang.git

# Configure cmake
# Optionally add -DENABLE_TESTING=ON for building tests
cmake -DCMAKE_BUILD_TYPE=Release -S . -B build

# Build the project
cmake --build build

# Run the compiler frontend binary
./build/frontend/artichoke-c <input_file>

# Run the tests if enabled
ctest --test-dir build/tests --output-on-failure

# Install library and frontend binary if wanted
cmake --install build --prefix=/usr/local

# Run the installed binary
arti-c

Contributing

The artichoke project is hosted on a personal, self-hosted Gitea instance. If you are interested in contributing, you have two options:

  1. Request an Account: Please contact support@artichoke.dev to request an account on the Gitea instance.
  2. Submit Patches: Alternatively, you can send patches or diffs directly to the same email address.

In all cases, proper attribution will be given for your contributions in the source files and/or the project wiki.

License

This project is licensed under the GNU Affero General Public License v3.0. The full license text can be found in the LICENSE file in this repository.

Description
artichoke programming language
https://lang.artichoke.dev/
Readme GNU-AGPLv3 672 KiB
Languages
C++ 98.4%
CMake 1.3%
Shell 0.3%