Skip to content

Systems format lexer: unary minus only recognized before digit characters #410

@bpowers

Description

@bpowers

Description

In src/simlin-engine/src/systems/lexer.rs (around line 381), the unary-minus branch requires chars[i + 1].is_ascii_digit(), so expressions like -Recruiters or -(a + b) tokenize the - as Op(Sub) followed by a name/paren atom. Since parse_atom doesn't handle a leading Op(Sub), these expressions produce a parse error rather than being treated as negation.

Why it matters

This is a feature gap in the systems format parser that prevents valid SD model equations containing negated identifiers or sub-expressions from being parsed. Users writing equations like outflow = -stock_adjustment or net = -(inflow + outflow) will get parse errors. The impact is on correctness and developer experience when importing or authoring models in the systems format.

The good news is that this produces clear parse errors rather than silent misbehavior, so models won't silently compute wrong results.

Components affected

  • src/simlin-engine/src/systems/lexer.rs (unary minus tokenization)
  • src/simlin-engine/src/systems/parser.rs (parse_atom does not handle leading Op(Sub))

Possible approaches

  1. Extend the lexer: Recognize unary minus before identifiers (a-z, A-Z, _) and open parentheses, not just digits. This would emit a UnaryMinus or Neg token distinct from Op(Sub).

  2. Handle in the parser: Keep the lexer as-is and modify parse_atom to consume a leading Op(Sub) token, wrapping the following atom in a negation AST node. This is arguably cleaner since it keeps lexer logic simple and handles all cases uniformly.

The Python systems package may handle this differently and could serve as a reference.

Context

Identified during review of the systems format lexer/parser while working on the systems-format branch.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions