gnd(test): Add mock-based subgraph test runner by dimitrovmaksim · Pull Request #6361 · graphprotocol/graph-node

dimitrovmaksim · 2026-02-13T14:47:13Z

Summary

Implements a native test runner for gnd test that executes JSON-defined tests through real graph-node infrastructure (store, WASM runtime, trigger processing) with only the blockchain layer mocked. Provides a faster, integrated alternative to Matchstick while maintaining backward compatibility.

Key Features

JSON test format - Define blocks, events, eth_calls, and GraphQL assertions
Real graph-node pipeline - Tests run through actual indexing infrastructure
Auto block triggers - All block handler filters work (once, polling, no filter)
startBlock support - Auto-extracts from manifest and numbers blocks accordingly
eth_call mocking - Pre-populate call cache before indexing
Smart assertions - Order-insensitive comparison, type coercion, detailed diffs
Database isolation - Fresh pgtemp database per test
Backward compatible - --matchstick flag preserves legacy behavior

Architecture

JSON test → Parse → ABI encode events → StaticStreamBuilder (mock stream)
  → Real graph-node indexing (WASM + triggers + storage)
  → GraphQL assertions → Report results

Design decisions:

Reuses actual graph-node stores, runtime, and trigger processing
Only blockchain RPC is mocked (via StaticStreamBuilder)
Isolated database per test - Each test gets a pgtemp database dropped on completion (default), or a shared persistent database with post-test cleanup (--postgres-url)
Pre-populated eth_call cache for deterministic results

Usage

gnd test                    # Run all tests in tests/
gnd test tests/foo.json     # Run specific test
gnd test --skip-build       # Skip auto-build
gnd test --matchstick       # Legacy Matchstick mode
gnd test --help             # To see usage and all options

Test Format Example

{
  "name": "Transfer creates entity",
  "blocks": [
    {
      "number": 1,
      "events": [
        {
          "address": "0x1234...",
          "event": "Transfer(address indexed from, address indexed to, uint256 value)",
          "params": {"from": "0xaaaa...", "to": "0xbbbb...", "value": "1000"}
        }
      ],
      "ethCalls": [
        {
          "address": "0x1234...",
          "function": "balanceOf(address)(uint256)",
          "params": ["0xaaaa..."],
          "returns": ["1000000000000000000"]
        }
      ]
    }
  ],
  "assertions": [
    {
      "query": "{ transfer(id: \"1\") { from to value } }",
      "expected": {"transfer": {"from": "0xaaaa...", "to": "0xbbbb...", "value": "1000"}}
    }
  ]
}

Implementation

Modified Graph-Node Files

graph/src/data_source/subgraph.rs - Minor exports for startBlock support

Supported Features

Feature	Status
Log events	✅ Supported
Block handlers (`once`, `polling`, no filter)	✅ Supported
eth_call mocking	✅ Supported
Dynamic/template data sources	✅ Supported
Transaction receipts	⚠️ Not implemented
File data sources / IPFS	⚠️ Not implemented
Call triggers & Block Handler (`call`)	❌ TBD if will be supported

Breaking Changes

None. Adds native testing while preserving --matchstick for backward compatibility.

Code Attribution

⚠️ Almost all implementation code was generated by Claude Code through investigation of graph-node's indexing pipeline, existing test infrastructure, and iterative design refinement.

Human contributions: Initial requirements, code review, validation, and bug fixes.

Checklist

Implementation complete
User documentation (776-line README)
Backward compatible via --matchstick
Validated with real subgraphs
Support for BlockHandler filters (except call)
startBlock handling working
No breaking changes

lutter

Very nice! I think this is a really useful tool

lutter · 2026-02-25T23:52:42Z

gnd/README.md

-| `--recompile` | `-r` | Force recompilation before testing |
-| `--version` | `-v` | Matchstick version to use |
+| `--manifest` | `-m` | Path to subgraph manifest (default: `subgraph.yaml`) |
+| `--skip-build` | | Skip building the subgraph before testing |


I would think that people will use this a lot when developing tests; maybe it should have a short option, too?

lutter · 2026-02-25T23:53:30Z

gnd/README.md

+| `--docker` | `-d` | Run Matchstick in Docker (requires `--matchstick`) |
+| `--coverage` | `-c` | Run with coverage reporting (requires `--matchstick`) |
+| `--recompile` | `-r` | Force recompilation (requires `--matchstick`) |
+| `--force` | `-f` | Force redownload of Matchstick binary (requires `--matchstick`) |


It would be nice to communicate somehow that we want to get rid of Matchstick and give people a bit of time to migrate their tests - it really feels like nobody is using it.

I can add a warn notification when using matchstick, also add a note about that in the docs.

lutter · 2026-02-25T23:59:13Z

gnd/tests/gnd_test.rs

+}
+
+/// Assert that `asc` (AssemblyScript compiler) is available in PATH or in local node_modules.
+fn verify_asc_available(subgraph_dir: &Path) {


gnd/src/compiler/asc.rs has is_asc_available and get_asc_version. Maybe they can be reused here?

It seems I'm overwriting is_asc_available in my PR with find_asc_binary

Ah yes, I remember now. is_asc_available was looking onl for a globally installed asc. I added a lookup into the local node_modules as well.

lutter · 2026-02-25T23:59:59Z

gnd/tests/gnd_test.rs

+    copy_dir_recursive(&fixture, &subgraph_dir).expect("Failed to copy fixture to temp directory");
+
+    // Install npm dependencies (graph-ts, graph-cli)
+    let npm_output = Command::new("npm")


In other places like integration tests, we already require pnpm, maybe we should do that here, too

lutter · 2026-02-26T00:13:45Z

gnd/src/commands/test/runner.rs

+        )
+        .await
+        .ok();
+    }


At least for failed tests, it might be nicer to not clean up afterwards and leave the data in place to make it easier to troubleshoot what went wrong

This works for persisted DBs. For PgTemp, it is doable, it could be persisted, but then on success needs to be manually deleted, so it kind of becomes an inverted logic.

lutter · 2026-02-26T00:19:39Z

gnd/src/commands/test/assertion.rs

+/// in a different order — every line shows as changed even if only one field
+/// differs. This function reorders `actual` so that elements are paired with
+/// their closest match in `expected`, producing a diff that highlights only
+/// real value differences.


lutter · 2026-02-26T00:21:18Z

gnd/src/commands/test/assertion.rs

+/// Convert graph-node's internal `r::Value` (GraphQL result) to `serde_json::Value`.
+///
+/// Graph-node uses its own value type for GraphQL results. This converts to
+/// standard JSON for comparison with the expected values in the test file.


r::Value implements Serialize, so this should not be needed, and by using the existing implementation, we catch any quirks that that serialization might have.

lutter · 2026-02-26T00:30:02Z

gnd/docs/gnd-test.md

+
+### Event Signature Format
+
+**Important:** Include `indexed` keywords in the signature:


That's the exact same form in which the event must be listed in the manifest, right? I wonder if there is a way to make the notation more concise, maybe by allowing the use of <ABI>.<Event>, i.e. something like ERC20.Transfer in addition to the exact event signature. But definitely not something that needs to go into this PR

Yeah, it is needed to know if the params should be set as topics (and potentially padded to 32 bytes or keccak hashed if dynamically sized) in case they are indexed. But this should be doable, scan the abi, look at the event definition and encode accordignly. This will be a good QOL improvement.

Replaces monolithic gnd/src/commands/test.rs with organized test/ directory containing: - mod.rs: Main entry point and test orchestration - runner.rs: Test execution and infrastructure setup - assertion.rs: GraphQL assertion logic - block_stream.rs: Mock block stream implementation - noop.rs: Stub trait implementations - schema.rs: JSON schema and test types - trigger.rs: ABI encoding for test triggers - output.rs: Test result formatting - mock_chain.rs: Block pointer helpers Updates main.rs to make Test command async (.await). Adds dependencies for test runner (graph-chain-ethereum, graph-graphql, graph-store-postgres).

Adds supporting modules for test infrastructure: - mock_chain: Helpers for block pointer construction - schema: JSON schema types and parsing - output: Console output formatting - trigger: ABI encoding of test triggers

Adds module declarations for refactored components: - mod assertion - mod block_stream - mod noop Updates module documentation to reflect the new structure and improved separation of concerns.

Removes ~500 lines from runner.rs by delegating to new focused modules: - block_stream: Mock block delivery infrastructure - noop: Stub trait implementations - assertion: GraphQL assertion logic runner.rs now focuses exclusively on test orchestration: - setup_stores: Initialize PostgreSQL and chain store - setup_chain: Construct mock Ethereum chain - setup_context: Wire up graph-node components - wait_for_sync: Poll store until indexing completes Reduced from 1198 to 729 lines (39% reduction). Improves readability by separating concerns.

Moves assertion execution to gnd/src/commands/test/assertion.rs: - run_assertions: Execute all test assertions - run_single_assertion: Execute and compare a single query - r_value_to_json: Convert graph-node's r::Value to serde_json - json_equal: Compare JSON with string-vs-number coercion Makes TestContext fields pub(super) to allow assertion module access.

Moves unused adapter stubs to gnd/src/commands/test/noop.rs: - StaticBlockRefetcher - NoopRuntimeAdapter / NoopRuntimeAdapterBuilder - NoopAdapterSelector - NoopTriggersAdapter These satisfy Chain constructor trait bounds but are never called during normal test execution since triggers are pre-built and host functions are not available in mocks.

- Add baseFeePerGas field to TestBlock schema - Parse and apply base fee when creating test blocks - Replace graph-node helper functions with direct alloy types - Extract dummy_transaction creation into dedicated function - Use alloy Block::empty() constructor for cleaner block creation

- Rename 'triggers' field to 'events' in TestBlock - Remove TestTrigger enum and BlockTrigger type - Keep LogEvent as the only event type users specify - Auto-inject Start and End block triggers for every block - This ensures block handlers fire correctly without explicit config - Update docs to reflect that block triggers are automatic

- Extract min startBlock from manifest in extract_start_block_from_manifest() - Use startBlock as default test block numbering base - Create start_block_override to bypass on-chain validation - Pass override through setup_context() to SubgraphRegistrar - This allows testing subgraphs that specify startBlock without needing a real chain

Signed-off-by: Maksim Dimitrov <dimitrov.maksim@gmail.com>

- Replace positional `manifest` arg with `--manifest` / `-m` flag (default: subgraph.yaml) - Add positional `tests` args accepting file or directory - When no args given, default to scanning `tests/` - Bare filenames resolve to `tests/<filename>` for convenience (e.g., `gnd test foo.json` → `tests/foo.json`) - Remove `--test-dir` flag (replaced by positional args) - Update README with new usage examples Signed-off-by: Maksim Dimitrov <dimitrov.maksim@gmail.com>

…ersion - Add Array, FixedArray, Tuple branches to json_to_sol_value - Fix strip_prefix("0x") usage (trim_start_matches incorrectly strips repeated 0s) - Fix i64::MIN two's complement handling via I256::into_raw() - Add block number overflow check against i32::MAX

- discover_test_files now walks subdirectories recursively - Skip entries starting with non-alphanumeric chars (.hidden, _fixture)

Add upfront arity checks in encode_function_call and encode_return_value so mismatches produce clear errors instead of silently truncating. Also simplify redundant .with_context() wrappers in populate_single_call.

- Remove FilterStoreEventEndedDrain log filter and unused logger field - Always call stop_subgraph after test, even on error - Warn when a test has blocks but no assertions - Add block number overflow check against i32::MAX

…ker mode Add alphanumeric/hyphen/underscore validation before interpolating the datasource name into Docker's sh -c command. Also simplify redundant .with_context() wrappers.

- block_stream: .unwrap() -> .expect() on mutex lock - noop: unimplemented!() -> Err(anyhow!(...)) - mod: Fail early with bail! on missing test file

Signed-off-by: Maksim Dimitrov <dimitrov.maksim@gmail.com>

Add `-v` / `--verbose` flag with count semantics for controlling graph-node log verbosity during tests (-v=info, -vv=debug, -vvv=trace). GRAPH_LOG env var always takes precedence when set. Extract manifest loading into `ManifestInfo` struct loaded once per run, avoiding redundant parsing across tests. Thread a single logger through setup_stores/setup_chain instead of creating ad-hoc loggers.

Add gnd_test integration test suite with fixture subgraph and test cases for blocks, transfers, templates, and expected failures. Fix pgtemp Unix socket path overflow on macOS by overriding unix_socket_directories to /tmp. Reduce default pool_size to 2.

Makes the two database lifecycle paths explicit and self-documenting. `TestDatabase::Temporary` vs `TestDatabase::Persistent` (--postgres-url, needs cleanup) replaces the opaque `Option<TempPgHandle>`. Cleanup in `setup_stores` is now gated on `db.needs_cleanup()` instead of running unconditionally.

When using --postgres-url, cleanup only ran at the start of each test (to remove stale state from a previous run). The last test's deployment was left in the DB, which broke unrelated unit test suites calling remove_all_subgraphs_for_test_use_only() — they don't set GRAPH_NODE_DISABLE_DEPLOYMENT_HASH_VALIDATION, so parsing the file-path deployment hash fails. Add post-test cleanup for persistent databases, mirroring the pre-test cleanup. Pre-test handles interrupted runs; post-test handles the normal case. Together they keep the DB clean regardless of how the run ends.

Each test run now computes a fake-but-valid DeploymentHash as "Qm" + hex(sha1(manifest_path + seed)) where seed is the Unix epoch in milliseconds. This: - Passes DeploymentHash validation without bypassing it - Produces a unique hash and subgraph name per run, so sequential runs never conflict in the store - Removes the pre-test cleanup (it would never match a fresh hash) - Registers the hash as a FileLinkResolver alias so clone_for_manifest can resolve it to the real manifest path - Reuses the existing sha1 dep — no new dependencies

…efault - Flatten TestResult from enum to struct (handler_error + assertions fields) - Remove AssertionOutcome/TestResult accessor methods; use public fields directly - Remove baseFeePerGas from test block format (YAGNI) - Change default block timestamp from number * 12 to number (chain-agnostic, avoids future timestamps on high-block-number chains like Arbitrum)

Update the contract address in the fixture subgraph manifest and transfer.json test from the old placeholder address to the correct one used by the test assertions.

dummy_transaction was returning Transaction<TxEnvelope> instead of the required AnyTransaction (Transaction<AnyTxEnvelope>). Wrap the envelope in AnyTxEnvelope::Ethereum(...) to match the expected type. Also wrap ConsensusHeader in AnyHeader::from(...) when constructing the block Header, producing Header<AnyHeader> as required by LightEthereumBlock::new.

- Add -s short flag for --skip-build - Print deprecation warning when --matchstick is used; update README - Replace npm with pnpm in integration test setup - Replace hand-rolled r_value_to_json with serde_json::to_value via r::Value's built-in Serialize impl (also fixes Timestamp serialization) - Make find_asc_binary pub and re-export from compiler module; reuse it in gnd_test.rs instead of duplicating the lookup logic - Skip DB cleanup on test failure for persistent databases and print the connection URL so the data can be inspected

graph-node returns all hex-encoded values (addresses, tx hashes, bytes) in lowercase. Add a note in the Assertions section's comparison behavior table and in Tips & Best Practices. Event inputs are normalised automatically and can be mixed case. Also fix the transfer.json fixture assertion to use lowercase token ID.

dimitrovmaksim force-pushed the feat/gnd-cli-test-rewrite branch 7 times, most recently from 6c6f19f to 117e2bb Compare February 17, 2026 19:41

dimitrovmaksim marked this pull request as ready for review February 18, 2026 20:54

dimitrovmaksim self-assigned this Feb 18, 2026

lutter self-requested a review February 25, 2026 23:50

lutter approved these changes Feb 26, 2026

View reviewed changes

dimitrovmaksim added 19 commits February 26, 2026 21:13

gnd(test): Add remaining test module files

25e5f5b

Adds supporting modules for test infrastructure: - mock_chain: Helpers for block pointer construction - schema: JSON schema types and parsing - output: Console output formatting - trigger: ABI encoding of test triggers

gnd(test): Update module structure with new submodules

48cc6a8

Adds module declarations for refactored components: - mod assertion - mod block_stream - mod noop Updates module documentation to reflect the new structure and improved separation of concerns.

gnd(test): Add eth_call mocking and refactor test output

3099684

gnd(test): Fix issues after rebase

85fc735

Signed-off-by: Maksim Dimitrov <dimitrov.maksim@gmail.com>

gnd(test): Fix matchstick path

05ab2c5

Signed-off-by: Maksim Dimitrov <dimitrov.maksim@gmail.com>

gnd(test): Reuse load_manifest()

3512429

Signed-off-by: Maksim Dimitrov <dimitrov.maksim@gmail.com>

gnd(test): Add README, add important inline notes

448aadb

Signed-off-by: Maksim Dimitrov <dimitrov.maksim@gmail.com>

gnd: Fix missing fields, update readme

cd3992d

Signed-off-by: Maksim Dimitrov <dimitrov.maksim@gmail.com>

gnd(test): Fix diff colors — red for expected, green for actual

0ddc69c

gnd(test): Recurse into subdirectories when discovering test files

ccbbed7

- discover_test_files now walks subdirectories recursively - Skip entries starting with non-alphanumeric chars (.hidden, _fixture)

dimitrovmaksim added 17 commits February 26, 2026 21:13

gnd(test): Validate eth_call parameter and return value counts

83fed7a

Add upfront arity checks in encode_function_call and encode_return_value so mismatches produce clear errors instead of silently truncating. Also simplify redundant .with_context() wrappers in populate_single_call.

gnd(test): Validate datasource name to prevent shell injection in Doc…

8cfb64d

…ker mode Add alphanumeric/hyphen/underscore validation before interpolating the datasource name into Docker's sh -c command. Also simplify redundant .with_context() wrappers.

gnd(test): Replace unwrap/unimplemented with proper error handling

fc66da3

- block_stream: .unwrap() -> .expect() on mutex lock - noop: unimplemented!() -> Err(anyhow!(...)) - mod: Fail early with bail! on missing test file

gnd(test): Fix ServerAddress initialization

54eab56

Signed-off-by: Maksim Dimitrov <dimitrov.maksim@gmail.com>

gnd(compiler): Look for asc binary in local if global does not exist.

dd9e18f

gnd(test): Use --postgres-url for tests

007aedb

gnd(test): Fix test cleanup for --postgres-url

6e1c829

gnd(test): Fix stale docs after pre-test cleanup removal

0c4c50a

gnd(test): Rename and move README into docs/

9bc4397

gnd(test): Move docs to correct location under gnd/docs/

9944093

dimitrovmaksim force-pushed the feat/gnd-cli-test-rewrite branch from 9fe6d41 to bad0c9c Compare February 26, 2026 20:37

dimitrovmaksim added 4 commits February 26, 2026 22:47

gnd(test): Fix fixture contract address

4c6c033

Update the contract address in the fixture subgraph manifest and transfer.json test from the old placeholder address to the correct one used by the test assertions.

dimitrovmaksim force-pushed the feat/gnd-cli-test-rewrite branch from bad0c9c to 6a8fae9 Compare February 26, 2026 20:48

dimitrovmaksim merged commit 2d7323a into graphprotocol:master Feb 26, 2026
6 checks passed


		### Event Signature Format

		Important: Include `indexed` keywords in the signature:

Conversation

dimitrovmaksim commented Feb 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Key Features

Architecture

Usage

Test Format Example

Implementation

Modified Graph-Node Files

Supported Features

Breaking Changes

Code Attribution

Checklist

Uh oh!

lutter left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimitrovmaksim Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimitrovmaksim Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimitrovmaksim Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dimitrovmaksim commented Feb 13, 2026 •

edited

Loading

dimitrovmaksim Feb 26, 2026 •

edited

Loading

dimitrovmaksim Feb 26, 2026 •

edited

Loading

dimitrovmaksim Feb 26, 2026 •

edited

Loading