Skip to content

Introduce GC benchmarks#488

Merged
eightbitraptor merged 4 commits intomainfrom
mvh-gc-harness
Mar 6, 2026
Merged

Introduce GC benchmarks#488
eightbitraptor merged 4 commits intomainfrom
mvh-gc-harness

Conversation

@eightbitraptor
Copy link
Contributor

This commit introduces a Ruby port of the Ellis-Kovac-Boehm GCbench benchmark. This Ruby port is based on the version translated by Noel Padavan and Chris Seaton as part of the initial Shopify work on MMTk.

In order to run this and reliably report on mark and sweep time seperately. This PR also introduces a new harness designed for running any benchmarks and reporting on GC statistics during the run.

Also introduced is a new benchmark category, gc. Benchmarks can be tagged with category: gc in benchmarks.yml. Running the benchmarks using run_benchmarks.rb --category=gc will run all benchmarks tagged with the gc category.

Output looks like the following:

interp: ruby 4.0.1 (2026-01-13 revision e04267a14b) +PRISM [arm64-darwin24]
yjit: ruby 4.0.1 (2026-01-13 revision e04267a14b) +YJIT +PRISM [arm64-darwin24]

-------  -------------  ----------------  -----------------  ------------  --------------  ---------------  ------------  -----------  ----------------  -----------------
bench      interp (ms)  interp mark (ms)  interp sweep (ms)     yjit (ms)  yjit mark (ms)  yjit sweep (ms)  yjit 1st itr  interp/yjit  mark interp/yjit  sweep interp/yjit
gcbench  1539.6 ± 1.5%        6.1 ± 8.8%        91.4 ± 1.3%  433.5 ± 1.3%      5.6 ± 8.7%      91.1 ± 1.9%         3.445  3.552 (***)       1.089              1.003
-------  -------------  ----------------  -----------------  ------------  --------------  ---------------  ------------  -----------  ----------------  -----------------

Legend:
- yjit 1st itr: ratio of interp/yjit time for the first benchmarking iteration.
- interp/yjit: ratio of interp/yjit time. Higher is better for yjit. Above 1 represents a speedup.
- mark interp/yjit: ratio of GC marking time. Higher is better for yjit. Above 1 represents faster marking.
- sweep interp/yjit: ratio of GC sweeping time. Higher is better for yjit. Above 1 represents faster sweeping.
- ***: p < 0.001, **: p < 0.01, *: p < 0.05 (Welch's t-test)

@eightbitraptor eightbitraptor merged commit bdd28b8 into main Mar 6, 2026
11 checks passed
@eightbitraptor eightbitraptor deleted the mvh-gc-harness branch March 6, 2026 22:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants