fix: delete orphaned digest directories when last tag is removed#111
Open
hiroTamada wants to merge 4 commits intomainfrom
Open
fix: delete orphaned digest directories when last tag is removed#111hiroTamada wants to merge 4 commits intomainfrom
hiroTamada wants to merge 4 commits intomainfrom
Conversation
Previously, DeleteImage only removed the tag symlink, leaving the digest directory (containing the erofs rootfs) on disk indefinitely. This caused orphaned digests to accumulate over time. Now DeleteImage checks if the digest is orphaned after removing the tag, and deletes the digest directory if no other tags reference it. This is eager GC - cleanup happens immediately at delete time. Made-with: Cursor
added 2 commits
February 27, 2026 11:51
The CI workflow runs `make oapi-generate` which uses `oapi-codegen@latest`. The latest generator outputs code using `runtime.StyleParamWithOptions` which was added in runtime v1.2.0, but go.mod had v1.1.2 pinned. This caused CI failures on main starting with commit 08958b8. Made-with: Cursor
The CI workflow regenerates oapi.go using `oapi-codegen@latest`. When v2.6.0 was released, it changed struct tag generation (adding omitempty to some fields), causing type mismatches with code in instances.go that uses inline struct literals. Pinning to v2.5.1 ensures CI regenerates the same code that's committed. Made-with: Cursor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
1. Race condition (medium severity): Hold createMu during the orphan check and delete sequence to prevent a concurrent CreateImage from creating a tag pointing to the same digest between count and delete. 2. Silent errors (low severity): Actually log errors when countTagsForDigest or deleteDigest fails, instead of silently swallowing them. Made-with: Cursor
sjmiller609
approved these changes
Feb 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
DeleteImageremoves the last tag referencing a digest, the digest directory is now automatically deletedBackground
Previously,
DeleteImageonly removed the tag symlink, leaving digest directories (containing erofs rootfs files) on disk indefinitely. This was documented in the README as "Old digests remain until explicitly garbage collected" but no GC was ever implemented.At current volume (~20 GB/month), the 3.5 TB disk on
prod-iad-hypeman-1has runway, but this becomes a concern at higher deployment volumes post-rollout.Changes
storage.go: AddedcountTagsForDigest()anddeleteDigest()helper functionsmanager.go: UpdatedDeleteImage()to check for orphaned digests and delete themmanager_test.go: Updated existing test + addedTestDeleteImagePreservesSharedDigestREADME.md: Updated documentation to reflect new behaviorCI fixes (pre-existing issues on main)
go.mod: Updatedoapi-codegen/runtimev1.1.2 → v1.2.0 (fixes missingStyleParamWithOptions)Makefile: Pinnedoapi-codegento v2.5.1 (v2.6.0 generates incompatible struct tags)Test plan
TestDeleteImage- verifies digest is deleted when orphanedTestDeleteImagePreservesSharedDigest- verifies digest is preserved when other tags reference itTestDeleteImageNotFound- unchanged, still passesNote on CI fixes
This PR includes fixes for two pre-existing CI issues unrelated to the GC changes:
Runtime version: CI regenerates oapi.go with
oapi-codegen@latest, which usesruntime.StyleParamWithOptionsadded in v1.2.0, but go.mod had v1.1.2.Generator version: oapi-codegen v2.6.0 changed struct tag generation (adds
omitemptyto some fields), causing type mismatches. Pinned to v2.5.1 to match committed code.Both issues caused CI failures starting with commit 08958b8 on main.
Note
Medium Risk
Changes deletion semantics for on-disk image storage by removing digest directories automatically, with added locking to avoid races; bugs here could delete still-referenced data or leave dangling tags under concurrency.
Overview
Image deletion now performs eager cleanup.
DeleteImageresolves the tag’s digest, removes the tag symlink, then (undercreateMu) counts remaining tags pointing at that digest and deletes the digest directory when it becomes orphaned.Storage helpers + tests/documentation updated. Adds
countTagsForDigest/deleteDigest, updates tests to assert digest dir removal and adds a shared-digest preservation test, and updates the images README to reflect the new GC behavior.Build/CI stability tweaks. Pins
oapi-codegenin theMakefileand bumpsgithub.com/oapi-codegen/runtimetov1.2.0to match generated code expectations.Written by Cursor Bugbot for commit 0f6b308. This will update automatically on new commits. Configure here.