Skip to content

feat: Allow dropping empty tables during auto-migration#4593

Open
Ludv1gL wants to merge 1 commit intoclockworklabs:masterfrom
Ludv1gL:feat/allow-drop-empty-tables
Open

feat: Allow dropping empty tables during auto-migration#4593
Ludv1gL wants to merge 1 commit intoclockworklabs:masterfrom
Ludv1gL:feat/allow-drop-empty-tables

Conversation

@Ludv1gL
Copy link
Contributor

@Ludv1gL Ludv1gL commented Mar 9, 2026

Summary

  • Removing a table from the schema previously always triggered AutoMigrateError::RemoveTable, forcing --delete-data which nukes the entire database
  • Now, empty tables can be dropped seamlessly during spacetime publish
  • Non-empty tables fail with an actionable error guiding the user to clear rows first

Changes

  • auto_migrate.rs: Replace hard RemoveTable error with AutoMigrateStep::RemoveTable. Compute removed_tables set (same pattern as new_tables) and pass to auto_migrate_indexes/auto_migrate_sequences/auto_migrate_constraints to skip sub-object diffs for removed tables (cascade handled by drop_table)
  • update.rs: Execute RemoveTable step — O(1) emptiness check via table_row_count_mut, then drop_table. Fails with clear message if table has data
  • formatter.rs / termcolor_formatter.rs: Add format_remove_table to MigrationFormatter trait + implementation
  • publish.rs (smoketests): Update existing test — removing empty table now succeeds without flags

Safety

  • Transaction safety: Emptiness check and drop run in the same MutTx — no window for concurrent inserts
  • Cascade: drop_table already handles removing all indexes, constraints, and sequences for the table
  • Sub-object filtering: Indexes/constraints/sequences belonging to removed tables are filtered from their respective diffs, preventing orphan RemoveIndex/RemoveConstraint/RemoveSequence steps
  • Rollback: If the emptiness check fails, the entire migration aborts before any changes are applied

Example error output (non-empty table)

Cannot remove table `MyTable`: table contains data. Clear the table's rows (e.g. via a reducer) before removing it from your schema.

Test plan

  • cargo check -p spacetimedb-schema -p spacetimedb-core passes
  • All 14 auto_migrate schema tests pass (2 new: remove_table_produces_step, remove_table_does_not_produce_orphan_sub_object_steps)
  • All 3 update execution tests pass (2 new: remove_empty_table_succeeds, remove_nonempty_table_fails)
  • Updated existing auto_migration_errors test (no longer expects RemoveTable error)
  • Updated smoke test: cli_can_publish_remove_empty_table expects success

🤖 Generated with Claude Code

assert!(
err.to_string().contains("table contains data"),
"error should mention that the table contains data, got: {err}"
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check that pending_schema_changes is empty here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — asserting pending_schema_changes has len 1 (the TableRemoved entry from drop_table).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any new assertion, and the list should be the empty slice, not of length 1.

false,
true,
);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this test changed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test was originally cli_cannot_publish_breaking_change_without_flag — it tested that removing a table fails during publish. Now that empty table removal succeeds via auto-migration, the test was updated to cli_can_publish_remove_empty_table and expects success. The smoke test module starts with empty tables (fresh publish), so the removal succeeds as expected.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point of the test is not to test that removing a table fails during publish, but that flag is required to acknowledge breaking changes, which removing an empty table still should be, so this test should not need to change, or there is a bug in the PR.

@Centril Centril self-assigned this Mar 10, 2026
@Ludv1gL Ludv1gL force-pushed the feat/allow-drop-empty-tables branch from e35fcdd to e45d973 Compare March 10, 2026 14:33
"removing a table should disconnect clients"
);
assert!(stdb.table_id_from_name_mut(&tx, "droppable")?.is_none());
// drop_table records a TableRemoved schema change for the subscription system.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Rust convention is that code terms (keywords, functions, binding names, ...) are referenced in commentary using single back-quotes, e.g., drop_table.

assert!(stdb.table_id_from_name_mut(&tx, "droppable")?.is_none());
// drop_table records a TableRemoved schema change for the subscription system.
assert_eq!(
tx.pending_schema_changes().len(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like this to expect TableRemoved, not just that there was one change.

assert!(
err.to_string().contains("table contains data"),
"error should mention that the table contains data, got: {err}"
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any new assertion, and the list should be the empty slice, not of length 1.

Comment on lines +468 to +469
|(mut added, mut removed), d| {
match d {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
|(mut added, mut removed), d| {
match d {
|(mut added, mut removed), diff| {
match diff {

Comment on lines +2436 to +2439
vec![
AutoMigrateStep::RemoveTable(&drop_table),
AutoMigrateStep::DisconnectAllUsers,
],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
vec![
AutoMigrateStep::RemoveTable(&drop_table),
AutoMigrateStep::DisconnectAllUsers,
],
&[
AutoMigrateStep::RemoveTable(&drop_table),
AutoMigrateStep::DisconnectAllUsers,
],

These are tests, but no need to make a heap allocation for an explicit list on the stack.

let plan = ponder_auto_migrate(&old, &new).expect("removing a table should produce a valid plan");
assert_eq!(
plan.steps,
vec![
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

Previously, removing a table always triggered AutoMigrateError::RemoveTable,
forcing --delete-data which nukes the entire database.

Now:
- Empty tables: dropped via auto-migration (requires --break-clients)
- Non-empty tables: actionable error suggesting clearing rows first
- Sub-object diffs (indexes, constraints, sequences) for removed tables
  are filtered since drop_table handles cascade

Co-Authored-By: Claude <noreply@anthropic.com>
@Ludv1gL Ludv1gL force-pushed the feat/allow-drop-empty-tables branch from e45d973 to ed4f6ed Compare March 24, 2026 22:02
@Ludv1gL
Copy link
Contributor Author

Ludv1gL commented Mar 24, 2026

Rebased onto latest master and addressed all round 2 feedback:

  1. Backtick code terms — fixed in test comments
  2. Assert TableRemoved variant — now checks matches!(c, PendingSchemaChange::TableRemoved(..))
  3. Empty pending_schema_changes on failureremove_nonempty_table_fails now asserts is_empty()
  4. Rename ddiff — done in fold closure
  5. &[...] instead of vec![...] — done in both test assertions
  6. Smoketest reverted — the original test is unchanged from upstream. Removing an empty table still requires --break-clients acknowledgment since it's a client-breaking change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants