Skip to content

Canonical shared encryption config #386

Open
freshtonic wants to merge 9 commits intomainfrom
refactor/canonical-config
Open

Canonical shared encryption config #386
freshtonic wants to merge 9 commits intomainfrom
refactor/canonical-config

Conversation

@freshtonic
Copy link
Copy Markdown
Contributor

@freshtonic freshtonic commented Apr 1, 2026

refactor(proxy): replace local ColumnEncryptionConfig with canonical CanonicalEncryptionConfig

Migrate from the proxy-local encryption config types to the canonical types provided by the cipherstash-config crate. This removes ~200 lines of duplicated type definitions (CastAs, Column, Indexes, etc.) and consolidates config parsing into the shared crate.

  • Bump cipherstash-client/cts-common to 0.34.0-alpha.5
  • Add cipherstash-config workspace dependency
  • Add InvalidEncryptionConfig error variant for config crate errors
  • Update manager to use CanonicalEncryptionConfig with Identifier conversion
  • Rename Utf8Str -> Text and JsonB -> Json to match canonical types

Summary by CodeRabbit

  • Documentation

    • Added comprehensive test plan for proxy backward-compatibility validation of configuration migration.
  • Chores

    • Updated EQL version from 2.2.1 to 2.3.0-pre.1.
    • Updated dependency versions for cipherstash-client and cts-common to 0.34.1-alpha.3.
    • Added cipherstash-config as a workspace dependency.
  • Refactor

    • Migrated encryption configuration handling to use external configuration library with improved error handling and validation.

Add implementation plan for migrating proxy to use CanonicalEncryptionConfig
from the cipherstash-config crate.
…CanonicalEncryptionConfig

Migrate from the proxy-local encryption config types to the canonical
types provided by the cipherstash-config crate. This removes ~200 lines
of duplicated type definitions (CastAs, Column, Indexes, etc.) and
consolidates config parsing into the shared crate.

- Bump cipherstash-client/cts-common to 0.34.0-alpha.5
- Add cipherstash-config workspace dependency
- Add InvalidEncryptionConfig error variant for config crate errors
- Update manager to use CanonicalEncryptionConfig with Identifier conversion
- Rename Utf8Str -> Text and JsonB -> Json to match canonical types
- Add [patch.crates-io] for local development against cipherstash-suite

NOTE: The [patch.crates-io] section and version bumps to 0.34.0-alpha.5
must be updated when the canonical types are published to crates.io.
…g migration

Covers proxy-side integration tests for JSON -> CanonicalEncryptionConfig -> EncryptConfig
pipeline including Identifier bridging, ColumnType mapping, error propagation, and a
realistic schema fixture matching the integration test database.
…igration

Add 9 unit tests covering ColumnType-to-Postgres type mapping and
CanonicalEncryptionConfig parsing to establish a safety net before
further refactoring:

- column.rs: 4 tests for column_type_to_postgres_type (text, json,
  json accessor, exhaustive variant coverage)
- config.rs: 5 tests for config map construction (identifier bridging,
  multi-table configs, invalid index validation, realistic EQL schema
  fixture, malformed JSON error handling)
@freshtonic freshtonic requested a review from tobyhede April 1, 2026 12:37
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 1, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 50255364-014f-4232-b21d-8663c86421d1

📥 Commits

Reviewing files that changed from the base of the PR and between a088022 and 63b9aba.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (6)
  • Cargo.toml
  • mise.local.example.toml
  • mise.toml
  • packages/cipherstash-proxy/src/proxy/encrypt_config/config.rs
  • packages/cipherstash-proxy/src/proxy/encrypt_config/manager.rs
  • packages/cipherstash-proxy/src/proxy/encrypt_config/mod.rs
💤 Files with no reviewable changes (2)
  • packages/cipherstash-proxy/src/proxy/encrypt_config/mod.rs
  • packages/cipherstash-proxy/src/proxy/encrypt_config/config.rs
✅ Files skipped from review due to trivial changes (1)
  • mise.local.example.toml
🚧 Files skipped from review as they are similar to previous changes (3)
  • mise.toml
  • packages/cipherstash-proxy/src/proxy/encrypt_config/manager.rs
  • Cargo.toml

📝 Walkthrough

Walkthrough

The PR updates workspace dependencies (cipherstash-client, cts-common, and the new cipherstash-config) to version 0.34.1-alpha.3, removes the proxy's legacy ColumnEncryptionConfig model in favor of CanonicalEncryptionConfig from the new dependency, renames column types (Utf8StrText, JsonBJson), updates SQL conversion logic across multiple files, and bumps the EQL version to eql-2.3.0-pre.1.

Changes

Cohort / File(s) Summary
Workspace & Package Dependencies
Cargo.toml, packages/cipherstash-proxy/Cargo.toml
Updated workspace dependency versions for cipherstash-client and cts-common from 0.34.0-alpha.4 to =0.34.1-alpha.3; added new cipherstash-config dependency at =0.34.1-alpha.3.
Configuration Architecture Refactor
packages/cipherstash-proxy/src/proxy/encrypt_config/config.rs, packages/cipherstash-proxy/src/proxy/encrypt_config/manager.rs, packages/cipherstash-proxy/src/proxy/encrypt_config/mod.rs
Removed legacy ColumnEncryptionConfig module and replaced with CanonicalEncryptionConfig deserialization and conversion logic; load_encrypt_config now uses canonical_to_map helper to transform canonical identifiers to client form; added comprehensive test coverage for defaults, cast mappings, and error cases.
Column Type Migrations
packages/cipherstash-proxy/src/postgresql/context/column.rs, packages/cipherstash-proxy/src/postgresql/data/from_sql.rs, packages/cipherstash-proxy/src/postgresql/data/to_sql.rs
Replaced column type enum variants: ColumnType::Utf8StrColumnType::Text and ColumnType::JsonBColumnType::Json across SQL-to-plaintext and plaintext-to-SQL mappings; added unit tests verifying mappings and non-panicking behavior for all ColumnType variants.
Error Handling
packages/cipherstash-proxy/src/error.rs
Added InvalidEncryptionConfig variant to ConfigError enum with automatic conversion from cipherstash_config::errors::ConfigError.
Environment & Build Configuration
mise.local.example.toml, mise.toml, .gitignore
Updated CS_EQL_VERSION from eql-2.2.1 to eql-2.3.0-pre.1; added /.work to .gitignore.
Test Documentation
docs/plans/2026-04-01-proxy-backwards-compat-tests.md
Added comprehensive test plan documenting verification tasks for proxy's canonical config migration pipeline, including identifier bridging, type mapping validation, error propagation, and integration fixture testing.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Suggested reviewers

  • tobyhede
  • auxesis
  • coderdan

Poem

🐰 The config hops to a newer way,
With canonical shapes in place to stay,
Text and Json dance where strings once lay,
EQL skips forward—version play!
Dependencies aligned, tests light the way. ✨

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Canonical shared encryption config' directly addresses the main objective of replacing proxy-local config types with canonical shared types from cipherstash-config.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch refactor/canonical-config

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

🧹 Nitpick comments (1)
packages/cipherstash-proxy/src/proxy/encrypt_config/config.rs (1)

352-355: Assert the error shape, not only is_err().

These tests currently pass for any error. Tighten assertions to validate the intended failure mode (validation vs. deserialization-shape), so regressions don’t slip through.

Suggested tightening
 let config: CanonicalEncryptionConfig = serde_json::from_value(json).unwrap();
-let result = config.into_config_map();
-assert!(result.is_err(), "ste_vec on text column should fail validation");
+let err = config.into_config_map().unwrap_err();
+assert!(
+    err.to_string().contains("ste_vec"),
+    "expected ste_vec validation failure, got: {err}"
+);

 let result = serde_json::from_value::<CanonicalEncryptionConfig>(json);
-assert!(result.is_err());
+let err = result.unwrap_err();
+assert!(
+    err.to_string().contains("tables"),
+    "expected tables shape error, got: {err}"
+);

Also applies to: 440-441

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/cipherstash-proxy/src/proxy/encrypt_config/config.rs` around lines
352 - 355, The test currently only checks that
CanonicalEncryptionConfig::into_config_map() returns Err, which is too loose;
change the assertion to inspect the error shape and message so it specifically
asserts the validation failure (not a serde/deserialization error). After
calling config.into_config_map(), call unwrap_err() and assert it is the
expected validation variant (e.g., matches EncryptionConfigError::Validation(_)
or contains the expected validation message like "ste_vec on text column"), and
apply the same tighter assertion pattern to the other test occurrence that calls
CanonicalEncryptionConfig::into_config_map() later in the file.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@Cargo.toml`:
- Around line 60-64: The shared Cargo manifest currently contains a
[patch.crates-io] section referencing local paths for cipherstash-client,
cipherstash-config, cts-common, and zerokms-protocol which break CI; remove that
entire [patch.crates-io] block from Cargo.toml (the entries for
cipherstash-client, cipherstash-config, cts-common, zerokms-protocol) before
merging, and instead move those local path overrides into a non-shared local
config (e.g., Cargo.local or a developer-only manifest) so CI uses the published
crates.

In `@docs/plans/2026-04-01-migrate-to-canonical-encryption-config.md`:
- Line 17: The heading "Task 1: Add `cipherstash-config` dependency" is
currently H3 causing a level jump from the H1 at the top; change this heading
from "### Task 1: Add `cipherstash-config` dependency" to an H2 ("## Task 1: Add
`cipherstash-config` dependency") or insert an appropriate intermediate H2 above
it so the document has a proper H1 -> H2 -> H3 hierarchy and satisfies
markdownlint MD001.
- Line 13: The plan file references a local filesystem path that only works on
the author's machine; update the pointer in
docs/plans/2026-04-01-migrate-to-canonical-encryption-config.md so the Design
doc link points to a repo-relative path or a stable permalink within the
cipherstash-suite repository (e.g., reference
docs/plans/2026-04-01-canonical-encryption-config-design.md or the repo URL/PR
permalink) instead of `~/cipherstash/...` so all readers can resolve the design
doc.

In `@docs/plans/2026-04-01-proxy-backwards-compat-tests.md`:
- Line 11: The task headings in the document (e.g., "Task 1: Test Identifier
bridging in EncryptConfig") use level-3 headings (###) which violates
markdownlint MD001; change each task heading to level-2 (##) so tasks use
consistent heading level; update the occurrences listed (including the headings
at the shown fragment and the other task headings referenced on lines 68, 134,
178, and 266) by replacing the leading "###" with "##" for those task titles.

In `@packages/cipherstash-proxy/src/error.rs`:
- Around line 188-189: The InvalidEncryptionConfig error variant currently lacks
a remediation/documentation link; update the #[error("Invalid encryption
configuration: {0}")] message for InvalidEncryptionConfig(#[from]
cipherstash_config::errors::ConfigError) to include the same friendly
remediation/docs URL used by the other ConfigError variants (or call the same
helper/formatter they use) so operators receive a pointer to the fix path when
this canonical config validation error occurs.

In `@packages/cipherstash-proxy/src/proxy/encrypt_config/config.rs`:
- Around line 306-307: The test currently calls
config.get(&ident).expect("column exists") which uses a generic expect message;
replace this with config.get(&ident).unwrap() (or, alternatively, provide an
identifier-specific message like expect(&format!("column {:?} exists", ident)))
so the test follows the guideline to use unwrap() in tests; update the use site
where the local variable column is assigned (variable name: column, lookup:
config.get(&ident)) accordingly.

In `@packages/cipherstash-proxy/src/proxy/encrypt_config/manager.rs`:
- Around line 215-218: The serde_json deserialization of json_value into
CanonicalEncryptionConfig is currently using the bare `?` which routes serde
errors into the general Error::Encrypt; instead map deserialization failures
into the config domain by calling serde_json::from_value(json_value).map_err(|e|
crate::error::ConfigError::Parse(e).into()) (or equivalent) before unwrapping,
so that CanonicalEncryptionConfig::from_value errors are converted to
ConfigError::Parse; keep the subsequent call to
canonical.into_config_map().map_err(crate::error::ConfigError::from)? as-is.

---

Nitpick comments:
In `@packages/cipherstash-proxy/src/proxy/encrypt_config/config.rs`:
- Around line 352-355: The test currently only checks that
CanonicalEncryptionConfig::into_config_map() returns Err, which is too loose;
change the assertion to inspect the error shape and message so it specifically
asserts the validation failure (not a serde/deserialization error). After
calling config.into_config_map(), call unwrap_err() and assert it is the
expected validation variant (e.g., matches EncryptionConfigError::Validation(_)
or contains the expected validation message like "ste_vec on text column"), and
apply the same tighter assertion pattern to the other test occurrence that calls
CanonicalEncryptionConfig::into_config_map() later in the file.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 723dbdb6-f3ff-4278-8bff-343dd233160e

📥 Commits

Reviewing files that changed from the base of the PR and between 30aafc6 and 83611df.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (10)
  • Cargo.toml
  • docs/plans/2026-04-01-migrate-to-canonical-encryption-config.md
  • docs/plans/2026-04-01-proxy-backwards-compat-tests.md
  • packages/cipherstash-proxy/Cargo.toml
  • packages/cipherstash-proxy/src/error.rs
  • packages/cipherstash-proxy/src/postgresql/context/column.rs
  • packages/cipherstash-proxy/src/postgresql/data/from_sql.rs
  • packages/cipherstash-proxy/src/postgresql/data/to_sql.rs
  • packages/cipherstash-proxy/src/proxy/encrypt_config/config.rs
  • packages/cipherstash-proxy/src/proxy/encrypt_config/manager.rs

Comment thread Cargo.toml Outdated
Comment thread docs/plans/2026-04-01-migrate-to-canonical-encryption-config.md Outdated
Comment thread docs/plans/2026-04-01-migrate-to-canonical-encryption-config.md Outdated
Comment thread docs/plans/2026-04-01-proxy-backwards-compat-tests.md
Comment thread packages/cipherstash-proxy/src/error.rs
Comment thread packages/cipherstash-proxy/src/proxy/encrypt_config/config.rs Outdated
Comment thread packages/cipherstash-proxy/src/proxy/encrypt_config/manager.rs Outdated
Comment thread docs/plans/2026-04-01-migrate-to-canonical-encryption-config.md Outdated
Comment thread packages/cipherstash-proxy/src/proxy/encrypt_config/manager.rs Outdated
Comment thread packages/cipherstash-proxy/src/proxy/encrypt_config/config.rs Outdated
Comment thread docs/plans/2026-04-01-migrate-to-canonical-encryption-config.md Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (2)
Cargo.toml (2)

46-48: ⚠️ Potential issue | 🔴 Critical

Blocker: shared manifest is pinned to local sibling paths (CI/non-local builds will fail).

Line 46-Line 48 and Line 60-Line 64 require ../cipherstash-suite/..., which is not portable and has already caused CI breakage. Keep published versions in the shared Cargo.toml, and move local overrides to developer-local Cargo config.

Suggested manifest direction
 [workspace.dependencies]
 sqltk = { version = "0.10.0" }
-cipherstash-client = { path = "../cipherstash-suite/packages/cipherstash-client" }
-cipherstash-config = { path = "../cipherstash-suite/packages/cipherstash-config" }
-cts-common = { path = "../cipherstash-suite/packages/cts-common" }
+cipherstash-client = "0.34.0-alpha.5"
+cipherstash-config = "0.2.3"
+cts-common = "0.34.0-alpha.5"

-[patch.crates-io]
-cipherstash-client = { path = "../cipherstash-suite/packages/cipherstash-client" }
-cipherstash-config = { path = "../cipherstash-suite/packages/cipherstash-config" }
-cts-common = { path = "../cipherstash-suite/packages/cts-common" }
-zerokms-protocol = { path = "../cipherstash-suite/packages/zerokms-protocol" }
#!/bin/bash
set -euo pipefail

echo "=== Cargo.toml dependency section ==="
sed -n '44,70p' Cargo.toml

echo
echo "=== Check whether local sibling paths exist in this checkout ==="
for p in \
  "../cipherstash-suite/packages/cipherstash-client" \
  "../cipherstash-suite/packages/cipherstash-config" \
  "../cipherstash-suite/packages/cts-common" \
  "../cipherstash-suite/packages/zerokms-protocol"
do
  if [ -e "$p" ]; then
    echo "FOUND: $p"
  else
    echo "MISSING: $p"
  fi
done

Also applies to: 60-64

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@Cargo.toml` around lines 46 - 48, The shared Cargo.toml currently pins local
sibling paths for crates cipherstash-client, cipherstash-config, cts-common (and
similarly zerokms-protocol), which breaks CI/non-local builds; change the
dependency entries in Cargo.toml to reference the published crate versions
(remove the path = "../..." overrides) and move any developer-only local path
overrides into a developer-local cargo config (e.g., .cargo/config.toml) using
[patch.crates-io] or [replace] so local development can still override those
crates without contaminating the shared manifest.

60-63: ⚠️ Potential issue | 🟠 Major

Global [patch.crates-io] masks real release compatibility testing.

The cipherstash-config override on line 62 redirects all cipherstash-config dependencies to the local path, including the explicit cipherstash-config = "0.2.3" declaration in packages/cipherstash-proxy-integration/Cargo.toml. This prevents validating that the integration crate actually works with the published 0.2.3 version and can hide compatibility breakage before release.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@Cargo.toml` around lines 60 - 63, The patch entry is overriding
cipherstash-config globally which prevents testing against the published 0.2.3
release; remove the cipherstash-config = { path =
"../cipherstash-suite/packages/cipherstash-config" } line from the
[patch.crates-io] section in Cargo.toml so the integration crate's explicit
dependency (cipherstash-config = "0.2.3" in
packages/cipherstash-proxy-integration/Cargo.toml) will resolve to the published
crate; if you still need a local override for development, limit the patch to
only the specific workspace crates that must use local sources (e.g., keep
cipherstash-client and cts-common) or use a temporary, targeted override
mechanism instead of globally patching cipherstash-config.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@Cargo.toml`:
- Around line 46-48: The shared Cargo.toml currently pins local sibling paths
for crates cipherstash-client, cipherstash-config, cts-common (and similarly
zerokms-protocol), which breaks CI/non-local builds; change the dependency
entries in Cargo.toml to reference the published crate versions (remove the path
= "../..." overrides) and move any developer-only local path overrides into a
developer-local cargo config (e.g., .cargo/config.toml) using [patch.crates-io]
or [replace] so local development can still override those crates without
contaminating the shared manifest.
- Around line 60-63: The patch entry is overriding cipherstash-config globally
which prevents testing against the published 0.2.3 release; remove the
cipherstash-config = { path = "../cipherstash-suite/packages/cipherstash-config"
} line from the [patch.crates-io] section in Cargo.toml so the integration
crate's explicit dependency (cipherstash-config = "0.2.3" in
packages/cipherstash-proxy-integration/Cargo.toml) will resolve to the published
crate; if you still need a local override for development, limit the patch to
only the specific workspace crates that must use local sources (e.g., keep
cipherstash-client and cts-common) or use a temporary, targeted override
mechanism instead of globally patching cipherstash-config.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9f014b88-8d9c-4de2-98f6-ae8fc35650b5

📥 Commits

Reviewing files that changed from the base of the PR and between 83611df and a088022.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (5)
  • .gitignore
  • Cargo.toml
  • mise.local.example.toml
  • mise.toml
  • packages/cipherstash-proxy/src/proxy/encrypt_config/config.rs
✅ Files skipped from review due to trivial changes (2)
  • .gitignore
  • mise.local.example.toml
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/cipherstash-proxy/src/proxy/encrypt_config/config.rs

Comment thread packages/cipherstash-proxy/src/proxy/encrypt_config/config.rs Outdated
Comment thread packages/cipherstash-proxy/src/proxy/encrypt_config/config.rs Outdated
@freshtonic freshtonic force-pushed the refactor/canonical-config branch from a088022 to c21471e Compare April 15, 2026 05:42
Move the test-only config.rs module into manager.rs alongside the code
under test, extract a shared canonical_to_map helper used by both
load_encrypt_config and the tests, and add coverage for the `real`
cast_as alias that maps to ColumnType::Float.

Addresses PR #386 review feedback.
@freshtonic freshtonic force-pushed the refactor/canonical-config branch from d42c5d2 to 63b9aba Compare April 15, 2026 06:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants