propose: add uv support alongside pip workflow

propose/UV-SUPPORT-PROPOSE.md

@@ -0,0 +1,207 @@
+# Add `uv` support alongside existing `pip`-based workflow
+
+**Status**: draft
+**Author**: Dmitriy Teriaev + Perplexity Computer
+**Date**: 2026-05-12
+
+## TL;DR
+
+- Adopt `uv` as a supported install / dev-workflow tool **alongside** the existing `pip install -r requirements.txt` path. Both work, neither is removed.
+- Add a `[dependency-groups.dev]` table to `pyproject.toml` listing `pytest`, `ruff`, `pytest-asyncio`, and `unidiff` (the dev-only deps that today are ambient assumptions). Runtime `[project.dependencies]` is untouched.
+- Generate and commit `uv.lock` (uv's standard reproducible-install artefact).
+- **`requirements.txt` stays as-is, hand-maintained.** It is **not** regenerated from `uv export`. Locked decision: the two artefacts are independent; uv contributors do not touch `requirements.txt`.
+- README §1 gets a parallel "Install with uv" subsection. Existing `pip install -r requirements.txt` path remains the documented default.
+- Breaking change posture: **none**. Pure addition.
+- Ships in **2 PRs**: PR-1 adds `[dependency-groups.dev]` + commits `uv.lock` + README; PR-2 (optional, deferred) adds a CONTRIBUTING note documenting the two-track lockfile policy.
+
+## §1 — Frame
+
+> **`uv` is added as a developer convenience for fast, reproducible installs. It does not become the source of truth for runtime dependency pins; `requirements.txt` stays hand-maintained and authoritative for pip-based installs.**
+
+The repo already has the structural prerequisite: a PEP-621 `pyproject.toml` with `[project]`, `[project.dependencies]`, and `[project.scripts]`. uv reads this file natively; no metadata migration is needed. The only question this propose answers is **how to integrate uv without destabilising the existing pip workflow** — specifically, without letting uv's resolver re-derive the pin set that `requirements.txt` has already frozen against known-good upstream versions (e.g. `cocoindex==1.0.0a43`, the LanceDB / sentence-transformers / tree-sitter-java triplet).
+
+Two integration models exist:
+
+1. **Single source of truth via uv.** Delete `requirements.txt`. `pyproject.toml` + `uv.lock` are the only dependency artefacts. uv's resolver re-derives the pin set on first `uv lock`.
+2. **Coexist with hand-maintained `requirements.txt`.** uv reads `pyproject.toml` for the dependency *ranges*, computes its own lockfile, and resolves to whatever uv's SAT solver picks. `requirements.txt` continues to live independently with its existing pinned versions; nobody regenerates it from uv.
+
+This propose locks model #2. The current `requirements.txt` is a battle-tested freeze; re-resolving it via uv risks shifting transitive pins in ways that have to be re-validated against the codebase's test suite and against runtime behaviour on pre-release deps. The cost of "two artefacts to maintain" is bounded (uv contributors edit `pyproject.toml`'s ranges; pip contributors edit `requirements.txt`'s pins; CI doesn't enforce coherence between them) and far smaller than the cost of an unscheduled dependency-resolver migration.
+
+## §2 — Design principles
+
+1. **Pure addition; nothing is removed or reshaped.** Existing `pip install -r requirements.txt` continues to work byte-for-byte identically.
+2. **`requirements.txt` is hand-maintained and authoritative for pip installs.** It is **not** regenerated from `uv export`. uv contributors do not edit it.
+3. **`uv.lock` is the authoritative lockfile for `uv sync` installs.** It is regenerated by `uv lock` on every `pyproject.toml` change a uv contributor makes.
+4. **The two lockfiles are independent and may legitimately diverge** on transitive pins. CI does not enforce coherence. Drift is acceptable because no single workflow consumes both.
+5. **Dev dependencies belong in `[dependency-groups.dev]`**, not in the runtime `[project.dependencies]` table. Today they are ambient assumptions; this propose makes them explicit.
+6. **No CI changes in this propose.** The repo has no `.github/workflows/` directory today; introducing CI for uv-vs-pip equivalence is a separate decision.
+
+## §3 — The proposed surface
+
+### `pyproject.toml` change
+
+Add after the existing `[project]` block, before `[tool.setuptools]`:
+
+```toml
+[dependency-groups]
+dev = [
+    "pytest>=8.0,<9",
+    "pytest-asyncio>=0.24,<2",
+    "ruff>=0.6,<1",
+    "unidiff>=0.7.3,<1",
+]
+```
+
+`unidiff` appears twice (once in runtime `[project.dependencies]`, once in `[dependency-groups.dev]`) **only if** the test suite uses it differently from the runtime PR-analysis path. If the same dependency line covers both, list it only in `[project.dependencies]`. Verified at PR-1 review time.
+
+No `[tool.uv]` table needed — uv's defaults (lockfile name `uv.lock`, single workspace, no Python version constraint beyond `requires-python = ">=3.11"` from `[project]`) match what we want.
+
+### New file
+
+`uv.lock` — generated by `uv lock` from `pyproject.toml`, committed to the repo. Treated as a build artefact: humans never edit it directly; uv regenerates on dependency changes.
+
+### `requirements.txt` change
+
+**None.** The file stays as-is.
+
+### README §1 — Install section
+
+Add a parallel subsection after the existing `pip install -r requirements.txt` block:
+
+```markdown
+### Install with uv (alternative, recommended for development)
+
+```bash
+cd /path/to/java-codebase-rag
+uv sync                  # runtime deps only, from uv.lock
+uv sync --group dev      # adds pytest, ruff, pytest-asyncio, unidiff for development
+```
+
+uv reads dependency *ranges* from `pyproject.toml` and resolves to pinned versions in `uv.lock`. The pin set in `uv.lock` is **independent of** `requirements.txt` — the two are maintained separately. Contributors using `pip install -r requirements.txt` should not run `uv lock`; contributors using `uv sync` should not regenerate `requirements.txt`.
+```
+
+### `.gitignore` change
+
+**None.** `.venv/` and `venv/` are already ignored (lines 7–8). uv's default virtualenv path is `.venv/`, covered.
+
+## §4 — What this deliberately does NOT do
+
+| Question / feature | Why we skip it |
+|---|---|
+| Delete `requirements.txt` | Locked design choice — see §2 principle 2 and §1 frame paragraph 2 |
+| Regenerate `requirements.txt` from `uv export` (any frequency) | Re-introduces resolver coupling the design chose to avoid |
+| Add CI for `pip-install ↔ uv-sync` equivalence checking | Out of scope; no `.github/workflows/` exists today |
+| Add `[tool.uv]` table with custom resolver options | YAGNI; uv defaults match what we want |
+| Migrate to `uv` as the only dependency manager | Explicit out-of-scope; this is a coexistence propose |
+| Add pre-commit hook running `uv lock` on `pyproject.toml` change | Out of scope; documented developer responsibility is enough |
+| Add `uv` to the runtime dependency surface | uv is a developer-side tool; it is not bundled with the package |
+| Switch the build backend from `setuptools` to `hatchling` / `uv_build` | Out of scope; setuptools works, no reason to change |
+| Pin transitive deps in `[dependency-groups.dev]` (e.g. pinning `pytest-asyncio==0.24.0`) | Use ranges in `pyproject.toml`; let `uv.lock` carry the pins |
+| Document `uv pip install -r requirements.txt` as a third install path | Confusing — surfaces uv's pip-compat shim while the propose deliberately keeps the two ecosystems separate |
+
+## §5 — Use-case re-walk
+
+Fifteen developer / CI scenarios walked against the post-cutover surface.
+
+| # | Use case | Path used | Outcome |
+|---|---|---|---|
+| UC1 | New contributor clones repo, no uv installed | `python3 -m venv .venv && .venv/bin/pip install -r requirements.txt` | ✅ Existing workflow unchanged |
+| UC2 | New contributor with uv installed | `uv sync --group dev` | ✅ Single command; runtime + dev deps installed from `uv.lock` |
+| UC3 | Contributor adds a new runtime dependency (e.g. `httpx`) | Edit `pyproject.toml` `[project.dependencies]`; `uv lock` regenerates `uv.lock`; **also** add a pinned line to `requirements.txt` by hand | ⚠️ Two edits required. Documented as developer responsibility in §3 README block |
+| UC4 | Contributor adds a new dev dependency (e.g. `pytest-mock`) | Edit `pyproject.toml` `[dependency-groups.dev]`; `uv lock` regenerates `uv.lock` | ✅ `requirements.txt` untouched — it never held dev deps |
+| UC5 | Contributor bumps an existing runtime dep (e.g. `kuzu` range widened) | Edit `pyproject.toml` range; `uv lock`; edit `requirements.txt` pin to match | ⚠️ Same two-edit pattern as UC3 |
+| UC6 | Contributor runs the test suite locally | `uv sync --group dev && uv run pytest` | ✅ uv handles venv + invocation |
+| UC7 | Contributor runs the test suite via existing pip workflow | `.venv/bin/pip install -r requirements.txt && pip install pytest ruff pytest-asyncio unidiff && .venv/bin/python -m pytest` | ✅ Works, but dev deps are ambient (pre-propose status quo) |
+| UC8 | Reproduce a CI failure locally with exact pins | `pip install -r requirements.txt` matches pip-CI; `uv sync` matches uv-CI | ✅ Each install path is internally reproducible |
+| UC9 | `uv.lock` diverges from `requirements.txt` on a transitive pin | No CI enforcement; both lockfiles install successfully; runtime tests pass under both | ✅ By design (§2 principle 4) |
+| UC10 | Contributor forgets to run `uv lock` after `pyproject.toml` change | `uv sync` detects stale lockfile and re-resolves; warning surfaces in dev shell | ✅ uv's standard behaviour |
+| UC11 | Contributor forgets to update `requirements.txt` after `pyproject.toml` change | No automated detection. `pip install -r requirements.txt` keeps installing the old pin. Next pip-using contributor catches it in PR review | ⚠️ Known cost of the two-track policy |
+| UC12 | `uv.lock` merge conflict on a feature branch | `uv lock` regenerates from `pyproject.toml`; conflict resolves trivially | ✅ Standard uv merge workflow |
+| UC13 | `requirements.txt` merge conflict | Hand-resolve as today | ✅ Unchanged |
+| UC14 | Contributor with old Python (3.10) tries `uv sync` | `pyproject.toml` declares `requires-python = ">=3.11"`; uv refuses to resolve | ✅ Existing constraint enforced |
+| UC15 | Contributor wants to add a one-off dev tool (e.g. `mypy`) without committing | `uv pip install mypy` into the active venv; `pyproject.toml` untouched | ✅ uv's pip-compat shim covers ad-hoc installs |
+
+Three use cases (UC3, UC5, UC11) surface the same cost: contributors who add or bump a runtime dep must edit both `pyproject.toml` (for uv) and `requirements.txt` (for pip). This is the price of the two-track policy and is documented in the README install subsection.
+
+## §6 — Migration plan — 2 PRs
+
+### PR-1: `chore: add uv support alongside pip workflow`
+
+Single atomic commit. Touches:
+- `pyproject.toml` — add `[dependency-groups.dev]` block after `[project]`
+- `uv.lock` — **new** file, generated by `uv lock`, committed
+- `README.md` §1 — add "Install with uv" subsection after the existing pip block; include the two-track-lockfile note
+
+No code change. No `requirements.txt` change. No `.gitignore` change (`.venv/` already ignored).
+
+Tests: existing suite passes under both install paths. Manually verify:
+- `python3 -m venv .venv && .venv/bin/pip install -r requirements.txt && .venv/bin/python -m pytest` (existing path)
+- `uv sync --group dev && uv run pytest` (new path)
+
+CI: no `.github/workflows/` exists today, so no automation to update.
+
+### PR-2 (optional, deferred): `docs: add CONTRIBUTING.md with dependency-management policy`
+
+Touches:
+- `CONTRIBUTING.md` — **new** file documenting the two-track policy: who edits what, how to add a dep, the explicit non-coupling between `uv.lock` and `requirements.txt`, the documented developer responsibility for updating both on runtime-dep changes.
+
+Deferred because a single README paragraph in PR-1 may be sufficient. PR-2 ships only if PR-1 review surfaces enough recurring contributor questions to justify a dedicated doc.
+
+## §7 — Decisions taken (no longer open)
+
+1. **uv coexists with pip; nothing is removed.** Existing `pip install -r requirements.txt` path stays the documented default.
+2. **`requirements.txt` stays hand-maintained and authoritative for pip installs.** It is **not** regenerated from `uv export`.
+3. **`uv.lock` is the authoritative lockfile for `uv sync` installs** and is committed to the repo.
+4. **The two lockfiles are independent and may diverge** on transitive pins. CI does not enforce coherence.
+5. **Dev dependencies live in `[dependency-groups.dev]`** in `pyproject.toml`. They are not added to `requirements.txt`. Existing ambient-assumption pattern for pip users continues (manual `pip install pytest ruff pytest-asyncio`).
+6. **No `[tool.uv]` table.** uv defaults apply.
+7. **No CI changes in this propose.** No `.github/workflows/` exists today.
+8. **`uv pip install -r requirements.txt` is not a documented install path.** The propose deliberately keeps the two ecosystems separate; surfacing uv's pip-compat shim would muddle the message.
+9. **No pre-commit hooks.** Contributor responsibility to run `uv lock` and update `requirements.txt` is documented in README §1.
+10. **Build backend stays `setuptools`.** No migration to `hatchling` / `uv_build`.
+11. **uv is not a runtime dependency.** It is a developer-side tool only.
+12. **`unidiff` placement** (runtime vs dev group) verified at PR-1 review time. If the test suite imports it differently from the runtime PR-analysis path, listing it in both is acceptable.
+
+## §8 — Risks and how we mitigate
+
+| Risk | Mitigation |
+|---|---|
+| `uv.lock` and `requirements.txt` drift to incompatible transitive pins; a bug only reproduces under one of the two install paths | Documented as accepted (§2 principle 4). Contributor reproducing a CI failure must use the same install path CI used. CI currently uses pip; the propose adds no CI for uv. |
+| Contributor edits `pyproject.toml` `[project.dependencies]` but forgets to update `requirements.txt` | Caught in PR review. README §1 install subsection explicitly calls out the two-edit responsibility. UC11 documents this as a known cost. |
+| `uv lock` produces a lockfile that resolves to a transitive pin which breaks runtime (e.g. `numpy 2.5` vs current `numpy 2.4`) | First `uv lock` is a smoke-test event. PR-1's manual-verification step runs `uv run pytest` under the new lockfile. If resolution differs from `requirements.txt` and tests fail, narrow the `pyproject.toml` range to match what `requirements.txt` has frozen. |
+| A future contributor "simplifies" the setup by deleting `requirements.txt` and regenerating it from `uv export` | Locked decision (§7 #2) prevents this in PR review. README §1 install subsection states the policy explicitly; CONTRIBUTING.md (PR-2) reinforces it. |
+| Dev deps drift between the (newly explicit) `[dependency-groups.dev]` list and what ad-hoc pip users actually install | Acceptable. Pip users have always installed dev deps ad hoc; the propose just makes the canonical list discoverable. No regression. |
+| `uv.lock` merge conflicts on long-lived feature branches | Standard `uv lock` re-resolve workflow; same pattern as `pnpm-lock.yaml` / `Cargo.lock`. Not unique to this propose. |
+| `unidiff` ends up in both `[project.dependencies]` and `[dependency-groups.dev]` accidentally | Verified at PR-1 review. uv handles dual listing gracefully; the cost is cosmetic redundancy, not behavioural. |
+
+## Appendix A — Concrete `[dependency-groups.dev]` block
+
+```toml
+[dependency-groups]
+dev = [
+    "pytest>=8.0,<9",
+    "pytest-asyncio>=0.24,<2",
+    "ruff>=0.6,<1",
+    "unidiff>=0.7.3,<1",
+]
+```
+
+Version ranges chosen to match the pyproject.toml convention (lower bound = currently-installed-or-greater, upper bound = next major). `pytest` lower bound `>=8.0` matches the modern test feature set the repo uses; `ruff` `>=0.6` matches the toolchain set in `[tool.ruff]`'s `target-version = "py311"` era.
+
+## Appendix B — What changed (traceability)
+
+First draft; no prior revisions.
+
+**What it adds beyond the current packaging:**
+- Makes `uv` a first-class developer-workflow option without disturbing the existing pip-based reproducible install.
+- Surfaces dev-only dependencies as a declared group instead of an ambient assumption.
+- Locks the two-track lockfile policy explicitly so future contributors don't relitigate the coupling question.
+
+**What stays unchanged:**
+- `[project]` table including `[project.dependencies]` and `[project.scripts]`.
+- `[tool.setuptools]` and the build backend.
+- `[tool.ruff]` config.
+- `requirements.txt` (every line, every pin).
+- `.gitignore`.
+- README §1's `pip install -r requirements.txt` path as the documented default.
+- The absence of `.github/workflows/`.