Skip to content

Flaky CLI tests: sentence-transformers modules_json_path=None during CocoIndex subprocess init #215

@HumanBean17

Description

@HumanBean17

Summary

Two CLI tests fail intermittently during full-suite runs (pytest tests -v):

  • test_init_after_erase_succeeds
  • test_increment_emits_kuzu_stale_warning_block

Both pass reliably in isolation and within test_java_codebase_rag_cli.py alone.

Error

sentence_transformers/base/model.py:1083: TypeError: expected str, bytes or os.PathLike object, not NoneType

Inside SentenceTransformer._load_config_modules, modules_json_path resolves to None when CocoIndex spawns a subprocess that loads the embedding model during java-codebase-rag init.

Root cause

sentence-transformers 5.4.0 intermittently fails to resolve modules_json_path from the HuggingFace cache during subprocess model loading. The HF cache on disk is valid (modules.json exists at the expected snapshot path). The failure only manifests during the full test suite, likely due to cache contention or a transient resolution bug in the library.

Environment

  • sentence-transformers==5.4.0
  • cocoindex latest (uses SentenceTransformer internally for init/increment)
  • Python 3.11, macOS Darwin 25.5.0
  • Model: sentence-transformers/all-MiniLM-L6-v2

Reproduction

.venv/bin/python -m pytest tests -v
# Run a few times — fails ~10-20% of the time

Never fails when run alone:

.venv/bin/python -m pytest tests/test_java_codebase_rag_cli.py -v

Possible fixes (not yet attempted)

  1. Pre-warm model in session fixture — load SentenceTransformer(SBERT_MODEL) once before CLI tests. Subprocess would hit a warm cache. Adds ~10s to suite.
  2. Retry subprocess once — wrap the failing assert returncode == 0 with a single retry on this specific error. Cheap but masks root cause.
  3. Pin sentence-transformers to a version without this bug — may conflict with CocoIndex deps.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions