Here's a comprehensive README.md:

letta-codebase-indexer

Automatically index your codebase into Letta archival memory for semantic code search by AI agents.

What It Does

This tool watches your codebase and indexes every file into a Letta agent's archival memory. Your agent can then semantically search your code using archival_memory_search("authentication logic") instead of manually reading files.

Use Cases:

• AI coding assistants that understand your entire codebase
• Semantic code search across large projects
• Context-aware development agents
• Code documentation and analysis

Installation

bash npm install -g letta-codebase-indexer Or use directly with npx: bash npx letta-codebase-indexer --help

Quick Start

1. One-Time Index

Index your project once: bash letta-index index
--agent-id agent-xxx-xxx-xxx
--project /path/to/your/project

2. Watch Mode

Continuously watch and update the index as you code: bash letta-index watch
--agent-id agent-xxx-xxx-xxx
--project /path/to/your/project Press Ctrl+C to stop watching.

CLI Options

Global Options

• --help - Show help
• --version - Show version

index Command

One-time indexing of your codebase. bash letta-index index [options] Options:

• -a, --agent-id <id> - Required. Your Letta agent ID
• -p, --project <path> - Project root path (default: current directory)
• -u, --base-url <url> - Letta server URL (default: $LETTA_BASE_URL or http://localhost:8283)
• -k, --api-key <key> - Letta API key (default: $LETTA_API_KEY)

Example: bash letta-index index
--agent-id agent-d0147e96-9db8-4a0a-b3ed-604c9b216564
--project ~/my-app

watch Command

Watch your codebase and automatically update the index on file changes. bash letta-index watch [options] Options:

• -a, --agent-id <id> - Required. Your Letta agent ID
• -p, --project <path> - Project root path (default: current directory)
• -u, --base-url <url> - Letta server URL
• -k, --api-key <key> - Letta API key
• --persist <path> - Path to persist file index (default: .letta-index.json)

Example: bash letta-index watch
--agent-id agent-d0147e96-9db8-4a0a-b3ed-604c9b216564
--project ~/my-app
--persist .index.json

Programmatic Usage

You can also use the indexer as a library in your Node.js/TypeScript projects: typescript import { CodebaseIndexer } from 'letta-codebase-indexer';

const indexer = new CodebaseIndexer({ agentId: 'agent-xxx-xxx-xxx', projectRoot: '/path/to/project', baseURL: 'http://localhost:8283/', fileExtensions: ['.ts', '.js', '.py', '.go'], // Optional ignorePatterns: ['.git', 'node_modules'], // Optional persistIndexPath: '.letta-index.json', // Optional });

// One-time index await indexer.initialIndex();

// Or watch for changes indexer.watch();

// Stop watching indexer.stop();

Configuration

Environment Variables

• LETTA_BASE_URL - Default Letta server URL
• LETTA_API_KEY - Default Letta API key

Set these in your shell: bash export LETTA_BASE_URL=http://localhost:8283/ export LETTA_API_KEY=your_key_here

File Extensions

By default, indexes these file types:

• .ts, .tsx (TypeScript)
• .js, .jsx (JavaScript)
• .py (Python)

Customize programmatically: typescript const indexer = new CodebaseIndexer({ fileExtensions: ['.ts', '.go', '.rs', '.java'], // ... });

Ignore Patterns

By default, ignores:

• .git
• node_modules
• dist
• __pycache__
• .venv

Customize programmatically: typescript const indexer = new CodebaseIndexer({ ignorePatterns: ['.git', 'node_modules', 'build', 'target'], // ... });

How It Works

1. Initial Index: Walks your project directory and creates archival passages for each file
2. Metadata: Each passage includes file path, type, content hash, and timestamp
3. Watch Mode: Uses chokidar to detect file changes and update passages
4. Persistence: Saves file-to-passage mappings in .letta-index.json for restart recovery

Finding Your Agent ID

Via Letta ADE (Desktop/Web)

1. Open your agent in the Agent Development Environment
2. Copy the agent ID from the URL or agent details
3. Format: agent-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Via SDK

typescript import Letta from '@letta-ai/letta-client';

const client = new Letta({ baseURL: 'http://localhost:8283/' }); const agents = await client.agents.list();

agents.items.forEach(agent => { console.log(${agent.name}: ${agent.id}); });

Example Workflows

Local Development with Docker

bash Terminal 1: Start Letta server docker run -p 8283:8283 letta/letta:latest

Terminal 2: Index your project letta-index watch --agent-id agent-xxx --project ~/my-app

Terminal 3: Chat with your agent in letta-code cd ~/my-app letta --agent agent-xxx Now ask your agent: "Find all the authentication functions" or "Where is the database connection logic?"

CI/CD Integration

Index your codebase in CI for documentation agents: yaml .github/workflows/index-codebase.yml name: Index Codebase on: [push] jobs: index: runs-on: ubuntu-latest steps:

uses: actions/checkout@v3 run: npx letta-codebase-indexer index --agent-id ${{ secrets.LETTA_AGENT_ID }} --base-url ${{ secrets.LETTA_BASE_URL }} --api-key ${{ secrets.LETTA_API_KEY }}

Troubleshooting

"Cannot find agent"

• Verify agent ID with letta-index --help and check your Letta ADE
• Ensure Letta server is running and accessible

"Connection refused"

• Check LETTA_BASE_URL points to your Letta server
• Verify server is running: curl http://localhost:8283/v1/health

Large files slow down indexing

• Consider adding file size limits (feature coming soon)
• Use .gitignore patterns to exclude large files

Index out of sync after restart

• The --persist flag saves the index to .letta-index.json
• Delete this file to force a full reindex

Roadmap

• File size limits
• Chunking for large files (>32k)
• .gitignore pattern support
• Progress bars for initial indexing
• Configurable embedding models
• Incremental updates (only changed files)

Contributing

Contributions welcome! This tool was built collaboratively on the Letta Discord.

1. Fork the repo
2. Create a feature branch: git checkout -b my-feature
3. Commit changes: git commit -am 'Add feature'
4. Push: git push origin my-feature
5. Open a Pull Request

License

MIT

Acknowledgments

Built with guidance from Ezra, Letta's community support agent, and the Letta development team. Special thanks to the Letta Discord community for feedback and testing.

Links

• Letta Documentation
• Letta GitHub
• Letta Discord
• Report Issues Letta (MemGPT) Letta (MemGPT) 1,083 Online 11,240 Members Est. Oct 2023 A server for AI developers and enthusiasts interested in building AI agents with advanced memory 👾 AI 🤖 Agents 🧪 Research

Go to Server