Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 100 additions & 8 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,11 @@ AWS_SECRET_ACCESS_KEY=your_aws_secret_access_key_here
# Voice Pipeline
# ==============================================

#i LLM model alias for the assistant. Must match a model_name in EVA_MODEL_LIST.
#d enum
#x pipeline_mode=LLM
EVA_MODEL__LLM=gpt-5.2

# Pipeline mode is controlled by the UI radio (LLM / S2S / AudioLLM).
# The #x conditions below ensure each variable is only active for the right mode.

Expand All @@ -73,16 +78,16 @@ EVA_MODEL__STT=cartesia
#x pipeline_mode=LLM
EVA_MODEL__STT_PARAMS='{"api_key": "your_cartesia_api_key", "model": "ink-whisper"}'

# --- LLM mode: TTS ---
# --- TTS (LLM and AudioLLM modes) ---
#i TTS provider for the voice pipeline.
#d enum
#e cartesia,chatterbox,elevenlabs,gemini,kokoro,nvidia-baseten,openai,xtts
#x pipeline_mode=LLM
#x pipeline_mode=LLM,AudioLLM
EVA_MODEL__TTS=cartesia

#i TTS provider parameters. Must include "api_key" and "model". Use "urls" for round-robin load balancing.
#d json_object
#x pipeline_mode=LLM
#x pipeline_mode=LLM,AudioLLM
EVA_MODEL__TTS_PARAMS='{"api_key": "your_cartesia_api_key", "model": "sonic"}'

# --- S2S mode ---
Expand Down Expand Up @@ -150,11 +155,6 @@ EVA_MODEL_LIST='[
}
]'

#i LLM model alias for the assistant. Must match a model_name in EVA_MODEL_LIST.
#d enum
#x pipeline_mode=LLM
EVA_MODEL__LLM=gpt-5.2

# ==============================================
# Framework & Runtime
# ==============================================
Expand Down Expand Up @@ -234,6 +234,98 @@ EVA_MODEL__LLM=gpt-5.2
# User Config
# ==============================================

# --- Language (mutually exclusive with Accent and Behavior) ---
#i ISO 639-1 language code for the user simulator. Datasets must exist for the selected language. Pattern for the agent ID pairs below: EVA_{LANG}_USER_{F|M}.
#d enum
#e en,es,fr,de,pt,ja,zh
#x perturbation_mode=Language
#v EVA_LANGUAGE=en

# --- Language agent IDs ---
#i ElevenLabs agent ID — English, female voice.
#d string
#x perturbation_mode=Language
#x EVA_LANGUAGE=en
#v EVA_EN_USER_F=

#i ElevenLabs agent ID — English, male voice.
#d string
#x perturbation_mode=Language
#x EVA_LANGUAGE=en
#v EVA_EN_USER_M=

#i ElevenLabs agent ID — Spanish, female voice.
#d string
#x perturbation_mode=Language
#x EVA_LANGUAGE=es
#v EVA_ES_USER_F=

#i ElevenLabs agent ID — Spanish, male voice.
#d string
#x perturbation_mode=Language
#x EVA_LANGUAGE=es
#v EVA_ES_USER_M=

#i ElevenLabs agent ID — French, female voice.
#d string
#x perturbation_mode=Language
#x EVA_LANGUAGE=fr
#v EVA_FR_USER_F=

#i ElevenLabs agent ID — French, male voice.
#d string
#x perturbation_mode=Language
#x EVA_LANGUAGE=fr
#v EVA_FR_USER_M=

#i ElevenLabs agent ID — German, female voice.
#d string
#x perturbation_mode=Language
#x EVA_LANGUAGE=de
#v EVA_DE_USER_F=

#i ElevenLabs agent ID — German, male voice.
#d string
#x perturbation_mode=Language
#x EVA_LANGUAGE=de
#v EVA_DE_USER_M=

#i ElevenLabs agent ID — Portuguese, female voice.
#d string
#x perturbation_mode=Language
#x EVA_LANGUAGE=pt
#v EVA_PT_USER_F=

#i ElevenLabs agent ID — Portuguese, male voice.
#d string
#x perturbation_mode=Language
#x EVA_LANGUAGE=pt
#v EVA_PT_USER_M=

#i ElevenLabs agent ID — Japanese, female voice.
#d string
#x perturbation_mode=Language
#x EVA_LANGUAGE=ja
#v EVA_JA_USER_F=

#i ElevenLabs agent ID — Japanese, male voice.
#d string
#x perturbation_mode=Language
#x EVA_LANGUAGE=ja
#v EVA_JA_USER_M=

#i ElevenLabs agent ID — Mandarin, female voice.
#d string
#x perturbation_mode=Language
#x EVA_LANGUAGE=zh
#v EVA_ZH_USER_F=

#i ElevenLabs agent ID — Mandarin, male voice.
#d string
#x perturbation_mode=Language
#x EVA_LANGUAGE=zh
#v EVA_ZH_USER_M=

# --- Default user simulator agents ---
#i ElevenLabs agent ID for the default female-voice user persona.
#d string
Expand Down
51 changes: 51 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,57 @@ streamlit run apps/config_editor.py

The editor covers all variables grouped by tab (API keys, voice pipeline, model deployments, runtime settings, perturbations, etc.), with proper widgets for each type. See [`apps/README.md`](apps/README.md) for details.

### Adding a Language

To run EVA in a language other than English, follow these steps:

**1. Register the language code** — add a new entry to the `LanguageType` enum in [`src/eva/models/config.py`](src/eva/models/config.py), and the language display name right below it:

```python
class LanguageType(StrEnum):
english = "en"
french = "fr" # already present
spanish = "es" # already present
# add yours here, e.g.:
italian = "it"
```

**2. Add ElevenLabs agent IDs to `.env.example`** (and your `.env`) — add a conditional block for the new language code so the config editor exposes the fields:

```bash
#x EVA_LANGUAGE=it
#v EVA_IT_USER_F=your_elevenlabs_agent_id_female
#x EVA_LANGUAGE=it
#v EVA_IT_USER_M=your_elevenlabs_agent_id_male
```

To make it visible in the config app, also modify
```bash
#e en,es,fr,de,pt,ja,zh
#x perturbation_mode=Language
EVA_LANGUAGE=fr
```

The `#e` line defines the available options for the `EVA_LANGUAGE` variable in the config editor. Add your new language code (e.g., `it`) to this list.

**3. Run `add_culture_data.py`** — this generates culturally appropriate names and translated utterances for every record in every domain dataset, writes a "respond in X" addendum to `configs/agents/language_addenda.yaml`, and translates the assistant's opening greeting into `configs/agents/initial_messages.yaml`:

```bash
PYTHONPATH=src python scripts/add_culture_data.py \
--language it \
--language-name Italian \
--native-name italiano \
--auto-generate-names
```

Re-running is safe — existing entries are skipped (idempotent). Use `--dry-run` to preview changes before writing.

**4. Set `EVA_LANGUAGE` and run**:

```bash
EVA_LANGUAGE=it EVA_DOMAIN=airline python main.py
```

### Exploring Results

EVA includes a Streamlit analysis app for visualizing and comparing results:
Expand Down
106 changes: 105 additions & 1 deletion apps/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,111 @@ Interactive UI for building and editing `.env` configuration files without hand-
streamlit run apps/config_editor.py
```

The app reads `.env.example` for the full variable set and loads existing values from `.env` if present. Each variable's widget type, enum options, ranges, and tooltips are declared directly in `.env.example` using annotation prefixes (`#i`, `#d`, `#e`, `#r`, `#x`, `#v`). Use the **Preview** button to inspect the generated file before saving, or **Download** to export it without writing to disk.
The app reads `.env.example` for the full variable set and loads existing values from `.env` if present. Each variable's widget type, enum options, ranges, and tooltips are declared directly in `.env.example` using annotation prefixes. Use the **Preview** button to inspect the generated file before saving, or **Download** to export it without writing to disk.

### `.env.example` Annotation Scheme

The editor is driven entirely from annotated comments in `.env.example` — there is no separate schema file. Each annotation prefix applies to the **immediately following** variable definition (active or inactive). Annotation order doesn't matter, but the block must be contiguous: any blank line or `# ` true-comment between annotations resets the accumulator.

| Prefix | Name | Purpose |
|---|---|---|
| `# ` | True comment | Human-readable prose. Preserved verbatim, never parsed as metadata. |
| `#i ` | Info | Tooltip text shown next to the widget. Multiple `#i` lines join with spaces. |
| `#d ` | Datatype | Widget type — see table below. If omitted, inferred from name + value. |
| `#e ` | Enum options | Comma-separated valid values for `enum` / `multi_enum`. |
| `#r ` | Range | Numeric `min,max` or `min,max,step` for `int` / `float`. |
| `#g ` | Group | Override tab assignment (otherwise inherited from section header). |
| `#x ` | Condition | `VAR=value` — only render when that var equals that value. Comma-separated values are OR (`#x pipeline_mode=LLM,AudioLLM`). Multiple `#x` lines = AND. |
| `#v ` | Inactive var | `#v VARNAME=value` — a variable definition that ships off by default but is fully configurable. |

#### Widget types (`#d`)

| Type | Renders as |
|---|---|
| `string` | `st.text_input` |
| `secret` | `st.text_input(type="password")` |
| `bool` | `st.checkbox` |
| `int` | `st.number_input` (integers, range from `#r`) |
| `float` | `st.number_input` (floats, range from `#r`) |
| `enum` | `st.selectbox` (options from `#e`) |
| `multi_enum` | `st.multiselect` (options from `#e`) |
| `csv_list` | `st.text_input` split/joined on comma |
| `path` | `st.text_input` with existence hint |
| `json_object` | Key/value table + raw JSON expander |
| `json_deployment_list` | Special-cased deployment-card editor for `EVA_MODEL_LIST` |

#### Widget inference (when `#d` is omitted)

- Name contains `KEY`, `SECRET`, `TOKEN`, or `PASSWORD` → `secret`
- Name contains `CREDENTIALS` or ends with `_PATH` / `_DIR` → `path`
- Value is `true` / `false` → `bool`
- Value parses as an integer → `int`, as a float → `float`
- Value looks like a JSON array containing `model_name` → `json_deployment_list`
- Value looks like a JSON array or object → `json_object`
- Otherwise → `string`

#### Section headers

Top-level groups are declared by a 3-line header block. Variables that follow inherit the group name until the next header.

```bash
# ==============================================
# Voice Pipeline
# ==============================================
```

The section title must match one of the tab name constants in [`config_schema.py`](config_schema.py) (`API Configs`, `Voice Pipeline`, `LiteLLM Deployments`, `Framework & Runtime`, `Turn Detection & VAD`, `User Config`, `Debug & Logging`).

#### Variable states

```bash
# Just a note — ignored entirely.

#i Maximum parallel conversations.
#d int
#r 1,100,1
EVA_MAX_CONCURRENT_CONVERSATIONS=5 # active — written to .env

#i Domain for dataset/agent paths.
#d enum
#e airline,itsm,medical_hr
#v EVA_DOMAIN=airline # inactive — user can enable in UI

#i French accent agent ID.
#d secret
#x perturbation_mode=Accent
#x EVA_PERTURBATION__ACCENT=french
#v EVA_FRENCH_ACCENT_USER_F= # only renders when both conditions hold
```

#### Conditions and modes

`#x` conditions can reference either:
- Another env variable's value (e.g. `#x EVA_PERTURBATION__ACCENT=french`)
- A UI-only state key managed by a mutex radio button (e.g. `#x pipeline_mode=LLM`)

Mutex radio buttons are declared in [`config_schema.py`](config_schema.py) via `MUTEX_RADIOS`. Each radio writes to a session-state key (`pipeline_mode`, `perturbation_mode`) that `#x` conditions can match against.

#### Serialization rules

When the user saves `.env`:

| In `.env.example` | User sets a value | Disabled by mutex / `#x` | Output |
|---|---|---|---|
| Active (`VAR=…`) | yes | no | `VAR=value` |
| Active (`VAR=…`) | no | no | original line verbatim |
| Active (`VAR=…`) | any | yes | `#v VAR=value` (or example value) |
| Inactive (`#v VAR=…`) | yes | no | `VAR=value` (activated) |
| Inactive (`#v VAR=…`) | no | any | `#v VAR=…` verbatim |
| Not in template, in user's loaded `.env` | — | — | appended in matching tab section (KEY/URL → API Configs, `EVA_*` → Framework & Runtime, otherwise Misc) |

Round-tripping is lossless: `serialize_env({}, parse_env_example(...))` reproduces the original file byte-for-byte.

#### Implementation

- [`config_io.py`](config_io.py) — `parse_env_example`, `load_env`, `serialize_env`, `compute_disabled`. Pure functions, no Streamlit dependency.
- [`config_schema.py`](config_schema.py) — group constants, tab ordering, mutex radio definitions. Everything else lives in `.env.example`.
- [`config_editor.py`](config_editor.py) — Streamlit UI that dispatches on `AnnotatedVar.widget`.

---

Expand Down
9 changes: 7 additions & 2 deletions apps/config_editor.py
Original file line number Diff line number Diff line change
Expand Up @@ -133,12 +133,17 @@ def _init_state() -> None:


def _is_visible_av(var: AnnotatedVar) -> bool:
"""Return True when all #x conditions for this var are satisfied."""
"""Return True when all #x conditions for this var are satisfied.

Comma-separated values in a single condition are treated as OR
(e.g. `#x pipeline_mode=LLM,AudioLLM`).
"""
for cond_key, cond_val in var.conditions:
actual = st.session_state.get(cond_key)
if actual is None:
actual = st.session_state.get("field_values", {}).get(cond_key)
if actual != cond_val:
allowed = {v.strip() for v in cond_val.split(",") if v.strip()}
if actual not in allowed:
return False
return True

Expand Down
3 changes: 2 additions & 1 deletion apps/config_io.py
Original file line number Diff line number Diff line change
Expand Up @@ -385,7 +385,8 @@ def compute_disabled(parsed: ParsedEnvExample, **state_values: str) -> set[str]:
disabled: set[str] = set()
for var in parsed.vars:
for cond_key, cond_val in var.conditions:
if state_values.get(cond_key, "") != cond_val:
allowed = {v.strip() for v in cond_val.split(",") if v.strip()}
if state_values.get(cond_key, "") not in allowed:
disabled.add(var.name)
break
return disabled
6 changes: 3 additions & 3 deletions apps/config_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,9 +61,9 @@ def __post_init__(self) -> None:
MutexRadio(
state_key="perturbation_mode",
group=GROUP_PERTURBATIONS,
label="Perturbation persona",
options=["None", "Accent", "Behavior"],
help="Accent and Behavior are mutually exclusive (each claims the agent ID slot).",
label="User mode",
options=["None", "Language", "Accent", "Behavior"],
help="Language, Accent, and Behavior are mutually exclusive each claims the ElevenLabs agent ID slot.",
default="None",
),
]
12 changes: 10 additions & 2 deletions configs/agents/airline_agent.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,16 @@ role: You are a voice agent for SkyWay Airlines handling inbound calls for fligh
tool_module_path: eva.assistant.tools.airline_tools
instructions: |
## Authentication

Every call begins with authentication. Ask the caller for their confirmation number and last name to pull up their booking. Confirm you have the correct reservation before proceeding.

### Name Disambiguation

If a lookup fails or returns no match, use the following escalation path:

1. Ask the caller to confirm or spell their last name — once, clearly.
2. If the spelling is confirmed correct and the lookup still fails, ask for a secondary signal: the departure date, origin city, or destination.
3. If the caller has a very common last name (one that alone is insufficient to narrow results), lead with the confirmation number as the primary key and treat the last name as a secondary check, not the other way around.

## Core Principles

Expand Down Expand Up @@ -463,4 +471,4 @@ tools:
technical_issue, unable_to_resolve)
- name: issue_summary
type: string
description: Brief summary of the issue and what has been attempted
description: Brief summary of the issue and what has been attempted, in the same language as the conversation you had with the user
2 changes: 2 additions & 0 deletions configs/agents/initial_messages.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
en: "Hello! How can I help you today?"
fr: "Bonjour ! Comment puis-je vous aider aujourd'hui ?"
2 changes: 1 addition & 1 deletion configs/agents/itsm_agent.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -1323,7 +1323,7 @@ tools:
description: "The reason for transferring the call."
- name: issue_summary
type: string
description: "A brief summary of the request, what was attempted, and why transfer is needed, so the live agent has full context."
description: "A brief summary of the request, what was attempted, and why transfer is needed, so the live agent has full context. This should be in the same language as the conversation you had with the user."

- id: mark_resolved
name: mark_resolved
Expand Down
Loading