refactor: consolidate agent configurations and improve test coverage by neelneelpurk · Pull Request #1712 · github/spec-kit

neelneelpurk · 2026-02-27T07:53:25Z

Summary

This PR refactors the AI agent configuration system to eliminate duplication, reduce the risk of configuration drift, and improve maintainability. It also enhances the test suite to ensure consistent command registration across all supported agents.

Changes

Consolidated Configuration: Moved the agent metadata from src/specify_cli/__init__.py to a new dedicated file: src/specify_cli/agent_config.py.
Dynamic Registry: Updated src/specify_cli/extensions.py to dynamically generate the CommandRegistrar.AGENT_CONFIGS registry from the shared source of truth.
Improved Internal Naming: Renamed internal configuration keys to be more descriptive and avoid namespace collisions:
- "format" → "command_format"
- "args" → "command_args"
- "extension" → "command_extension"
Generic Command Registration: Updated src/specify_cli/extensions.py to use a generic registration flow that works for all agents without special-case logic.
Expanded Test Coverage: Updated tests/test_extensions.py using @pytest.mark.parametrize to loop through every supported agent (including agy, codex, gemini, etc.) to verify command and alias registration.
Updated Documentation: Refactored AGENTS.md to point developers to the new configuration location and reflect the updated field names.

Impact

Maintenance: Adding support for a new agent now only requires updating one file: src/specify_cli/agent_config.py.
Reliability: Eliminates the previous drift issue where some agents were missing from the extension registration system.
Test Quality: All 18 configured agents are now automatically tested for command format and directory accuracy.

Testing

Tested locally with uv run specify --help
Ran existing tests with uv sync && uv run pytest
Tested with a sample project (if applicable)

============================= test session starts ==============================
platform darwin -- Python 3.12.11, pytest-9.0.2, pluggy-1.6.0 -- /Users/neelneelpurk/projects/spec-kit/.venv/bin/python3
cachedir: .pytest_cache
rootdir: /Users/neelneelpurk/projects/spec-kit
configfile: pyproject.toml
plugins: anyio-4.12.1, cov-7.0.0
collecting ... collected 56 items

tests/test_ai_skills.py::TestGetSkillsDir::test_claude_skills_dir PASSED
tests/test_ai_skills.py::TestGetSkillsDir::test_gemini_skills_dir PASSED
tests/test_ai_skills.py::TestGetSkillsDir::test_copilot_skills_dir PASSED
tests/test_ai_skills.py::TestGetSkillsDir::test_codex_uses_override PASSED
tests/test_ai_skills.py::TestGetSkillsDir::test_cursor_agent_skills_dir PASSED
tests/test_ai_skills.py::TestGetSkillsDir::test_unknown_agent_uses_default PASSED
tests/test_ai_skills.py::TestGetSkillsDir::test_all_configured_agents_resolve PASSED
tests/test_ai_skills.py::TestGetSkillsDir::test_override_takes_precedence_over_config PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_installed_with_correct_structure ✓ Installed 4 agent skills to .claude/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_generated_skill_has_parseable_yaml ✓ Installed 4 agent skills to .claude/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_empty_yaml_frontmatter ✓ Installed 4 agent skills to .claude/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_enhanced_descriptions_used_when_available ✓ Installed 4 agent skills to .claude/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_template_without_frontmatter ✓ Installed 4 agent skills to .claude/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_missing_templates_directory Warning: command templates not found, skipping skills installation
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_empty_templates_directory Warning: command templates not found, skipping skills installation
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_malformed_yaml_frontmatter Warning: Failed to install skill broken: while parsing a flow sequence
  in "<unicode string>", line 2, column 14:
    description: ', but got ':'
  in "<unicode string>", line 3, column 10:
      invalid: yaml: content: here
             ^
No skills were installed
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_additive_does_not_overwrite_other_files ✓ Installed 4 agent skills to .claude/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_return_value ✓ Installed 4 agent skills to .claude/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_return_false_when_no_templates Warning: command templates not found, skipping skills installation
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_non_md_commands_dir_falls_back ✓ Installed 9 agent skills to .gemini/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_install_for_all_agents[copilot] ✓ Installed 9 agent skills to .github/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_install_for_all_agents[claude] ✓ Installed 1 agent skills to .claude/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_install_for_all_agents[gemini] ✓ Installed 1 agent skills to .gemini/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_install_for_all_agents[cursor-agent] ✓ Installed 1 agent skills to .cursor/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_install_for_all_agents[qwen] ✓ Installed 1 agent skills to .qwen/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_install_for_all_agents[opencode] ✓ Installed 9 agent skills to .opencode/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_install_for_all_agents[codex] ✓ Installed 9 agent skills to .agents/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_install_for_all_agents[windsurf] ✓ Installed 9 agent skills to .windsurf/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_install_for_all_agents[kilocode] ✓ Installed 9 agent skills to .kilocode/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_install_for_all_agents[auggie] ✓ Installed 9 agent skills to .augment/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_install_for_all_agents[codebuddy] ✓ Installed 1 agent skills to .codebuddy/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_install_for_all_agents[qodercli] ✓ Installed 1 agent skills to .qoder/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_install_for_all_agents[roo] ✓ Installed 9 agent skills to .roo/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_install_for_all_agents[q] ✓ Installed 9 agent skills to .amazonq/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_install_for_all_agents[amp] ✓ Installed 1 agent skills to .agents/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_install_for_all_agents[shai] ✓ Installed 1 agent skills to .shai/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_install_for_all_agents[agy] ✓ Installed 9 agent skills to .agent/skills/
PASSED
tests/test_ai_skills.py::TestInstallAiSkills::test_skills_install_for_all_agents[bob] ✓ Installed 1 agent skills to .bob/skills/
PASSED
tests/test_ai_skills.py::TestCommandCoexistence::test_existing_commands_preserved_claude ✓ Installed 4 agent skills to .claude/skills/
PASSED
tests/test_ai_skills.py::TestCommandCoexistence::test_existing_commands_preserved_gemini ✓ Installed 9 agent skills to .gemini/skills/
PASSED
tests/test_ai_skills.py::TestCommandCoexistence::test_commands_dir_not_removed ✓ Installed 4 agent skills to .claude/skills/
PASSED
tests/test_ai_skills.py::TestCommandCoexistence::test_no_commands_dir_no_error ✓ Installed 4 agent skills to .claude/skills/
PASSED
tests/test_ai_skills.py::TestNewProjectCommandSkip::test_new_project_commands_removed_after_skills_succeed PASSED
tests/test_ai_skills.py::TestNewProjectCommandSkip::test_commands_preserved_when_skills_fail PASSED
tests/test_ai_skills.py::TestNewProjectCommandSkip::test_here_mode_commands_preserved PASSED
tests/test_ai_skills.py::TestSkipIfExists::test_existing_skill_not_overwritten ✓ Installed 3 agent skills to .claude/skills/
PASSED
tests/test_ai_skills.py::TestSkipIfExists::test_fresh_install_writes_all_skills ✓ Installed 4 agent skills to .claude/skills/
PASSED
tests/test_ai_skills.py::TestSkillDescriptions::test_all_known_commands_have_descriptions PASSED
tests/test_ai_skills.py::TestCliValidation::test_ai_skills_without_ai_fails PASSED
tests/test_ai_skills.py::TestCliValidation::test_ai_skills_without_ai_shows_usage PASSED
tests/test_ai_skills.py::TestCliValidation::test_ai_skills_flag_appears_in_help PASSED
tests/test_ai_skills.py::TestParameterOrderingIssue::test_ai_flag_consuming_here_flag PASSED
tests/test_ai_skills.py::TestParameterOrderingIssue::test_ai_flag_consuming_ai_skills_flag PASSED
tests/test_ai_skills.py::TestParameterOrderingIssue::test_error_message_provides_hint PASSED
tests/test_ai_skills.py::TestParameterOrderingIssue::test_error_message_lists_available_agents PASSED
tests/test_ai_skills.py::TestParameterOrderingIssue::test_ai_commands_dir_consuming_flag PASSED

============================== 56 passed in 0.41s ==============================


============================= test session starts ==============================
platform darwin -- Python 3.12.11, pytest-9.0.2, pluggy-1.6.0 -- /Users/neelneelpurk/projects/spec-kit/.venv/bin/python3
cachedir: .pytest_cache
rootdir: /Users/neelneelpurk/projects/spec-kit
configfile: pyproject.toml
plugins: anyio-4.12.1, cov-7.0.0
collecting ... collected 73 items

tests/test_extensions.py::TestExtensionManifest::test_valid_manifest PASSED
tests/test_extensions.py::TestExtensionManifest::test_missing_required_field PASSED
tests/test_extensions.py::TestExtensionManifest::test_invalid_extension_id PASSED
tests/test_extensions.py::TestExtensionManifest::test_invalid_version PASSED
tests/test_extensions.py::TestExtensionManifest::test_invalid_command_name PASSED
tests/test_extensions.py::TestExtensionManifest::test_no_commands PASSED
tests/test_extensions.py::TestExtensionManifest::test_manifest_hash PASSED
tests/test_extensions.py::TestExtensionRegistry::test_empty_registry PASSED
tests/test_extensions.py::TestExtensionRegistry::test_add_extension PASSED
tests/test_extensions.py::TestExtensionRegistry::test_remove_extension PASSED
tests/test_extensions.py::TestExtensionRegistry::test_registry_persistence PASSED
tests/test_extensions.py::TestExtensionManager::test_check_compatibility_valid PASSED
tests/test_extensions.py::TestExtensionManager::test_check_compatibility_invalid PASSED
tests/test_extensions.py::TestExtensionManager::test_install_from_directory PASSED
tests/test_extensions.py::TestExtensionManager::test_install_duplicate PASSED
tests/test_extensions.py::TestExtensionManager::test_remove_extension PASSED
tests/test_extensions.py::TestExtensionManager::test_remove_nonexistent PASSED
tests/test_extensions.py::TestExtensionManager::test_list_installed PASSED
tests/test_extensions.py::TestExtensionManager::test_config_backup_on_remove PASSED
tests/test_extensions.py::TestCommandRegistrar::test_parse_frontmatter_valid PASSED
tests/test_extensions.py::TestCommandRegistrar::test_parse_frontmatter_no_frontmatter PASSED
tests/test_extensions.py::TestCommandRegistrar::test_render_frontmatter PASSED
tests/test_extensions.py::TestCommandRegistrar::test_register_commands_for_each_agent[copilot-config0] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_register_commands_for_each_agent[claude-config1] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_register_commands_for_each_agent[gemini-config2] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_register_commands_for_each_agent[cursor-agent-config3] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_register_commands_for_each_agent[qwen-config4] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_register_commands_for_each_agent[opencode-config5] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_register_commands_for_each_agent[codex-config6] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_register_commands_for_each_agent[windsurf-config7] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_register_commands_for_each_agent[kilocode-config8] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_register_commands_for_each_agent[auggie-config9] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_register_commands_for_each_agent[codebuddy-config10] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_register_commands_for_each_agent[qodercli-config11] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_register_commands_for_each_agent[roo-config12] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_register_commands_for_each_agent[q-config13] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_register_commands_for_each_agent[amp-config14] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_register_commands_for_each_agent[shai-config15] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_register_commands_for_each_agent[agy-config16] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_register_commands_for_each_agent[bob-config17] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_command_with_aliases[copilot-config0] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_command_with_aliases[claude-config1] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_command_with_aliases[gemini-config2] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_command_with_aliases[cursor-agent-config3] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_command_with_aliases[qwen-config4] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_command_with_aliases[opencode-config5] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_command_with_aliases[codex-config6] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_command_with_aliases[windsurf-config7] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_command_with_aliases[kilocode-config8] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_command_with_aliases[auggie-config9] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_command_with_aliases[codebuddy-config10] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_command_with_aliases[qodercli-config11] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_command_with_aliases[roo-config12] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_command_with_aliases[q-config13] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_command_with_aliases[amp-config14] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_command_with_aliases[shai-config15] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_command_with_aliases[agy-config16] PASSED
tests/test_extensions.py::TestCommandRegistrar::test_command_with_aliases[bob-config17] PASSED
tests/test_extensions.py::TestVersionSatisfies::test_version_satisfies_simple PASSED
tests/test_extensions.py::TestVersionSatisfies::test_version_satisfies_range PASSED
tests/test_extensions.py::TestVersionSatisfies::test_version_satisfies_complex PASSED
tests/test_extensions.py::TestVersionSatisfies::test_version_satisfies_invalid PASSED
tests/test_extensions.py::TestIntegration::test_full_install_and_remove_workflow PASSED
tests/test_extensions.py::TestIntegration::test_multiple_extensions PASSED
tests/test_extensions.py::TestExtensionCatalog::test_catalog_initialization PASSED
tests/test_extensions.py::TestExtensionCatalog::test_cache_directory_creation PASSED
tests/test_extensions.py::TestExtensionCatalog::test_cache_expiration PASSED
tests/test_extensions.py::TestExtensionCatalog::test_search_all_extensions PASSED
tests/test_extensions.py::TestExtensionCatalog::test_search_by_query PASSED
tests/test_extensions.py::TestExtensionCatalog::test_search_by_tag PASSED
tests/test_extensions.py::TestExtensionCatalog::test_search_verified_only PASSED
tests/test_extensions.py::TestExtensionCatalog::test_get_extension_info PASSED
tests/test_extensions.py::TestExtensionCatalog::test_clear_cache PASSED

============================== 73 passed in 0.34s ==============================

AI Disclosure

I did not use AI assistance for this contribution
I did use AI assistance (describe below)

Used Antigravity for code generation and review.

Issue

Fixes #1705

Copilot

Pull request overview

This PR refactors the AI agent configuration system by consolidating agent metadata into a single source of truth (src/specify_cli/agent_config.py), eliminating duplication between __init__.py and extensions.py. The refactoring improves maintainability by ensuring consistent configuration across the CLI initialization and extension registration systems.

Changes:

Created agent_config.py as the single source of truth for all agent metadata, including newly added fields for command format, args, and extension
Updated extensions.py to dynamically generate CommandRegistrar.AGENT_CONFIGS from the shared configuration with improved internal field naming
Expanded test coverage with parametrized tests that verify command registration for all 18 configured agents (including AGY/Antigravity)

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
`src/specify_cli/agent_config.py`	New file defining AGENT_CONFIG dictionary with comprehensive agent metadata including command format specifications
`src/specify_cli/__init__.py`	Removed duplicate AGENT_CONFIG definition and imports from new agent_config module
`src/specify_cli/extensions.py`	Updated to import and dynamically transform AGENT_CONFIG into AGENT_CONFIGS, renamed internal fields (format→command_format, args→command_args, extension→command_extension)
`tests/test_extensions.py`	Enhanced with parametrized tests covering all agents for command registration and alias handling
`AGENTS.md`	Updated documentation to reference new config location and document new field names

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mnriem · 2026-02-27T14:51:24Z

@neelneelpurk Can you please resolve

Run uvx ruff check src/
Downloading ruff (10.7MiB)
Downloaded ruff
Installed 1 package in 2ms
E402 Module level import not at top of file
--> src/specify_cli/init.py:126:1
|
124 | return "\n".join(lines)
125 |
126 | from .agent_config import AGENT_CONFIG
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
127 |
128 | SCRIPT_TYPE_CHOICES = {"sh": "POSIX Shell (bash/zsh)", "ps": "PowerShell"}
|

Found 1 error.

neelneelpurk · 2026-02-28T15:20:42Z

@neelneelpurk Can you please resolve

Run uvx ruff check src/ Downloading ruff (10.7MiB) Downloaded ruff Installed 1 package in 2ms E402 Module level import not at top of file --> src/specify_cli/init.py:126:1 | 124 | return "\n".join(lines) 125 | 126 | from .agent_config import AGENT_CONFIG | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 127 | 128 | SCRIPT_TYPE_CHOICES = {"sh": "POSIX Shell (bash/zsh)", "ps": "PowerShell"} |

Found 1 error.

@mnriem Resolved this in my latest commit d26c592

refactor: consolidate agent configurations and improve test coverage

1fd770b

neelneelpurk requested a review from mnriem as a code owner February 27, 2026 07:53

neelneelpurk mentioned this pull request Feb 27, 2026

fix: extension command registration with agy/Antigravity #1704

Open

3 tasks

mnriem requested a review from Copilot February 27, 2026 14:00

Copilot started reviewing on behalf of mnriem February 27, 2026 14:01 View session

Copilot AI reviewed Feb 27, 2026

View reviewed changes

mnriem previously approved these changes Feb 27, 2026

View reviewed changes

refactor: reorder AGENT_CONFIG import statement to the top of the file

d26c592

neelneelpurk dismissed mnriem’s stale review via d26c592 February 28, 2026 15:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: consolidate agent configurations and improve test coverage#1712

refactor: consolidate agent configurations and improve test coverage#1712
neelneelpurk wants to merge 2 commits intogithub:mainfrom
neelneelpurk:fix/common_agent_config

neelneelpurk commented Feb 27, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

mnriem commented Feb 27, 2026

Uh oh!

neelneelpurk commented Feb 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

neelneelpurk commented Feb 27, 2026

Summary

Changes

Impact

Testing

AI Disclosure

Issue

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

mnriem commented Feb 27, 2026

Uh oh!

neelneelpurk commented Feb 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants