[FEATURE] Extensible tag classification model discovery through Entry Points by RoelBollens-TomTom · Pull Request #463 · OvertureMaps/schema

Roel Bollens (RoelBollens-TomTom) · 2026-03-10T23:48:57Z

Extensible tag classification model discovery through Entry Points

This replaces the hardcoded model classification system with tag-based classification model discovery through Entry Points. This is based on #440 by Seth and several schema (ad-hoc) coding sessions where Seth, Vic, Dana, Tristan and Roel participated in.

Model discovery moved into system, eliminating assumptions about Overture in the process. The hardcoded namespace concept ("overture", "annex") and the ModelKind classifier is replaced with tags -- string labels derived by tag providers. Tags become the filtering, grouping, and classification mechanism for model discovery, driven by introspection and package metadata rather than central coordination.

system provides generic tag-based grouping without understanding what any particular tag means. Any package can register tag providers that classify models without special support in the discovery layer.

Purpose

Tags serve three roles:

CLI filtering: select subsets of models for output and codegen (--tag system:feature, --tag draft)
Classification and endorsement: distinguish features from extensions, mark models as vetted or approved by an authority
Marketplace taxonomy: browse and classify models and extensions in a future extension catalog

These roles overlap -- a tag like overture:theme=buildings serves both filtering and taxonomy. The design accommodates this overlap through structured tags that encode both ownership and dimension.

Tag Format

Tags are strings following the pattern [prefix:]key[=value]:

Plain: overture, draft, feature
Prefixed: system:extension -- : separates ownership
Prefixed k/v: overture:theme=buildings

: signals ownership and enables prefix reservation (see Privileged Packages and Tag Reservation). = signals a dimension with a value (groupable via --group-by). One level of each -- no nested colons or multiple = signs.

Minimal launch set

Tag	Meaning
`feature` (was: `system:feature`)	This model is a feature type (has geometry, inherits from Feature
`overture:theme=<theme>`	Which Overture theme this belongs to (e.g., `buildings`, `transportation`)
`overture` (was: `overture:official`)	Placeholder for a lifecycle/endorsement tag — exact name deferred pending Dana and Tristan's work on extension lifecycle

Reserved tags

Tags can be reserved either as simple tags or by namespaces. These are the tags and namespaces that are currently reserved:

Tag	Reserved for use by
`feature`	Tag providers from `overture-schema-system`
`overture`	Tag providers from `overture-schema-core`
`overture:*`	Tag providers from `overture-schema-core`
`system:*`	Tag providers from `overture-schema-system`

Extensions

Additional extensions and accompanied tags will be introduced in a future PR. Extensions allows to augment existing types with new fields (columns).

Tag	Meaning
system:extension	This model is an extension (adds columns/fields to an existing type)

CLI

The list-types command has been updated to support filtering and grouping by tags. Currently, it no longer displays the description or fully qualified class name. The json-schema and validate commands from the overture-schema cli and generate command from the overture-codegen cli have been updated to be able to filter on tags instead of filtering by theme and type. Further changes can be introduced in a future update.

Examples

% overture-schema list-types
address            feature  overture  overture:theme=addresses
bathymetry         feature  overture  overture:theme=base
building           feature  overture  overture:theme=buildings
building_part      feature  overture  overture:theme=buildings
connector          feature  overture  overture:theme=transportation
division           feature  overture  overture:theme=divisions
division_area      feature  overture  overture:theme=divisions
division_boundary  feature  overture  overture:theme=divisions
infrastructure     feature  overture  overture:theme=base
land               feature  overture  overture:theme=base
land_cover         feature  overture  overture:theme=base
land_use           feature  overture  overture:theme=base
place              feature  overture  overture:theme=places
segment            feature  overture  overture:theme=transportation
sources            overture
water              feature  overture  overture:theme=base

% overture-schema list-types --group-by overture:theme
overture:theme=addresses (1)
→ address            feature  overture  overture:theme=addresses

overture:theme=base (6)
→ bathymetry         feature  overture  overture:theme=base
→ infrastructure     feature  overture  overture:theme=base
→ land               feature  overture  overture:theme=base
→ land_cover         feature  overture  overture:theme=base
→ land_use           feature  overture  overture:theme=base
→ water              feature  overture  overture:theme=base

overture:theme=buildings (2)
→ building           feature  overture  overture:theme=buildings
→ building_part      feature  overture  overture:theme=buildings

overture:theme=divisions (3)
→ division           feature  overture  overture:theme=divisions
→ division_area      feature  overture  overture:theme=divisions
→ division_boundary  feature  overture  overture:theme=divisions

overture:theme=places (1)
→ place              feature  overture  overture:theme=places

overture:theme=transportation (2)
→ connector          feature  overture  overture:theme=transportation
→ segment            feature  overture  overture:theme=transportation

% overture-schema list-types --tag overture --exclude-tag overture:theme=base 
address            feature  overture  overture:theme=addresses
building           feature  overture  overture:theme=buildings
building_part      feature  overture  overture:theme=buildings
connector          feature  overture  overture:theme=transportation
division           feature  overture  overture:theme=divisions
division_area      feature  overture  overture:theme=divisions
division_boundary  feature  overture  overture:theme=divisions
place              feature  overture  overture:theme=places
segment            feature  overture  overture:theme=transportation
sources            overture

Deviations

Tag providers are additive only and can't remove existing tags.
The execution order of tag providers is non-deterministic.
There is currently no warning on a tag amount limit
Agreed minimal tag set deviates from coding session outcome to make them less clunky by dropping namespace

Closes #512

Victor Schappert (vcschapp)

Let some comments, but I'm generally aligned and would merge once Roel Bollens (@RoelBollens-TomTom) and Seth Fitzsimmons (@mojodna) are jointly aligned on merging.

Left some thoughts on the AND/OR issue in the CLI, probably above there somewhere. 👆

github-actions · 2026-04-14T16:19:25Z

🗺️ Schema reference docs preview is live!


🌍 Preview	https://staging.overturemaps.org/schema/pr/463/schema/index.html
🕐 Updated	May 06, 2026 19:03 UTC
📝 Commit	e11a1c4
🔧 env SCHEMA_PREVIEW	`true`

Note

♻️ This preview updates automatically with each push to this PR.

Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

Co-authored-by: Seth Fitzsimmons <sethfitz@amazon.com> Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

… filtering logic - Removes overture tag provider (was deferred) - Simplified tags - Reserved tags instead of reserved namespaces - Fixes small issue introduced in earlier commit Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

… CLI commands Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>

`filter_models` selects feature types from the registry through three combinators applied to the same tag grammar (plain `feature`, namespaced `system:extension`, or compound `overture:theme=buildings`): --tag OR defines scope (any-of) --filter AND narrows scope (all-of) --exclude OR-NOT subtracts (none-of) --type OR closed-list match on ModelKey.name (orthogonal) T = ⋃ tag predicates (absent → U) F = ⋂ filter predicates (absent → U) E = ⋃ exclude predicates (absent → ∅) result = (T ∩ F \ E) restricted to type_names if non-empty The mental model is procedural: --tag widens, --filter narrows, --exclude subtracts. Without --tag the scope is every registered model. An empty selector imposes no filtering. A `TagSelector` value object carries the three tag predicates: class TagSelector: include_any: tuple[str, ...] = () require_all: tuple[str, ...] = () exclude_any: tuple[str, ...] = () Field names encode the combinator (any-of / all-of / none-of), deliberately distinct from CLI flag names. Flags are user-facing affordances; field names are implementation-facing and self-document at the call site. `type_names` lives on `filter_models` as a keyword, not on `TagSelector`. It's a closed-list match on `ModelKey.name`, orthogonal to the tag predicate algebra. Isolating it makes `TagSelector`'s purpose statable in one sentence and confines a future fold-in of `--type` to a kwarg deletion that doesn't disturb `TagSelector`. User-facing help text frames flags as acting on feature types ("Include feature types with these tags — defines scope (OR; repeatable)"). Internal API docstrings keep "models" since they describe the Python class layer; "feature types" is the user-facing vocabulary for entry-point-registered top-level types, distinct from the Pydantic models used for nested fields. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>

Use provider_key.name (always a string) instead of provider.__name__, which raises AttributeError when a provider is a callable instance without __name__ — masking the original error inside the except block. Add exc_info=True to preserve the traceback in the warning. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>

Replace unittest.TestCase classes with module-level pytest functions parametrized over the tag lists. Per-tag parametrization isolates failures to the offending input instead of stopping at the first assertion in a loop. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>

Fixes D100 reported by pydocstyle / make docformat. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>

Plain tags, namespaces, and predicates now share a single TAG_PART pattern: lowercase alphanumeric start followed by alphanumeric, hyphen, underscore, or dot. Values remain case-permissive. Drops the prior asymmetry where namespaces and predicates allowed dots but plain tags did not. Make generate_tags private (its sole caller is discover_models) and broaden TagProvider's return type to Iterable[str] so providers can yield, return lists, or return sets. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>

The provider's first argument is the value loaded from an `overture.models` entry point. For discriminated-union features (e.g. `Segment`) that's `Annotated[Union[...], Field(...)]`, not `type[BaseModel]` — the prior signature was a lie. Widen `TagProvider` and the in-tree providers to accept `Any` and document the boundary in `discovery/types.py`. Strip `typing_util.collect_types` to the cases discovery actually meets today: `Annotated`, `Union`/`X | Y`, plain class. Drop the unreached `NewType` and `Literal` branches. Point at `overture-schema-codegen`'s `extraction/type_analyzer.py:analyze_type` as the more capable implementation, with consolidation across system, core, and cli flagged as future work. `theme_provider` extracts the theme via `_theme_literal`, which asserts that `theme` is a single-value `str` `Literal[...]` and raises `TypeError` otherwise. `_generate_tags` catches and logs at WARNING, so third-party model-definition bugs surface visibly without crashing discovery. Promote tag-rejection logging from DEBUG to WARNING so authorization failures (invalid tags, reserved tags, reserved namespaces) don't disappear silently in normal operation. Convert filter tests from direct `_filter_tags` calls to a fake `TagProvider` driven through `_generate_tags`. Tests now exercise provider invocation and merge wiring, not just the filter, and decouple from the private filter name. Provider-behavior tests still call the providers directly. Add discriminated-union coverage for both `feature_provider` and `theme_provider`, plus a `TypeError` case for a non-Literal `theme`. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>

Add Discovery and Tagging sections to system's README, covering the overture.models / overture.tag_providers entry point groups, the tag format, provider contract, namespace and tag reservation, the built-in providers, and TagSelector-based filtering. Update core's README: replace the stale Discovery bullet (discovery has moved to system) with one describing the authority and theme tag providers core contributes. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>

Tag providers now receive the concrete BaseModel subclasses for the entry point instead of the raw entry-point value. _generate_tags walks the model once via collect_types and passes the result to every provider, so providers can't forget to handle discriminated unions and the walk happens once per model rather than once per provider. The TagProvider type alias drops Any in favor of Iterable[type[BaseModel]], honestly typing what providers receive. The first arg of _generate_tags is annotated Any to match the entry-point loader, which yields union expressions that aren't type[BaseModel]. All three registered providers (feature_provider, authority_provider, theme_provider) update to the new signature; unit tests pass concrete classes directly while union-handling tests move to the _generate_tags integration boundary, where the walk now lives. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>

Per discussion in the coding sesh. Signed-off-by: Victor Schappert <schapper@amazon.com>

The module defines ModelKey and TagProviderKey -- key types, not domain models. Rename clarifies intent and avoids confusion with theme model modules elsewhere in the codebase. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>

Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

Click 8.1 introduced typed decorator returns that preserve the TypeVar in `tag_selection_options`, so the lowest-direct mypy job no longer reports `Callable[..., Any]` reassignments. The 8.0 floor predated this and only affected lowest-direct. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>

…lain tag that were missed in the authority_provider removal Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

Roel Bollens (RoelBollens-TomTom) added the change type - cosmetic 🌹 Cosmetic change label Mar 10, 2026

Roel Bollens (RoelBollens-TomTom) force-pushed the discovery-rework-with-tags branch from b8ca2fe to c00022c Compare March 11, 2026 20:31

Seth Fitzsimmons (sethfitz) requested changes Mar 12, 2026

View reviewed changes

Roel Bollens (RoelBollens-TomTom) force-pushed the discovery-rework-with-tags branch from a2461e3 to 61bb58f Compare March 16, 2026 20:27

Roel Bollens (RoelBollens-TomTom) changed the title ~~[WIP] Extensible tag classification model discovery through Entry Points~~ Extensible tag classification model discovery through Entry Points Mar 25, 2026

Roel Bollens (RoelBollens-TomTom) force-pushed the discovery-rework-with-tags branch from b602345 to 1501cc9 Compare March 25, 2026 14:30

Roel Bollens (RoelBollens-TomTom) changed the title ~~Extensible tag classification model discovery through Entry Points~~ [FEATURE] Extensible tag classification model discovery through Entry Points Mar 25, 2026

Victor Schappert (vcschapp) reviewed Mar 25, 2026

View reviewed changes

Roel Bollens (RoelBollens-TomTom) force-pushed the discovery-rework-with-tags branch from 1501cc9 to e5ad4c5 Compare April 14, 2026 16:17

Roel Bollens (RoelBollens-TomTom) temporarily deployed to staging April 14, 2026 16:18 — with GitHub Actions Inactive

Roel Bollens (RoelBollens-TomTom) requested a review from Seth Fitzsimmons (sethfitz) April 14, 2026 16:22

Roel Bollens (RoelBollens-TomTom) marked this pull request as ready for review April 14, 2026 16:22

Copilot AI review requested due to automatic review settings April 14, 2026 16:22

Copilot started reviewing on behalf of Roel Bollens (RoelBollens-TomTom) April 14, 2026 16:22 View session

This comment was marked as outdated.

Sign in to view

Roel Bollens (RoelBollens-TomTom) force-pushed the discovery-rework-with-tags branch from e5ad4c5 to 3daf759 Compare April 14, 2026 17:36

Roel Bollens (RoelBollens-TomTom) temporarily deployed to staging April 14, 2026 17:37 — with GitHub Actions Inactive

Roel Bollens (RoelBollens-TomTom) force-pushed the discovery-rework-with-tags branch from 3daf759 to f19416c Compare April 14, 2026 17:55

Roel Bollens (RoelBollens-TomTom) temporarily deployed to staging April 14, 2026 17:56 — with GitHub Actions Inactive

Roel Bollens (RoelBollens-TomTom) requested a review from Copilot April 14, 2026 18:13

Copilot started reviewing on behalf of Roel Bollens (RoelBollens-TomTom) April 14, 2026 18:13 View session

This comment was marked as outdated.

Sign in to view

Roel Bollens (RoelBollens-TomTom) force-pushed the discovery-rework-with-tags branch from f19416c to c8aa891 Compare April 14, 2026 18:45

Roel Bollens (RoelBollens-TomTom) temporarily deployed to staging April 14, 2026 18:47 — with GitHub Actions Inactive

Roel Bollens (RoelBollens-TomTom) requested a review from Copilot April 14, 2026 18:48

Copilot started reviewing on behalf of Roel Bollens (RoelBollens-TomTom) April 14, 2026 18:48 View session

This comment was marked as outdated.

Sign in to view

Roel Bollens (RoelBollens-TomTom) force-pushed the discovery-rework-with-tags branch from c8aa891 to e9eabf3 Compare April 14, 2026 19:03

Roel Bollens (RoelBollens-TomTom) requested a review from Copilot April 14, 2026 19:03

Seth Fitzsimmons (sethfitz) force-pushed the discovery-rework-with-tags branch from 4c94921 to 3a28d80 Compare May 6, 2026 17:05

Seth Fitzsimmons (sethfitz) temporarily deployed to staging May 6, 2026 17:07 — with GitHub Actions Inactive

Seth Fitzsimmons (sethfitz) mentioned this pull request May 6, 2026

Extensible tag-based model classification and discovery #512

Open

Roel Bollens (RoelBollens-TomTom) and others added 23 commits May 6, 2026 20:46

Refactor model discovery and introduce tags.

13a3ea1

Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

chore(system): use logger.warning instead of print

2ef051a

Co-authored-by: Seth Fitzsimmons <sethfitz@amazon.com> Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

refactor(core): simplify using direct boolean return

5f4ec9b

Co-authored-by: Seth Fitzsimmons <sethfitz@amazon.com> Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

chore(system): update ModelKey docstring to reflect changed attributes

52fe751

Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

chore(system, core): fixes linting/formatting issues

f355be5

Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

refactor(codegen, cli): replace theme filtering with tag filtering in…

309f564

… CLI commands Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

refactor(system): Adds reserved namespaces and logging to tag filtering

77f3acd

Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

refactor(system): tighten discovery api and various small improvements

7ae97ab

Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

Simplify module path in entry point example

cfad066

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>

docs(system): add module docstring to discovery models

ef8c453

Fixes D100 reported by pydocstyle / make docformat. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>

Rename models.py to keys.py

4fed964

Per discussion in the coding sesh. Signed-off-by: Victor Schappert <schapper@amazon.com>

refactor(system): rename discovery models module to keys

25c29e1

The module defines ModelKey and TagProviderKey -- key types, not domain models. Rename clarifies intent and avoids confusion with theme model modules elsewhere in the codebase. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>

remove(core): drop authority_provider

5b34971

Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

chore(system,core,codegen): removes leftover references to overture p…

e11a1c4

…lain tag that were missed in the authority_provider removal Signed-off-by: Roel <75250264+RoelBollens-TomTom@users.noreply.github.com>

Roel Bollens (RoelBollens-TomTom) force-pushed the discovery-rework-with-tags branch from 3a28d80 to e11a1c4 Compare May 6, 2026 19:01

Roel Bollens (RoelBollens-TomTom) deployed to staging May 6, 2026 19:02 — with GitHub Actions View deployment

Roel Bollens (RoelBollens-TomTom) requested review from Seth Fitzsimmons (sethfitz) and Victor Schappert (vcschapp) May 6, 2026 19:28

Conversation

Roel Bollens (RoelBollens-TomTom) commented Mar 10, 2026 • edited by sethfitz Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Extensible tag classification model discovery through Entry Points

Purpose

Tag Format

Minimal launch set

Reserved tags

Extensions

CLI

Examples

Deviations

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Victor Schappert (vcschapp) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🗺️ Schema reference docs preview is live!

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Roel Bollens (RoelBollens-TomTom) commented Mar 10, 2026 •

edited by sethfitz

Loading

github-actions Bot commented Apr 14, 2026 •

edited

Loading