Skip to content

feat: add MTConnect Standard v2.6 + v2.7 support (#133)#139

Open
ottobolyos wants to merge 80 commits intoTrakHound:masterfrom
ottobolyos:feat/issue-133
Open

feat: add MTConnect Standard v2.6 + v2.7 support (#133)#139
ottobolyos wants to merge 80 commits intoTrakHound:masterfrom
ottobolyos:feat/issue-133

Conversation

@ottobolyos
Copy link
Copy Markdown

@ottobolyos ottobolyos commented Apr 25, 2026

Summary

Extends MTConnect.NET to cover MTConnect Standard v2.6 and v2.7 (closes #133). v2.6 + v2.7 type-system + Configuration polymorphism are complete, wire-format end-to-end, and round-trip-tested; the L1-L5 compliance test harness layer is scaffolded with all-version XSDs imported.

v2.7 type-system + Configuration polymorphism

  • Add MTConnectVersions.Version26 and Version27; advance Max to Version27.
  • Regenerate Common DataItems / Components / Configurations + JSON-cppagent formatters from mtconnect/mtconnect_sysml_model tag v2.6 (SHA 08185447bf86…) and v2.7 (SHA 25796ac591bb…).
  • New types in v2.6: AssetAddedDataItem, AssociatedAssetIdDataItem, CuttingTorchComponent, ElectrodeComponent. New enum value MediaType.QIF_MBD.
  • New types in v2.7: BindingState, Depth, FixtureAssetId, SwingAngle/Diameter/Radius, TaskAssetId, WaterHardness DataItems; PinTool, ToolHolder Components; Axis / Origin / Rotation / Scale / Translation configuration primitives (each with Abstract* base + concrete + *DataSet data-set variant); the DataSet representation base; updated Pallet asset measurements; BindingState observation enum.
  • Existing types updated where v2.6 / v2.7 changed docstrings or relationship semantics (AssetChanged description narrowed; Configuration.Relationships now allows asset-to-asset).

SysML importer — polymorphism-aware emission

  • Drop hand-coded primitive type overrides for Motion.Axis / Motion.Origin / CoordinateSystem.Origin / Translation.Rotation / Translation.Translation / SolidModel.Scale. The pre-existing MTConnectPropertyModel.ParseType switch was flattening polymorphic SysML properties (AbstractAxis, AbstractOrigin, etc.) into primitive vec3 types (UnitVector3D, Degree3D). The override pre-dated v2.5's introduction of *DataSet siblings; once v2.5 added <AxisDataSet> etc. as <xs:choice> alternatives to the simple form, the shortcut blocked end-to-end polymorphism. Importer now emits IAbstractAxis, IAbstractOrigin, IAbstractRotation, IAbstractScale, IAbstractTranslation on IMotion / ICoordinateSystem / ITransformation / ISolidModel.
  • Emit all SysML generalizations. v2.7 declares OriginDataSet : DataSet, AbstractOrigin (two <generalization> elements per class — same for AxisDataSet, RotationDataSet, ScaleDataSet, TranslationDataSet). The pre-existing importer captured only one generalization. Now: UmlClass.Generalizations[] collection; heuristic picks the abstract base as the C# class's primary base (since C# does single-inheritance), the rest land as interface inheritance on the matching I* declaration (multi-inheritance for interfaces is fine). After the regen: OriginDataSet : AbstractOrigin, IOriginDataSet, IDataSet; IOriginDataSet : IAbstractOrigin, IDataSet. Same shape across the 5 *DataSet siblings. A same-name self-reference filter prevents the v2.7 XMI's Devices.Device self-loop from emitting a degenerate IDevice : IDevice.
  • Parametrize build/MTConnect.NET-SysML-Import/Program.cs with --xmi / --output / --json-dump CLI args so the generator runs on Linux / macOS / CI. --xmi and --output are mandatory; legacy Windows-paths fallback removed.
  • Fix two Linux-hostile bugs in the importer (case-sensitive template-path lookups; backslash path-separator) that previously made the generator silently no-op on non-Windows.
  • Universal cross-package parent resolver in MTConnect.NET-SysML: when a class in one SysML package extends a class in another (e.g. Devices.Configurations.AxisDataSetObservation.Representations.DataSet in v2.7), the resolver grafts the missing parent into the local namespace under a minimal marker form.
  • Tighten the Result-suffix selector in CSharpTemplateRenderer to require a class type before casting.
  • Null-guard the importer's model factories, fail-fast on a null model, harden the CSharp template renderer against null/cycle/path edges, and null-guard the Json renderer's GetDirectoryName call.
  • Null-guard SysML descriptions and properties; harden XMI parsing against malformed input.
  • Pin XmlResolver=null on the XmiDeserializer XmlDocument loads.

Wire format — end-to-end Configuration polymorphism

  • New hand-authored wire-format classes for the v2.7 leaf types in MTConnect.NET-XML/Devices/, MTConnect.NET-JSON/Devices/, MTConnect.NET-JSON-cppagent/Devices/:
    • 5 concrete vec3-string elements: XmlAxis / XmlOrigin / XmlRotation / XmlScale / XmlTranslation (and Json + Json-cppagent siblings)
    • 5 Entry-keyed DataSet variants: XmlAxisDataSet / XmlOriginDataSet / XmlRotationDataSet / XmlScaleDataSet / XmlTranslationDataSet
  • Updated parent containers (XmlMotion, XmlCoordinateSystem, XmlTransformation, XmlSolidModel and their Json siblings) to handle the polymorphism via parallel optional fields + runtime-type-narrowing dispatch on read; runtime-type dispatch on write.
  • cppagent JSON v2 dialect: simple leaves stay inline on the parent ("Origin": [1.0, 2.0, 3.0] array form per existing convention); DataSet variants use flat-object PascalCase keys ("OriginDataSet": {"X": "1.0", "Y": "2.0", "Z": "3.0"}).

XML wire format — version-coverage extension

  • Extend libraries/MTConnect.NET-XML/Namespaces.cs and Schemas.cs with the v2.4 / v2.5 / v2.6 / v2.7 namespace URIs and schema-location strings. Pre-fix the maps stopped at v2.5 (Namespaces) and v2.3 (Schemas), so probe / current / sample / asset requests at versions ≥ 2.4 produced HTTP 500 from the formatter when it tried to write a null xmlns attribute.

Test infrastructure

  • Rewrite .github/workflows/dotnet.yml: fix paths (src/<lib>/libraries/ + tests/), align SDK to 8.0.x + 9.0.x, target master (was main), upgrade actions to v4, single matrix build-and-test with coverage upload. Trigger restricted to push-to-master + non-draft PRs against master. Pin GitHub Actions to commit SHAs and exclude XsdLoadStrict from the default test filter.
  • Add tests/coverlet.runsettings (shared) + .config/dotnet-tools.json (ReportGenerator pin).
  • Commit tools/{dotnet,test}.{sh,ps1} harness, document the --docker flag/env-var dual API, and rename PowerShell $Args to $DotnetArgs to avoid the automatic-variable shadowing.
  • Scaffold three missing paired test projects: MTConnect.NET-JSON-cppagent-Tests, MTConnect.NET-JSON-Tests, MTConnect.NET-AgentModule-MqttRelay-Tests.
  • Scaffold the layered compliance harness project tests/Compliance/MTConnect-Compliance-Tests/ with L1_XsdValidation/ populated and the L2_XmiOclAssertions/, L3_* (envelope-shape conformance, reserved), L4_CrossImpl/, L5_Regressions/ layers documented in the project README as not-yet-scaffolded follow-ups.
  • Import all 122 official MTConnect XSDs (v1.0 → v2.7) under tests/Compliance/.../Schemas/v*/. Test-only fixtures: bundled into the test assembly via <EmbeddedResource> and consumed by SchemaLoadTests through the manifest API. Not shipped — the compliance project has <IsPackable>false</IsPackable> and no library project references the XSDs.
  • Add parametric SchemaLoadTests that load each XSD via XmlSchemaSet. Tagged [Category("XsdLoadStrict")]; surfaces 54 failures across 5 root causes (XSD 1.1 features + missing xlink imports — neither resolvable with the .NET BCL validator). The default test filter excludes the category; opt in with dotnet test tests/Compliance/MTConnect-Compliance-Tests/MTConnect-Compliance-Tests.csproj --filter "Category=XsdLoadStrict".
  • Add the shared RepoRootLocator helper for repo-root resolution in test projects.

v2.6 / v2.7 unit + compliance + E2E coverage

Common-Tests (87 cases under tests/MTConnect.NET-Common-Tests/V2_6_V2_7/):

  • MTConnectVersionsTests — Version26/Version27 constants, Max == Version27, reflection sweep over all 17 versions, no v1.9 constant.
  • V2_6DataItemTypeTests / V2_6ComponentAndEnumTests — v2.6 additions + AssetChanged description regression pin.
  • V2_7DataItemTypeTests — 8 parametric cases pinning TypeId + Category for every v2.7 DataItem.
  • V2_7ConfigurationDataSetTests — DataSet base + IDataSet, all *DataSet inheritance, concrete leaves, Abstract* abstract-modifier preservation.
  • V2_7ComponentTests — PinTool + ToolHolder.
  • V2_7SampleObservationTests — round-trip coverage for WaterHardness.

XML / JSON / JSON-cppagent round-trip (180 cases) — every Configuration polymorphic leaf round-trips through serialise → deserialise → structural-equality with simple form and DataSet form, plus negative tests for partial-axis machines and <xs:choice> violations.

Compliance — Configuration polymorphism XPath shape (59 cases) in tests/Compliance/.../L1_XsdValidation/ConfigurationPolymorphicXsdValidationTests.cs: 19 polymorphism combinations × 3 versions (v2.5/2.6/2.7) including partial-axis fixtures (Y-only, X+Z-no-Y, A-only, etc.) + 2 negative <xs:choice> cases. Validation is structural (XPath), not schema-driven — the v2.5/2.6/2.7 MTConnectDevices.xsd files use XSD 1.1 features (notably <xs:anyAttribute notNamespace="..."/>) that .NET's XSD-1.0 XmlSchemaSet cannot load; pure structural pinning matches the wire-shape requirement without depending on an XSD-1.1 validator.

E2E — HTTP probe (6 cases) in tests/IntegrationTests/Workflows/ConfigurationPolymorphicHttpProbeWorkflowTests.cs: in-process agent + real MTConnectHttpClient issuing /probe for each polymorphic combination, asserting structural round-trip end-to-end.

Workflows catalog: docs/testing/workflows.md rows W08 / W09 / W10 describe the new E2E paths.

Each test cites the authoritative MTConnect Standard source — XMI tag URL/SHA, XSD URL, prose section — at the test fixture or method comment level.

Documentation

  • docs/testing/v2-6.md + docs/testing/v2-7.md per-version compliance matrices with the new types' status + pinned tests.
  • docs/testing.md (top-level testing topic doc) + docs/testing/workflows.md (workflow catalog) link out to the per-version pages and the harness scripts.
  • build/MTConnect.NET-SysML-Import/README.md documenting the importer's CLI, the determinism guarantee (zero-diff regen against a pinned XMI tag), the cross-package parent resolver, and an "Adding a new MTConnect Standard version" runbook for v2.8 onwards.
  • Root README.md + 21 csproj package descriptions advertise "Supports MTConnect versions up to 2.7." (matches the historical wording style; only the maximum version number changes).

Test result on this PR's tip

Per-project sweep on feat/issue-133 HEAD with Category!=RequiresDocker&Category!=XsdLoadStrict:

Project Tests Pass Fail
Common-Tests 87 87 0
XML-Tests 73 73 0
JSON-Tests 57 57 0
JSON-cppagent-Tests 50 50 0
SHDR-Tests 27 27 0
IntegrationTests 6 6 0
Compliance 59 59 0
Total 359 359 0

Companion PRs

The wider compliance / coverage / dependency program is split across the following PRs so each one stays independently reviewable; this PR depends on none of them.

  • chore/deps-update-2026-04-27 (PR build(repo): bump Scriban + test-infra packages #149) — bumps Scriban 5.9.0 → 7.1.0 to clear 1 critical + 7 high + 3 moderate advisories on the 5.x line, plus the test-infrastructure SDK / NUnit / coverlet bumps Dependabot left out.
  • test/coverage-and-compliance (PR test: comprehensive test-suite overhaul (100% coverage + E2E + compliance harness) #150) — ships the per-DataItem MinimumVersion regen for v2.4-v2.7, fixes the GenerateDevicesXml fileName-argument bug, makes integration-tests port allocation TIME_WAIT-resilient via OS-assigned ephemeral ports, and lays the scaffold for 100 % unit coverage on regenerated .g.cs files, the cppagent parity E2E matrix, round-trip output validation, and full L1-L5 compliance gating.
  • XSD 1.1 + xlink validation — blocked on validator-strategy input (discussion #152). The 54 currently-skipped SchemaLoadTests failures cannot be cleared with the .NET BCL XmlSchemaSet, which is XSD 1.0 only and does not pre-load the xlink schema; the affected XSDs use XSD 1.1 features (assertions, conditional type assignment) and reference xlink:href without an <xs:import>. A working solution needs a third-party XSD 1.1 validator (Saxon-HE is the obvious .NET candidate, but other approaches are viable). No PR opened yet — the choice of validator and how to wire it into the L1 harness is being deferred until upstream weighs in on the preferred approach.

@ottobolyos ottobolyos force-pushed the feat/issue-133 branch 2 times, most recently from c00fc59 to f43d427 Compare April 27, 2026 08:28
@ottobolyos
Copy link
Copy Markdown
Author

ottobolyos commented Apr 27, 2026

I may need to run the SysML import found in the MTConnect.NET-SysML-Import project. But if you are using Claude, etc. I would be curious to see how it would update it on its own.

@PatrickRitchie, I pushed feat/issue-133 to my fork — the v2.6 + v2.7 support is in there end-to-end. Since you asked specifically about how the SysML import got updated to support the new versions, here are the four changes that happened in build/MTConnect.NET-SysML-Import/. Posting them here so you can audit before reviewing the PR proper.

1. CLI flags (replace hardcoded Windows paths)

The importer used to bake three Windows paths in: D:\TrakHound\MTConnect\Standard\v2.5\MTConnectSysMLModel.xml, C:\temp\mtconnect-model.json, and an AppDomain.BaseDirectory-relative output. CI / Linux / macOS couldn't run it. Replaced with three CLI flags:

dotnet run --project build/MTConnect.NET-SysML-Import \
    -- --xmi <path-to-MTConnectSysMLModel.xml> \
       --output <repo-root> \
       [--json-dump <path>]

--xmi and --output are mandatory. --json-dump is optional (replaces the hardcoded C:\temp debug dump). Help text + full guide at build/MTConnect.NET-SysML-Import/README.md including a "Adding a new MTConnect Standard version" runbook for v2.8 and onwards.

2. The F5 flow in Visual Studio — solved with Properties/launchSettings.json

After the CLI change, dotnet run with no args exits with error: --xmi <path> is required. (exit code 2). To keep your "F5 just works" workflow, this PR ships build/MTConnect.NET-SysML-Import/Properties/launchSettings.json with three launch profiles. Full file content (committed verbatim):

{
  "$schema": "https://json.schemastore.org/launchsettings.json",
  "profiles": {
    "Import (env vars)": {
      "commandName": "Project",
      "commandLineArgs": "--xmi \"%MTCONNECT_XMI_PATH%\" --output \"%MTCONNECT_NET_REPO%\"",
      "environmentVariables": {
        "MTCONNECT_XMI_PATH": "set this on your machine — absolute path to the SysML XMI file (e.g. C:\\TrakHound\\mtconnect_sysml_model\\MTConnectSysMLModel.xml)",
        "MTCONNECT_NET_REPO": "set this on your machine — absolute path to the mtconnect.net repo root"
      }
    },
    "Import (sibling clone of mtconnect_sysml_model)": {
      "commandName": "Project",
      "commandLineArgs": "--xmi ../../../../mtconnect_sysml_model/MTConnectSysMLModel.xml --output ../../..",
      "comment": "Assumes mtconnect/mtconnect_sysml_model is cloned as a sibling of this repo. Switch standard version with `git -C ../mtconnect_sysml_model checkout v2.7` (or v2.6 / v2.5)."
    },
    "Import (json-dump enabled, sibling clone)": {
      "commandName": "Project",
      "commandLineArgs": "--xmi ../../../../mtconnect_sysml_model/MTConnectSysMLModel.xml --output ../../.. --json-dump ../../../.cache/mtconnect-model.json"
    }
  }
}

How to use each profile

A. Import (env vars) — works wherever you keep the SysML model on disk; one-time setup, no edits to checked-in files.

Set the two env vars once on your machine.

# Windows (PowerShell), persisted for the user
[Environment]::SetEnvironmentVariable('MTCONNECT_XMI_PATH', 'C:\TrakHound\mtconnect_sysml_model\MTConnectSysMLModel.xml', 'User')
[Environment]::SetEnvironmentVariable('MTCONNECT_NET_REPO', 'C:\source\mtconnect.net', 'User')
# Linux / macOS, persisted in your shell rc file
echo 'export MTCONNECT_XMI_PATH="$HOME/git/mtconnect_sysml_model/MTConnectSysMLModel.xml"' >> ~/.bashrc
echo 'export MTCONNECT_NET_REPO="$HOME/git/mtconnect.net"' >> ~/.bashrc
source ~/.bashrc

Then in Visual Studio: drop-down next to the green Run arrow → Import (env vars) → F5. Rider: Run / Debug Configuration dropdown → same profile name.

B. Import (sibling clone of mtconnect_sysml_model) — works without env vars; only requires that mtconnect/mtconnect_sysml_model is cloned next to mtconnect.net.

cd $(dirname $(pwd))   # one level up from mtconnect.net
git clone https://github.com/mtconnect/mtconnect_sysml_model
git -C mtconnect_sysml_model checkout v2.7   # or any other tag
# Now the directory layout is:
#   <parent>/mtconnect.net/
#   <parent>/mtconnect_sysml_model/

In VS / Rider: select Import (sibling clone of mtconnect_sysml_model) → F5. To regen against a different version, git -C ../mtconnect_sysml_model checkout v2.5 (or v2.6) and press F5 again.

C. Import (json-dump enabled, sibling clone) — same as B plus a debug-only JSON dump of the parsed model at <repo-root>/.cache/mtconnect-model.json. Useful when you suspect the parser is misreading something. The dump path is gitignored via .cache/ exclusion.

Pick the profile that suits your setup. If you'd prefer a different convention (e.g. absolute paths committed in your fork specifically, or a different env-var contract), trivial to adjust — say the word.

3. CI workflow rewrite + tools/ harness

Two non-importer pieces shipped in this PR that the importer work and every downstream PR rely on. Worth flagging because they touch .github/workflows/ and the repo-root tooling layout, neither of which I'd otherwise touch in a per-issue PR.

tools/{dotnet,test}.{sh,ps1} harness

Five small scripts at the repo root (tools/dotnet.sh, tools/dotnet.ps1, tools/test.sh, tools/test.ps1, plus the integration-branch refresh helper). The shell ones target Linux/macOS; the .ps1 siblings target Windows. They wrap the common dev-loop commands (dotnet build/test/format with the right flags + the coverlet runsettings + ReportGenerator hook-up). Pure convenience — running dotnet directly still works fine.

CI workflow rewrite — what was wrong + what changed

The original .github/workflows/dotnet.yml had three independent fatal bugs that meant it was effectively a dead workflow on the current tree:

  1. pull_request: branches: [main] — the repo's default branch is master, so the filter never matched and PR runs never fired.
  2. working-directory: src/MTConnect.NET-Common (and similar for the other 5 jobs) — actual paths are libraries/MTConnect.NET-*. cd to a non-existent directory broke every step.
  3. dotnet test against a library csprojMTConnect.NET-Common.csproj is a library, not a test project. dotnet test on a non-test project discovers zero tests, so even if (1) and (2) had been right, the workflow ran no actual tests.

The rewritten workflow consolidates the 6 per-library jobs into one solution-level job:

  • Triggers: push: branches: [master] + pull_request: branches: [master], types: [opened, synchronize, reopened, ready_for_review]. Job-level if: github.event_name == 'push' || github.event.pull_request.draft == false skips drafts; flipping a draft to ready fires CI immediately on the ready_for_review event.
  • Action versions: actions/checkout@v4, actions/setup-dotnet@v4 (was @v2/@v1; v1/v2 are deprecated, Node 20 EOL on Sept 2026).
  • SDK: 8.0 + 9.0 (was 7.0; libs target net8/9 — net7 SDK couldn't build).
  • dotnet build MTConnect.NET.sln --configuration Debug then dotnet test MTConnect.NET.sln --filter "Category!=RequiresDocker" — solution-level so it covers every library plus every paired test project.
  • Coverage: coverlet via tests/coverlet.runsettings → ReportGenerator → upload TRX + cobertura.xml + HTML report as artifact + surface text summary in the GitHub Actions step summary.

Two trade-offs in the rewrite worth flagging because you may want to revisit them:

  • Per-library fail-isolation lost. The original CI showed MTConnect-NET-Common: failed while HTTP/SHDR/etc. continued; now everything is one job. The TRX artifact preserves per-test detail, but if you'd rather have per-library granularity in the GitHub UI, splitting the matrix back out is straightforward.
  • --configuration Debug instead of Release. Debug builds only net8.0. Release multi-targets all 11 TFMs (net461..net48, netstandard2.0, net6.0..net9.0) — that's what NuGet ships, but the runner doesn't have .NET Framework 4.x dev packs (Linux can't host them) or net6/net7 SDKs. Using Release would fail at restore time on every run. Pragmatic Debug-only fix gives green CI but means a code change that breaks netstandard2.0 compilation isn't caught until next NuGet publish. Cleaner long-term shape: Debug for PR CI (fast feedback), a separate Release job on push-to-master + scheduled (slow but complete, can use Windows runner for the .NET Framework targets). Happy to layer that in once the foundation is merged.

Question — keep #139 as one PR, or split it?

The PR is large. v2.6 + v2.7 type-system regen + the test infrastructure + the importer cross-platform fixes + the CI workflow rewrite + the compliance-harness scaffold + the v2.6 / v2.7 unit tests are all interlinked, but each is reviewable in isolation. Three options:

A. Keep as a single PR. Pro: one merge; the 9 per-issue PRs (#140#148) all rebase against this single base, so the rebase happens once. Con: the diff is large (~900 generated .g.cs files plus 122 vendored XSDs plus the importer + tests + CI work).

B. Split into ~5–7 sequential PRs. A reasonable cut:

  1. chore/test-infrastructure.github/workflows/dotnet.yml rewrite + tests/coverlet.runsettings + .config/dotnet-tools.json + tools/{dotnet,test}.{sh,ps1} harness + paired test-project scaffolding. Lands first; every downstream PR depends on it.
  2. feat/sysml-import-cross-platform — importer Program.cs CLI flags + Linux-hostility bug fixes + Properties/launchSettings.json + cross-package parent resolver + the wildcard-glob template-copy fix. Generator changes only; no .g.cs regen yet.
  3. feat/v2-6-supportMTConnectVersions.Version26, regen against mtconnect/mtconnect_sysml_model@v2.6, docs/testing/v2-6.md matrix, the V2_6*Tests fixtures.
  4. feat/v2-7-supportMTConnectVersions.Version27, regen against @v2.7, docs/testing/v2-7.md, the V2_7*Tests fixtures (including the DataSet representation base + Pallet asset model).
  5. chore/compliance-harness-scaffoldtests/Compliance/MTConnect-Compliance-Tests/ skeleton with the L1 SchemaLoadTests + 122 XSDs imported.
  6. docs/v2-7-readme-bumps — root README + 21 csproj <Description> updates to advertise v1.0–v2.7.

Pros: each ~50–200-line diff, easy to review one at a time. Cons: 6 sequential reviews from your queue, vs one big one; the 9 per-issue PRs would each rebase against #1, then #2, etc. as they land — more rebase churn for me, more "is this branch ready yet?" coordination on your end.

C. Split partial — extract just (1) the CI / tools harness as a tiny prelude PR, keep the rest in #139. Pro: gets the broken master CI fixed even if #139 takes a long review cycle. Con: #139 is still big; only one extra PR to manage.

Reading your stated preference for one PR per task (#127 confirmation), the split (B) is closer to your style. But the 9 per-issue PRs already in flight all depend on #139's tree, and re-coordinating their bases against 6 splits would add a meaningful rebase burden. My read: A or C, with C as the practical compromise if you want quick CI signal but don't want 6 simultaneous reviews. Happy to execute whichever.

Documentation strategy — separate discussion

Opened the documentation-site / docs/index.html question as its own discussion (#153). Out of scope for this PR; a small follow-up will land it once the strategy is decided.

Determinism guarantee

Critical correctness check that I run after every generator change: regen against mtconnect/mtconnect_sysml_model tag v2.5 produces zero diff against HEAD. If a Scriban version, template edit, or parser change shifts output by even a whitespace, this dry-run catches it.

The PR carries:

  • v2.6: 20 file changes (4 new types + 16 modified — docstring tweaks, MediaType enum's QIF_MBD addition, etc.)
  • v2.7: 89 file changes (incl. the *DataSet family + Pallet asset model edits + new DataItems like BindingState/Depth/SwingAngle etc.)

Why the 122 XSD files are committed (and how to drop them)

The PR adds tests/Compliance/MTConnect-Compliance-Tests/Schemas/v*/ containing 122 official XSDs (MTConnectStreams_X.Y.xsd, MTConnectDevices_X.Y.xsd, etc. for v1.0 → v2.7). They sit there as static fixtures consumed by SchemaLoadTests and the per-envelope validation matrix that the XSD-validation follow-up PR will turn on. Three reasons they are committed rather than fetched at test time:

  1. Reproducibility. Tests pin against the exact bytes that were tested. If schemas.mtconnect.org ships a corrected XSD for a given version, the test result changes silently — bad for a regression suite.
  2. Offline + air-gapped CI. actions/checkout is the only network step the runner needs. No hard dependency on schemas.mtconnect.org being up at test time.
  3. Diff visibility on spec updates. When a future XSD revision lands, refreshing the fixtures shows up as an explicit diff under Schemas/, which the maintainer can review side-by-side with any test-side fix.

The size cost is real: 51.5 MB raw across 122 XML files, ~4 MB after git's pack-time deflate compression. Per-file average is ~420 KB raw / ~33 KB packed — the bulk comes from the count, not from any one outlier.

If you'd rather not vendor them, three alternatives, in order of how invasive each is:

  • A. Fetch-and-cache test setup. A [OneTimeSetUp] hook that downloads each XSD from schemas.mtconnect.org into a local cache directory keyed by URL hash. Cache is stored under ~/.cache/mtconnect-xmi-xsd/ (already used for the SysML XMI tags) and committed to .gitignore. Tests pin the expected SHA-256 of each fetched file so a server-side change fails the test loudly. CI flakes on schemas.mtconnect.org outage.
  • B. Submodule against mtconnect/schema. Tests reference XSDs by relative path inside the submodule. Pin the submodule SHA in the repo. Trades commit-size for submodule maintenance overhead.
  • C. Git LFS for the XSDs. Keeps the diff-visibility property of the current setup but moves the bytes out of the main pack. Requires git lfs on every clone, which is friction for new contributors.

My read: the current vendored copy is the right default for a regression-suite use case. If you prefer (A) for hygiene, easy to wire up in a follow-up PR — say the word and I'll do it.

Companion work shipping outside this PR

Two items surfaced during this work that don't fit the per-issue scope of #139:

  • MinimumVersion annotation gap — every regenerated DataItem subclass that doesn't carry an explicit version stereotype in the XMI silently falls back to Version10. Affects 9 existing DataItems too, not just v2.6 / v2.7 additions. The fix reads the UML stereotype and emits a Scriban-conditional MinimumVersion override line. Implemented; ships in a separate companion PR alongside the test-suite expansion.
  • XSD validation across all advertised versions — 122 official MTConnect XSDs imported; 54 / 122 fail to load via the .NET BCL XmlSchemaSet because they use XSD 1.1 features + a missing xlink import. The validator-strategy decision is open as a discussion (How to validate emitted envelopes against XSD 1.1 schemas? #152). Implementation will land as a separate PR once a strategy is agreed. The xlink-import gap is also being filed as a separate report against mtconnect/schema.

Repo-root scripts under tools/ that abstract the developer + CI dotnet
workflow into a single entry point per concern, runnable on Linux,
macOS, and Windows.

  dotnet.sh / dotnet.ps1 — wrap `dotnet` invocations with the project's
  pinned SDK + nuget feed configuration. Idempotent on repeated runs.

  test.sh / test.ps1 — drive `dotnet test` across the solution with
  coverlet coverage collection enabled, the default category filter
  applied, and per-project artifact directories. Forwards extra args
  to `dotnet test` so individual fixtures can be targeted.

  refresh-integration-branch.sh — fetches upstream and origin, resets
  integration/all-fixes to upstream/master, re-merges every in-flight
  plan branch listed in the IN_FLIGHT_BRANCHES variable, optionally
  force-pushes to origin. Designed to run after any feature branch
  advances so downstream consumers always see a single SHA carrying
  every in-flight fix.

  build-for-dime-connector.sh — packs every consumed MTConnect.NET-*
  library from the integration tree into a versioned local NuGet feed
  (~/.nuget/local-mtconnect-net-feed by default), then prints the
  exact NuGet.config + csproj edits the dime-connector consumer needs
  to make to pick up the build. Does not modify the consumer repo.

Coverlet defaults live in tests/coverlet.runsettings (cobertura output,
exclusions for generated .g.cs files and Scriban template internals).
.config/dotnet-tools.json pins reportgenerator so coverage HTML can be
produced from the cobertura output without a global tool install.
Replaces the previous dotnet.yml with a consolidated workflow that
builds the full solution, runs every NUnit + xUnit test project under
the default category filter, and uploads coverlet coverage artifacts.

Key shape:

  - Build step labelled 'Build (Debug)' invokes `dotnet build
    --configuration Debug`. The Debug-only build matches the net8.0
    single-TFM the libraries currently target; Release would require
    additional SDKs not installed on the runner.

  - Trigger only on pushes to master and pull_request events that are
    not in draft state. Draft PRs do not consume CI minutes — the
    flip-to-ready transition is the gate.

  - permissions block locks GITHUB_TOKEN to `contents: read`. The
    workflow only needs checkout + reading the tree to build, test,
    and upload artifacts; it never writes back to the repo.

  - Test runs forward the default category filter
    (Category!=RequiresDocker&Category!=XsdLoadStrict) so the suite
    surfaces the green-by-default set; Docker-requiring and strict-XSD
    suites are explicit opt-in via separate workflow runs.
The importer's entry point baked in three Windows paths (D:\TrakHound\
XMI input, C:\temp debug dump, AppDomain.BaseDirectory-relative output
that broke any non-VS invocation), making it unrunnable on Linux /
macOS / CI.

Replace with three CLI flags:
  --xmi <path>          required — SysML XMI file to consume
  --output <path>       required — repository root
  --json-dump <path>    optional — replaces the C:\temp dump

--xmi and --output are mandatory; the importer fails with a clear
error message when either is missing. Help text and a full usage
guide live in build/MTConnect.NET-SysML-Import/README.md (added in
this commit; see "Documentation" below).

Cross-platform path corrections:

  - Template-path lookups use on-disk casing (CSharp/Templates,
    Json-cppagent/Templates, Xml/Templates) across every Render*()
    method that loads a Scriban template. Previously lowercase
    components no-op'd silently on case-sensitive filesystems
    (ext4 / APFS).

  - Replace literal backslash separators in template id resolution
    with Path-aware splits so module names with dots resolve correctly
    on Linux.

  - csproj copies templates via three forward-slash wildcards (one per
    renderer's Templates/ subtree) instead of 22 explicit per-template
    entries; picks up Devices.Device.scriban which the previous list
    was missing.

VS launchSettings profiles seed an F5 import workflow:

  - 'Import (env vars)' reads MTCONNECT_XMI_PATH and MTCONNECT_NET_REPO
    from the user's shell / system env. The profile no longer carries
    placeholder defaults — earlier drafts populated those variables
    with the literal help-text strings, which then crashed on launch
    as the CLI tried to interpret the documentation as a path. The
    user must set both environment variables before launching the
    debugger; the README documents the failure mode if they are not.

  - Two hard-coded sibling-clone profiles seed concrete --xmi /
    --output paths for users who prefer not to rely on env vars,
    including a json-dump-enabled variant for parser debugging.

Documentation:

  - build/MTConnect.NET-SysML-Import/README.md: full usage guide,
    CLI reference, "Adding a new MTConnect Standard version" runbook,
    generator architecture map, cross-package parent resolver details,
    determinism guarantee, common pitfalls. The runbook walks an
    operator through pinning the SysML XMI tag locally, running the
    importer, diffing the regen, running the test suite, and
    committing per the per-version commit shape.

  - libraries/MTConnect.NET-SysML/README.md: cross-link to the
    importer README so consumers landing on the parser library find
    the codegen entry point.
Every Render*() method previously did

    if (File.Exists(path)) {
        try { template.Parse(...).Render(...) }
        catch (Exception ex) { Console.WriteLine(ex.Message) }
    }
    return null;

which silently no-op'd on a missing template (Linux case-mismatch,
missing CopyToOutputDirectory) and silently logged-then-swallowed any
Scriban parse / render exception. Both modes produced empty .g.cs
output with no operator signal — the regen looks like it worked, the
build later fails with CS0246 cascades.

Wire TemplateLoader.LoadOrThrow through every renderer (13 files,
one per UML element type the importer emits). LoadOrThrow:

  - Throws FileNotFoundException with the absolute path attempted when
    the template is not present, so missing templates fail fast at the
    first renderer that needs them rather than at build time.

  - Returns the parsed Scriban Template instance from a per-process
    cache keyed by absolute path. Each template is parsed at most once
    per importer run, regardless of how many times it is invoked
    against different model elements.

  - Re-raises any Scriban parse exception with the template path in
    the message, so authoring errors surface immediately at parse
    time rather than as silent renders.

The output-path hardening pins every file write to a path that is
guaranteed to be under the --output root (rooted absolute path,
no .. traversal), so a malformed model element name cannot escape
the target tree.
Adds ResolveDanglingParents to MTConnectClassModel and a post-parse
pass in MTConnectModel.Parse that drives it across every Devices.*
Classes list. After the per-package parsers finish, the resolver
scans every parsed class for a generalization target whose name is
absent from the local class set, looks the parent up in the global
XMI by xmi:id, and grafts a freshly-parsed model instance into the
same list under the same idPrefix. Iterates until fixed point.

Membership is matched by xmi:id, not by class name — the authoritative
reference. Multiple UML classes can share a name across packages
(latent in today's XMI; a hard failure the moment two same-named
classes legitimately need to coexist). Each graft is pruned at the
graft point (ParentName + ParentUmlId nulled), so the chain
terminates and the loop self-converges without an iteration cap.

A Result-suffix type guard handles the special case where the
generalization target is a *DataSet companion type — emitted as a
sibling Configuration class with the same idPrefix, recognized by
the trailing 'Result' suffix on the parent name.

Together these two fixes make the SysML importer produce compileable
output for every advertised MTConnect version, including future
versions that introduce new cross-package class hierarchies. Without
them, generalizations that point into another XMI package were
silently dropped, leaving the generated .g.cs without its base class
and the build with CS0246 cascades.
Defence-in-depth against XML External Entity (XXE) resolution. .NET 6+
defaults XmlDocument.XmlResolver to null already, but pinning it
explicitly survives a future framework upgrade and any accidental
restoration of the default XmlUrlResolver (which would fetch DTDs and
external entities over the network at parse time).

Applied to both FromFile and FromXml. The XMI sources we consume are
local files from the mtconnect/mtconnect_sysml_model repo — they
declare no DTDs / external entities — so the change is observably
no-op on the workload but raises the bar for any future XMI variant
that might.
Adds NUnit test projects pairing existing libraries that previously
had no test surface, plus a layered MTConnect Standard compliance
harness scaffold.

  tests/MTConnect.NET-JSON-Tests — sanity coverage for the standard
  JSON formatters library.

  tests/MTConnect.NET-JSON-cppagent-Tests — sanity coverage for the
  cppagent-compatible JSON formatters library.

  tests/agent/Modules/MTConnect.NET-AgentModule-MqttRelay-Tests —
  sanity coverage for the MQTT relay module.

Each new project ships a SanityTests fixture (one assertion: the
project under test exposes a public type) so the test runner picks
up the project and per-project coverage artifacts begin accumulating.

  tests/Compliance/MTConnect-Compliance-Tests — layered compliance
  harness skeleton with one sentinel fixture per layer:

    L1_XsdValidation    — XSD-driven envelope shape compliance
    L2_XmiOclAssertions — XMI/OCL constraint compliance
    L4_CrossImpl        — cppagent parity compliance
    L5_Regressions      — pinned regressions for spec-edge bugs

  L3 is intentionally absent — its semantic-prose-driven tests live
  alongside the libraries they exercise, not in the compliance
  project. README.md in the project root describes the layer
  taxonomy and documents the opt-in XsdLoadStrict category for
  XSD 1.1-feature suites that the .NET BCL validator cannot consume.

Solution file picks up the seven new projects under their existing
solution folders.
  Schemas/v1_0/ through Schemas/v2_7/ — 122 XSD files mirrored from
  schemas.mtconnect.org via curl. Every (envelope-kind × version)
  tuple the standard publishes is present (v1.0 / v1.1 / v1.2 each
  publish a single XSD per kind; v1.3+ also publish a _1.0
  schemaVersion variant).

  L1_XsdValidation/SchemaLoadTests.cs — parametric NUnit test that
  loads each XSD into XmlSchemaSet and asserts compilation succeeds.

Tagged [Category("XsdLoadStrict")] because the suite surfaces 54
failures across 5 root causes:

  - notNamespace attribute (XSD 1.1 only)
  - maxOccurs > 1 on <xs:all> particles (XSD 1.1 only)
  - unresolved xlink:href / xlink:role / xlink:title imports
  - duplicate type definitions (legitimate XSD 1.0 override pattern
    that the .NET BCL validator declines)
  - assert / alternative elements (XSD 1.1 only)

The category opts the suite out of the default test filter; the
running shape is `dotnet test --filter Category=XsdLoadStrict` for
explicit verification of strict-XSD coverage. The default filter
keeps these red — the failures pin currently-incompliant surfaces
the campaign will close out in later phases.

Defence-in-depth: the SchemaLoadTests reader and XmlSchemaSet both
pin XmlResolver=null, so loading an XSD never fetches external
<xs:include> / <xs:import> URIs over the network. The shipped
Schemas/ tree is fully self-contained, but a future XSD that
imports the W3C xlink schema by URL would otherwise hit the network
at test time. When the L1 layer starts validating xlink-importing
envelopes, the right move is to pre-seed an XmlPreloadedResolver
with a local copy of the xlink XSD — not to re-enable the default
XmlUrlResolver.
  libraries/MTConnect.NET-Common/MTConnectVersions.cs: add
  Version26 = new Version(2, 6); flip Max => Version26.

Regenerated against mtconnect/mtconnect_sysml_model tag v2.6
(SHA 08185447bf86201160b8fa091e255f024655dbbb).

New types:
  - DataItems: AssetAdded (split out from AssetChanged),
    AssociatedAssetId.
  - Components: CuttingTorchComponent, ElectrodeComponent.

Modified types (docstring + structural updates per v2.6 spec):
  - AssetChangedDataItem — description trimmed to v2.6 wording,
    no longer claims to also fire on asset add events (v2.6 split).
  - Multiple Pallet measurement types, Configuration relationship
    types, MediaType + Units descriptions — propagated XMI updates.

JSON-cppagent formatters regenerated against the same v2.6 XMI:
JsonComponents picks up CuttingTorch + Electrode entries;
JsonEvents picks up AssetAdded + AssociatedAssetId entries.

Generation re-run is byte-deterministic against the same input XMI:
re-running the importer against the same XMI SHA produces zero diff.
  libraries/MTConnect.NET-Common/MTConnectVersions.cs: add
  Version27 = new Version(2, 7); flip Max => Version27.

Regenerated against mtconnect/mtconnect_sysml_model tag v2.7
(SHA 25796ac591bbe90018919fc9ccf757b4be148a92).

New types:
  - DataItems: BindingState, Depth, FixtureAssetId, SwingAngle,
    SwingDiameter, SwingRadius, TaskAssetId, WaterHardness.
  - Components: PinTool, ToolHolder.
  - Configurations: Axis / AxisDataSet, Origin / OriginDataSet,
    Rotation / RotationDataSet, Scale / ScaleDataSet,
    Translation / TranslationDataSet, plus their Abstract*
    base classes and DataSet companion type.
  - Observation events: BindingState event observation.

JSON-cppagent formatters regenerated against the same v2.7 XMI:
JsonComponents picks up PinTool + ToolHolder; JsonEvents picks up
the new event entries; JsonSamples picks up the new sample types.

Generation re-run is byte-deterministic against the same input XMI:
re-running the importer against the same XMI SHA produces zero diff.
42 tests across 7 fixtures under tests/MTConnect.NET-Common-Tests/V2_6_V2_7/.
Each test cites the authoritative MTConnect Standard source — XMI tag
URL/SHA, XSD URL, prose section — at the test fixture or method
comment level, so a future reader debugging a regression knows what
the spec authority is for the asserted behavior.

  MTConnectVersionsTests             (5 tests)
    - Version26 / Version27 constants equal the expected Version values
    - Max getter advances to Version27
    - All-versions enumeration includes v2.6 and v2.7
    - Version comparison ordering holds across the new constants

  V2_6ComponentAndEnumTests          (4 tests)
    - CuttingTorchComponent + ElectrodeComponent expose expected
      TypeId values; component subtype hierarchy holds.

  V2_6DataItemTypeTests              (5 tests)
    - AssetAdded + AssociatedAssetId DataItem TypeIds and subtypes
      match v2.6 XMI; AssetChanged description split confirmed.

  V2_7ComponentTests                 (4 tests)
    - PinTool + ToolHolder TypeIds + subtype hierarchy.

  V2_7ConfigurationDataSetTests      (10 tests)
    - Axis / Origin / Rotation / Scale / Translation Configuration
      types + their DataSet companions parse / serialize correctly.

  V2_7DataItemTypeTests              (8 tests)
    - BindingState, Depth, FixtureAssetId, SwingAngle, SwingDiameter,
      SwingRadius, TaskAssetId, WaterHardness — TypeId + subtypes.

  V2_7SampleObservationTests         (6 tests)
    - Sample observation envelope shape for the new sample types.

Every assertion's source citation references the public spec sources
directly (XMI tag URL, XSD URL, Part_*.0 prose); no internal-tracker
references.
The 21-csproj sweep that previously updated <Description> fields to a
"Supports MTConnect Versions up to 2.6" wording is reframed to span
the full supported range:

  "Supports MTConnect Standard versions v1.0 through v2.7"

The range framing communicates that older MTConnect clients keep
working alongside modern v2.7 emitters; the previous "up to" phrasing
read as a ceiling rather than a continuous range.

  - Root README.md: matching range phrasing in the supported-versions
    callout.

  - libraries/MTConnect.NET/README-Nuget.md: NuGet README for the
    umbrella package — was "Supports MTConnect Versions up to 2.4" /
    "compatible up to the latest MTConnect v2.4". Brought in line with
    the root README and the csproj descriptions.

  - libraries/MTConnect.NET-SysML and agent/Modules/.../ShdrAdapter:
    these two csprojs were missed by the earlier sweep (the SysML
    csproj was at "v2.3", the ShdrAdapter csproj at "v2.2"). Both
    csprojs ship surface that supports the v2.7 wire shape — the
    SysML library parses v2.7 XMI, the ShdrAdapter module relays
    every observation type the agent emits — so they correctly
    advertise the same range as the rest of the libraries.

Total: root README + 21 csproj <Description> fields + NuGet README,
all carrying the same supported-version phrasing.
Two per-version compliance matrices under docs/testing/, modelled on
the cppagent compliance-matrix pattern. Each enumerates the
DataItems / Components / enum values / configuration types that the
target MTConnect Standard version introduced or modified, with
status + pinned-test column populated from
tests/MTConnect.NET-Common-Tests/V2_6_V2_7/.

  docs/testing/v2-6.md — v2.6 matrix:
    - DataItems: AssetAdded, AssociatedAssetId, AssetChanged
      description split.
    - Components: CuttingTorchComponent, ElectrodeComponent.
    - Constants: Version26, Max => Version26 (advanced to Version27
      in the next release boundary).

  docs/testing/v2-7.md — v2.7 matrix:
    - DataItems: BindingState, Depth, FixtureAssetId, SwingAngle,
      SwingDiameter, SwingRadius, TaskAssetId, WaterHardness.
    - Components: PinTool, ToolHolder.
    - Configurations: Axis / AxisDataSet, Origin / OriginDataSet,
      Rotation / RotationDataSet, Scale / ScaleDataSet,
      Translation / TranslationDataSet, plus their Abstract*
      bases and the DataSet companion type.
    - Constants: Version27, Max => Version27.
    - XsdLoadStrict opt-in incantation inlined for the strict
      schema-load test category.

Per-version compliance matrices are objective records of what the
library exposes (DataItem TypeIds, Component types, enum members,
constants) per MTConnect Standard version — readable independently
of any in-flight plan or PR.
The new public symbols introduced by this PR (Version26, Version27,
the bumped Max getter) lacked <summary> XML-doc, so IntelliSense
hovering them in a consumer codebase showed no description. Each
new constant's summary briefly enumerates what its target MTConnect
Standard version introduced, so a consumer choosing a version
constant can pick the right one from hover alone.

MTConnectModel and its Parse entry point — part of the SysML
importer's public surface (the build/MTConnect.NET-SysML-Import
generator's first call) — also lacked <summary> XML-doc on hover.
Add a class-level summary explaining what the model represents and
a method-level summary on Parse explaining input / output /
null-return semantics, with a cref to the cross-package parent
resolver it runs as a post-parse pass.
@ottobolyos ottobolyos force-pushed the feat/issue-133 branch 4 times, most recently from 640d54f to 1b03808 Compare April 28, 2026 19:41
@ottobolyos ottobolyos marked this pull request as ready for review April 29, 2026 16:46
@ottobolyos
Copy link
Copy Markdown
Author

ottobolyos commented Apr 29, 2026

@PatrickRitchie — flipping this one to ready for review. Below is a sequencing checklist for the 13 PRs covering issues #127#138 + #154 plus a pointer to an integration branch I've been using as a smoke-check.

I've got 13 PRs open against MTConnect.NET covering issues #127#138 + #154. They're independent in scope but their integration order matters for rebase churn. Here's a checklist for sequencing review — feel free to tick each off as it lands so we both have visibility on progress.

I've also assembled an integration branch on my fork that has all 13 PRs merged together, so the union builds and tests cleanly end-to-end before any of them lands upstream:

  • Integration branch (all 13 PRs together): https://github.com/ottobolyos/mtconnect.net/tree/integration/all-fixes
    • rebuilt by a small refresh script after every per-branch push
    • default dotnet test filter is green; RequiresDocker E2E suites (Mosquitto + cppagent parity, via Testcontainers) also pass when MTCONNECT_E2E_DOCKER=true is exported
    • useful as a smoke-check that the per-PR diffs do not collide once they all land

If you'd like to test-drive the union before approving each PR individually, that branch is the place.

Code surfaces are mostly disjoint between the per-issue PRs. Rebasing each against master after a sibling merges is a no-op in the typical case.

One ordering dependency: PR #146 (fix/issue-132) carries one extra commit on top of PR #147 (fix/issue-130-131)'s MTConnectAgent.cs changes — both touch the same auto-injection block. Landing PR #147 first means PR #146 rebases naturally; landing PR #146 first leaves PR #147 with one trivial fixup. Either works.

Open questions blocking related follow-up work

Three items I'd like your steer on before opening the corresponding PRs — each is its own discussion thread so we can converge separately:

  • XSD 1.1 / xlink-aware envelope validation — discussion #152. 54 of 122 published MTConnect XSDs need an XSD 1.1 validator; .NET BCL only does XSD 1.0. Asking which library / approach you'd accept.
  • Documentation site / docs/index.html — discussion #153. The 12-byte HELLO WORLD placeholder suggests an unfinished docs-site setup. Asking whether you'd like a static site (VitePress / Docusaurus / DocFX), a docs/index.md hub page, or just deletion.
  • Generator improvements + code-quality cleanup + performance pass — three internal-hygiene plans that span the codebase (e.g. wire MaximumVersion deprecation marking into the SysML importer; resolve the IPNetwork BCL collision + the obsolete MQTTnet TLS API call sites; cache hot-path lookups). Each would land as its own PR. I haven't opened them because they're "would you like this?" rather than "fix this issue" — I'd rather scope each with you first than push speculative work. Happy to draft separate discussions for each if useful, or tackle them piecemeal as you greenlight individual items.

What this isn't

  • Not a strict requirement — any merge order produces the same end state.
  • Not a deadline ask — happy to follow your review cadence.
  • Not coupled to any review tooling on my end — every PR is independently reviewable on GitHub's web view.

If a different order or batching shape works better for your queue, just say.

@ottobolyos ottobolyos marked this pull request as draft April 29, 2026 17:46
@ottobolyos ottobolyos force-pushed the feat/issue-133 branch 2 times, most recently from 082b60f to 9fc8c5f Compare April 30, 2026 05:52
Uncomment the seven branches that were marked as blocked / awaiting
agent-completion when the script was first scaffolded. All eleven
in-flight branches now exist on origin and merge cleanly into integration
in the documented deterministic order (foundation → per-issue numeric
ascending → cross-cutting). Add test/coverage-and-conventions to the
list. Add an inline comment at the top of the array explaining the
canonical ordering.
The XML wire-format library declared MTConnect namespace URIs only up to
v2.5 in Namespaces.cs and only up to v2.3 in Schemas.cs, while
AgentConfiguration's DefaultVersion advances with MTConnectVersions.Max
(now v2.7). When a client requests probe / current / sample / asset
output for an advertised version that the library does not have a
mapping for, Namespaces.GetDevices and friends return null and the XML
serializer fails when writing the xmlns attribute, surfacing as HTTP
500 from the agent's response handler.

Add the missing entries:

- Namespaces.cs: cases 6 + 7 in GetDevices / GetStreams / GetAssets /
  GetError, with matching Version26 + Version27 inner classes.
- Schemas.cs: cases 4-7 in GetDevices / GetStreams / GetAssets /
  GetError (v2.4 + v2.5 were already absent before the v2.6 / v2.7
  bump), with matching Version24-Version27 inner classes.

The schemaLocation strings reuse the canonical https schemas.mtconnect.org
URLs and the matching XSD filenames the v2.4-v2.7 specs publish.
The tests-plan branch was originally named with a 'conventions' suffix
that overlaps semantically with this campaign's internal-only
convention rule-book and reads as a dangling reference on the public
PR list. Pick a name that captures the two most concrete deliverables
of the plan (100% coverage gate + L1-L5 compliance harness) instead.
ottobolyos and others added 25 commits April 30, 2026 10:41
Row 3: `umlClass.Comments?.FirstOrDefault().Body` NRE'd when Comments was
non-null but empty (FirstOrDefault returns null, `.Body` then dereferences
null). Chain `?.` through the FirstOrDefault: `?.FirstOrDefault()?.Body`.

Row 16: `umlClass.Properties?.Where(o => !o.Name.StartsWith(...))` NRE'd
on any property whose Name was null. The outer `?.` only protects the
collection. Guard `o.Name != null` per element.

Row 17: `xRoot.ElementName = xDoc.DocumentElement.LocalName` NRE'd on an
XmlDocument loaded from a fragment with no root. Throw
InvalidOperationException with context instead.

Row 18: Honour `CancellationToken` — call `ThrowIfCancellationRequested()`
at entry and after the synchronous XmlSerializer construction so callers
get a cooperative abort point. Prefer honouring over dropping since the
public method signature is part of the SysML library NuGet surface.

Row 19: `ResolveDanglingParents` — drop the `while (true)` wrapper (the
docstring explains a single pass converges; the loop was dead defence).
Build `localUmlIds` once before the loop and mutate it as grafts land so
the existence check is O(1). Dedupe via `HashSet.Add` instead of
GroupBy/First.

Row 51: Wrap `XmlDocument.Load` / `LoadXml` in `XmlReader.Create(...)`
with `XmlReaderSettings { DtdProcessing = DtdProcessing.Prohibit,
XmlResolver = null }`. .NET 6+ defaults are safe but pinning survives a
future framework downgrade and refuses billion-laughs DoS unconditionally.
Per GitHub's hardening guidance, third-party actions should be
referenced by commit SHA rather than tag — tags are mutable and a
compromised maintainer can retag a malicious commit. Pin
`actions/checkout`, `actions/setup-dotnet`, and
`actions/upload-artifact` to the v4 release commit SHAs and
annotate with a `# v4` trailing comment so Dependabot updates
remain reviewer-friendly.
Both V2_6_V2_7 fixture summaries cited the branch `feat/issue-133` —
that branch disappears after merge, leaving a dangling reference.
Swap for the stable issue link
(`[TrakHound#133](https://github.com/TrakHound/MTConnect.NET/issues/133)`).
Row 20: Program.cs — `MTConnectModel.Parse(xmiPath)` may return null; the
renderers below internally null-check and silently no-op, producing zero
output and exit 0. Surface the parse failure with a stderr message and
non-zero exit before any rendering runs.

Row 21: Json-cppagent renderer — `Path.GetDirectoryName(resultPath)` may
return null/empty when `outputPath` is a bare relative path (e.g.
`--output .`). `EnsureDirectory` then throws ArgumentException. Fall back
to `.` when the directory comes back null.
The csproj <Description> was bumped to advertise v2.7 support; the
NuGet README was not. Add a one-line note that v2.7 XMI parses and
that the new `ResolveDanglingParents` helper handles cross-package
parents.
Row 47: V2_7DataItemTypeTests — `Activator.CreateInstance(dataItemType)`
may throw MissingMethodException if a v2.7 DataItem lacks a parameterless
constructor. The next `Assert.That(instance, Is.Not.Null)` never fires;
failure surfaces as a bare exception with no test-friendly diagnostic.
Wrap the call in `Assert.DoesNotThrow` with the offending type name in
the message so triage is one-look.
`--compliance` runs every test in the compliance project including
the XSD-1.1 strict-load category that's filtered out of the default
sweep. Add a NOTE to both `tools/test.sh` and `tools/test.ps1`
help text so a contributor opting in knows to expect ~54 failures
until the XSD-1.1 validator follow-up lands.
The --docker flag and MTCONNECT_DOTNET_USE_DOCKER env var on
tools/dotnet.{sh,ps1} look like duplicate spellings of one knob,
but the env-var form exists specifically to support the nested
wrapper chain: tools/test.{sh,ps1} reads its --docker flag once,
exports the env var, then every per-project dotnet invocation
through tools/dotnet.{sh,ps1} picks it up automatically without
test.{sh,ps1} having to splat --docker per call site.

Add a paragraph to all four tool headers explaining the dual API
and why removing either form would break the nested-call pattern.
No behaviour change.
CI uploads only TestResults/**/coverage.cobertura.xml; the opencover
emit is unconsumed and doubles per-test coverage-write I/O. Drop it
to save ~5-10% test-time wall on coverage-heavy runs and proportional
disk in TestResults/.

If a tool ever needs opencover, re-add via the comma-separated <Format>
list.
The 122 spec-XSDs (~52 MB) introduced in this PR were declared as
<None Include="Schemas\**\*.xsd"><CopyToOutputDirectory>PreserveNewest>,
which copied the entire tree into bin/<Cfg>/<TFM>/Schemas/ on every
build. Multi-target Release/Package builds duplicated this across 4
TFMs (~208 MB total per clean build) and inflated CI artefact caches
by the same factor, on top of 122 filesystem-stats per incremental
rebuild.

Switching to <EmbeddedResource> bundles the XSDs into the test
assembly's manifest. SchemaLoadTests now enumerates them via
Assembly.GetManifestResourceNames() and reads each via
GetManifestResourceStream(). The on-disk Schemas/ tree under bin/ is
gone; the only artefact is the test DLL itself.

Display-path mapping (v2_7/MTConnectDevices_2.7.xsd) is recovered by
splitting the resource name on the first dot after the namespace
prefix — the on-disk convention of v<major>_<minor> directory naming
keeps that split unambiguous, regardless of how many dots the XSD
filename contains.

Failure pattern preserved exactly: 54 fail / 68 pass / 122 total,
matching the documented xlink + XSD-1.1 gaps the strict-load category
surfaces. The follow-up XSD-1.1 PR will swap XmlResolver=null for a
manifest-aware resolver to close those failures.

Closes Row 58 of the PR-139 review ledger.
Sweep four prose patterns across review-fix comments, tooling, and
test fixtures: ledger-row references ('(row N)'), sibling-symbol
pointers ('Mirror ClassModel.Create's guard', 'See FromFile for
rationale'), British-spelling drift in commentary and prose strings,
and self-referential meta ('this PR is the one that advances Max',
'These tests exist to make any future change visible'). Each comment
is rewritten to describe the invariant or failure mode inline, in
present tense, in American English, without naming siblings or prior
designs.

Behaviour is unchanged; only comment text and prose strings changed.

Conversions of note:
  - Defence -> Defense, Honour -> Honor, behaviour -> behavior,
    catalogue -> catalog, honoured -> honored, cancellation token ->
    cancelation token (in prose only; identifiers retain canonical
    .NET form).
  - 'this PR is the one that advances Max to Version27' ->
    'Locks Max to Version27 against accidental rollback'.
  - 'no 1.9 constant has crept in' ->
    'Pin that no Version19 constant exists'.
  - Removed '[TrakHound#133](...)' decoration link from the file-level header.
  - 'Pairs with tools/dotnet.sh' / 'matches the bash sibling' ->
    rewritten to describe what each script does in present tense.
The fail-fast comment in Program.cs referenced 'Row 20' — a row in a
gitignored review-findings ledger that no public reader can resolve.
Forbidden under CONVENTIONS §14c (every spelling: parenthesised,
capitalised, lower-cased, with or without a colon, with or without #).

Drop the row reference; the prose still describes what the guard does
and why (renderers internally null-check + silently no-op, producing
zero output and exit 0; surface the parse failure here).
The "Adding a new MTConnect Standard version" runbook prescribed a
`feat/v<NN>` (or `feat/issue-NNN`) branch-name pattern. Per-PR feature
branches are temporary working space owned by whoever ships the
expansion; the SysML-importer doc has no business pinning a naming
scheme that doesn't survive the PR landing. Drop the section.
The importer collapsed every generalization to a single base, then
applied hand-coded type overrides that flattened polymorphic property
types to their concrete leaves. That output could not represent
Configuration's Axis/Origin/Rotation/Scale/Translation hierarchy or
the matching DataSet variants — readers saw concrete leaves where the
schema declares abstract bases, blocking compliance with the
v2.5/2.6/2.7 Devices XSDs.

Drop the type-override table so generated property types follow the
SysML model. Emit every generalization the SysML graph carries: the
primary as the C# class base, and every additional generalization as
interface inheritance on the generated class. Together this produces
Abstract* base types whose leaves participate by inheritance, which
the wire-format and test layers depend on for round-trip and XSD
validation.
The compliance gap for v2.5/2.6/2.7 MTConnectDevices XSDs requires
Configuration's Axis/Origin/Rotation/Scale/Translation properties
(and the matching DataSet variants) to expose abstract bases rather
than concrete leaves, so the schema's Abstract* hierarchy survives
serialize/deserialize round-trips. The model layer and every wire
format must move together — a half-step state where the model
exposes IAbstract* but the wire layer reads/writes the concrete
leaves does not compile.

Regenerate the model layer:
  - Motion, CoordinateSystem, Transformation, SolidModel, and the
    Configuration DataSet family now declare property types as
    IAbstractAxis, IAbstractOrigin, IAbstractRotation,
    IAbstractScale, IAbstractTranslation, plus AbstractDataSet
    variants. Concrete leaves (UnitVector3D, Degree3D, AxisDataSet,
    OriginDataSet, RotationDataSet, ...) participate by interface
    inheritance from the matching Abstract* bases.

Update all three wire formats to match:
  - XML: read/write each abstract base via element-name dispatch
    against the SysML-derived concrete leaves.
  - JSON: same dispatch through System.Text.Json converters
    that preserve the abstract-base typing on round-trip.
  - JSON-cppagent: same dispatch in the cppagent v2 dialect's
    flattened envelope.

Each layer also handles the Axis/Origin/Rotation DataSet variants
that the v2.5+ schemas mandate.

Align V2_7 DataSet assertions with the new shape: tests now pin the
abstract-base type on each property and the interface-inheritance
chain on every leaf, matching the regenerated surface exactly.
The previous header carried 'dotnet test --settings tests/coverlet.runsettings'
inside an <!-- ... --> block. XML disallows '--' anywhere inside a comment,
so vstest rejected the runsettings on every test invocation with
"Settings file provided does not conform to required format. An XML
comment cannot contain '--'" — silently disabling coverage collection
and the include/exclude filters across the whole test sweep.

Reword the prose to drop the inline command and reflow block formatting
so the file parses cleanly. The substantive runsettings (collector,
filters, attributes) are unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Exercise the XML wire-format's polymorphic Axis/Origin/Rotation/Scale/
Translation hierarchy and the matching DataSet variants end-to-end:
each round-trip goes through serialize -> XSD validate -> deserialize
and asserts both the abstract-base type and the concrete leaf survive
intact. Validation runs against v2.5, v2.6, and v2.7 MTConnectDevices
schemas.

Also covers the previously untested CoordinateSystem.Description
write branch — the writer's nullable-string handling is exercised
with both populated and missing Description values.
Adds five round-trip fixtures under tests/MTConnect.NET-JSON-Tests/Devices/
Configurations covering simple-leaf and DataSet variants for the Json wire
format. Each fixture pins:

- Positive: serialize a parent built from the data model with the simple
  leaf or the DataSet variant and assert the produced JSON contains the
  expected camelCase keys; deserialize back through ToMotion / ToCoordinate
  System / ToTransformation / ToSolidModel and verify the I*DataSet narrowing.
- Negative: null input on the wire-format ctor keeps default field values,
  parameterless ctor produces sensible defaults, both simple and DataSet
  present in the same parent — DataSet wins on narrow.

Each fixture's class-level XML doc-comment cites the SysML XMI tag URL +
v2.7 XSD URL + UML class names per the spec-source-reference rule on every
spec test.

Adds a small TestHelpers/JsonRoundTripHelper.cs that wraps System.Text.Json
with the project-wide options (DefaultIgnoreCondition.WhenWritingNull, so
absent properties don't emit JSON null payloads).
Adds five round-trip fixtures under
tests/MTConnect.NET-JSON-cppagent-Tests/Devices/Configurations covering the
cppagent v2 JSON shape:

- Simple leaves (Axis, Origin, Rotation, Scale, Translation) serialize as
  IEnumerable<double> numeric arrays inline on the parent — e.g.
  "Origin": [1, 2, 3].
- DataSet variants (AxisDataSet, OriginDataSet, RotationDataSet,
  ScaleDataSet, TranslationDataSet) serialize as flat PascalCase objects
  with the spec-defined keys — e.g. "OriginDataSet": {"X": "1", "Y": "2",
  "Z": "3"} or "AxisDataSet": {"X": 1, "Y": 2, "Z": 3} when the underlying
  DataSet uses doubles.

Each fixture pins both positive (round-trip preserves the leaf-type
narrowing) and negative cases (null property emits no field, both-present
narrows to DataSet, parameterless ctor + null-input ctor produce sensible
defaults). Class-level XML doc-comments cite the SysML XMI tag URL +
cppagent reference URL per the spec-source-reference rule.

Adds a small TestHelpers/JsonRoundTripHelper.cs that wraps
System.Text.Json with the project-wide options.
Adds tests/MTConnect.NET-Common-Tests/Devices/Configurations/
PolymorphicLeafCoverageTests.cs — a single fixture that walks the
MTConnect.NET-Common assembly via reflection to pin the v2.7 Configuration
polymorphic-leaf shape. The fixture asserts:

- Each IAbstract<Leaf> interface (IAbstractAxis, IAbstractOrigin,
  IAbstractRotation, IAbstractScale, IAbstractTranslation) has exactly two
  concrete implementing classes in the assembly.
- The simple variant (Axis, Origin, Rotation, Scale, Translation)
  implements I<Leaf> and inherits IAbstract<Leaf>, but does NOT implement
  IDataSet.
- The DataSet variant (AxisDataSet, OriginDataSet, RotationDataSet,
  ScaleDataSet, TranslationDataSet) implements I<Leaf>DataSet AND
  IAbstract<Leaf> AND IDataSet.
- Both variants construct via a public parameterless ctor.
- Both variants extend an Abstract* abstract base class.
- The IAbstract<Leaf> marker interfaces declare no members (a regen that
  promoted a member would couple consumers to one variant's shape and
  break LSP).

Cases run as TestCaseSource rows so a future leaf added to the polymorphic
surface only needs a single row added to LeafShapes.
Cross-version XSD validation pins every generated Configuration
polymorphic combination (concrete Axis/Origin/Rotation/Scale/
Translation leaves and DataSet variants) against the v2.5, v2.6, and
v2.7 MTConnectDevices schemas. Validation is XSD-1.1-only because
v2.5/2.6/2.7 MTConnectDevices XSDs use 1.1 features (assertions,
xs:override) that .NET's XmlSchemaSet cannot load — the L1 layer's
SchemaLoadTests confirm that load failure deterministically, and
these tests are tagged XsdLoadStrict so the default test filter skips
them when run without an XSD-1.1-capable validator.
Spin up the agent end-to-end, populate Configuration with each
polymorphic DataSet variant (AxisDataSet, OriginDataSet,
RotationDataSet), and assert the HTTP probe response round-trips
the abstract-base typing through serialize -> network -> deserialize
without narrowing to the concrete leaf. The fixture stabilises
against the shared-broker race by using a per-test broker port,
so parallel test runs no longer flake on bind contention.

Also documents the W08/W09/W10 workflow rows that describe the E2E
fixture's preconditions, assertions, and tear-down sequence.
Cross-version structural pin for every generated Configuration
polymorphic combination (concrete Axis/Origin/Rotation/Scale/
Translation leaves and DataSet variants) against produced XML shaped
per the v2.5, v2.6, and v2.7 MTConnectDevices XSDs.

XPath-based validation (not schema-driven): the published XSDs use
XSD 1.1 features (notably <xs:anyAttribute notNamespace="..."/>) that
.NET's XSD-1.0 XmlSchemaSet cannot load. The L1 SchemaLoadTests
fixture confirms that load failure deterministically. Pure structural
pinning matches the wire-shape requirement without depending on an
XSD-1.1 validator that does not exist on .NET BCL.

Coverage covers full-3-key shapes plus partial-axis combinations that
real machines legitimately emit per the XSD's <xs:sequence minOccurs='0'
maxOccurs='3'/>: 2D mills with no Z-axis (X+Y only), single-axis rotation
(A or B or C only), and empty <OriginDataSet></OriginDataSet> for an
unset origin. Each Entry-key set is asserted to be (a) a subset of the
leaf-type's spec alphabet (X|Y|Z for XYZDataSetType, A|B|C for
ABCDataSetType), (b) equal to the fixture-declared expected set, (c)
unique within the parent (no duplicate '<Entry key="X">' siblings),
and (d) non-empty in value text.

19 combinations × 3 versions = 57 cases plus 2 negative cases pinning
the <xs:choice> constraint (a container must not carry both simple
form and DataSet form). 59/59 pass in the default test sweep — no
[Category] gating; runs in CI without an XSD-1.1 validator dependency.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- JsonRoundTripHelper: switch British "honour" to American "honor" so
  the prose stays in American English.
- SchemaLoadTests: drop the "today" date-stamp from the comment that
  documents the strict-load gap.
- V2_7ComponentTests + V2_7ConfigurationDataSetTests: add SysML XMI
  URL, MTConnectDevices XSD URL, and Standard prose-section citations
  on the class-level doc-comment, mirroring the pattern already in
  place on V2_7DataItemTypeTests / V2_7SampleObservationTests.
…lution

Adds verbatim copies of the W3C xml.xsd and xlink.xsd schemas under
Schemas/w3c/ so the L1 schema-load harness can resolve MTConnect XSD
<xs:import namespace='.../xlink' schemaLocation='xlink.xsd'/> directives
without enabling network resolution (XmlResolver = null is kept for
defense-in-depth, OWASP XXE).

The existing EmbeddedResource glob (Schemas/**/*.xsd) picks the new
files up automatically; no csproj edit needed. README.md cites the
canonical W3C source URLs and the W3C document license that permits
unmodified redistribution.

Sources:
  https://www.w3.org/1999/xlink.xsd
  https://www.w3.org/2001/xml.xsd
Pre-loads the W3C xml + xlink XSDs into the XmlSchemaSet before adding
each MTConnect XSD, so the <xs:import namespace='.../xlink'
schemaLocation='xlink.xsd'/> directives resolve via target-namespace
match (the schemaLocation file the upstream tree never shipped is no
longer fetched).

XSDs that still fail to compile under .NET's XSD-1.0-only XmlSchemaSet
(notNamespace, maxOccurs>1 on xs:all, xs:any in unsupported context)
are tagged per-case with [Category("XsdLoadStrict")]. The default CI
sweep (Category!=XsdLoadStrict) now runs the 90 XSDs that load cleanly
with the xlink seed; the 32 XSD-1.1-only blockers stay opt-in.

The W3C xlink.xsd and xml.xsd resources themselves are excluded from
the test sweep — they are helper inputs, not MTConnect XSDs.

Result: default sweep 0 failures (was 55); strict subset surfaces only
genuine XSD-1.1 blockers, deferred to the Saxon-HE follow-up.
@ottobolyos ottobolyos marked this pull request as ready for review May 1, 2026 19:42
@ottobolyos
Copy link
Copy Markdown
Author

@PatrickRitchie — landed a few additional fixes since the original ready-for-review flip a couple of days ago that improve the foundation this PR ships:

  • SysML importer Linux-portability — the importer was silently no-op'ing on case-sensitive filesystems (lowercase csharp/templates/ paths against the on-disk CSharp/Templates/, literal \\ separators, hardcoded C:\temp\ output). Now portable across Linux / macOS / Windows via Path.DirectorySeparatorChar + case-correct paths + a CLI-driven --xmi/--output flag. Without this, every regen on a CI runner produced no output.
  • Multi-generalization importerUmlClass.Generalization (singular) became UmlClass.Generalizations[] (collection) so the importer can capture the SysML's multi-inheritance shape. This is what unblocked the v2.7 polymorphism work (OriginDataSet : AbstractOrigin, IDataSet, etc.) — without it the second generalization was silently lost and the wire-format polymorphism couldn't compile.
  • ResolveDanglingParents — when a class's generalization points into a sibling SysML package the local parser does not walk, the resolver grafts a structurally-minimal C# base into the local namespace so the generated children compile. Version-agnostic — fires only when there's a dangling parent, so older XMIs are no-ops.
  • A handful of generator hardcodes flagged + fixed — Result-Class TABLE-vs-DATA_SET classifier, Interface DataItem REQUEST/RESPONSE subtypes enum, ControllersComponent MinimumVersion = v2.0 per the SysML's introduced='2.0'. Each was a small one-liner in the importer that produced surprising .g.cs defaults until corrected.

PR #139 is ready for review. The merge-order checklist comment above stays current — if you start with this PR and follow the rest in any order, the rebase-against-master cost between siblings is near zero in practice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for MTConnect Standard v2.6 and v2.7

1 participant