Source space data from nanopub-query spaces repo#468
Draft
tkuhn wants to merge 21 commits into
Draft
Conversation
… spaces repo
Replaces the four-query fixed-point loop in Space.triggerSpaceDataUpdate
(GET_ADMINS + GET_SPACE_MEMBER_ROLES + GET_SPACE_MEMBERS) and the URL-regex
sub-space inference in SpaceRepository with SPARQL queries against the new
/repo/spaces endpoint exposed by nanopub-query >=1.11. Admin closure,
gen:hasRole validation, and role-instantiation tier checks are now applied
server-side in the materialised space-state graph, so nanodash issues one
query per call site instead of an iterative client-side closure.
Pinned templates/queries and setCoreData's root-nanopub seed stay on the
legacy GET_PINNED_TEMPLATES / GET_PINNED_QUERIES paths; the adminPubkeyMap
they rely on continues to be built from User.getUserData() inside addAdmin.
The new SpacesRepoAccess helper wraps an RDF4J SPARQLRepository pointed at
\${NANODASH_MAIN_QUERY}/repo/spaces. Until other query instances ship the
spaces repo, deployments must pin NANODASH_MAIN_QUERY +
NANOPUB_QUERY_INSTANCES to a spaces-aware instance (see NOTES.md). Empty
results are treated as authoritative; no fallback.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces GET_MAINTAINED_RESOURCES with a single SPARQL query against /repo/spaces. The materialiser already gates each declaration on admin-of-maintaining-space authority, so the resulting (resource, space) links can be consumed directly from `?resource npa:isMaintainedBy ?space` in the current space-state graph. Per-resource label and namespace are joined in from the declaring nanopub's admin and assertion graphs. The old `refresh(ApiResponse)` sink is gone (no external callers). The no-arg `refresh()` now reads from the spaces repo and the previously duplicated invalidating variant becomes `refreshAndInvalidate()`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…anel Removes GET_ADMINS, GET_SPACE_MEMBERS, GET_SPACE_MEMBER_ROLES, and GET_MAINTAINED_RESOURCES from QueryApiAccess now that no Java call sites remain. The MaintainedResource list on SpacePage previously used GET_MAINTAINED_RESOURCES purely as an ItemListPanel async trigger (its ApiResponse was discarded); it now drives off MaintainedResourceRepository.ensureLoaded directly via the ReadyFunction/ResultFunction constructor, removing the spurious roundtrip. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
These two grlc queries are deprecated. Pinned templates and queries are parsed inline from the Space root nanopub's assertion (kpxl:hasPinnedTemplate / kpxl:hasPinnedQuery / nt:hasTag), already handled in setCoreData, so no separate fetch is needed. Removes the two admin-pubkey-gated for-loops in Space.triggerSpaceDataUpdate, the spaceIds Multimap that fed only them, the corresponding constants in QueryApiAccess, and the now-unused Multimap/ArrayListMultimap/ApiResponse/QueryRef/Template imports. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pinning is being retired. Removes the pinnedResources / pinGroupTags / pinnedResourceMap fields on SpaceData, their getters on Space, and the inline parsing of kpxl:hasPinnedTemplate / kpxl:hasPinnedQuery / nt:hasTag in setCoreData. No external callers existed in the main app (the legacy Project class still consumes the same predicates via its own code path — left untouched). Empirical check: the spaces repo on query.nanodash.net contains zero kpxl:hasPinnedTemplate or kpxl:hasPinnedQuery triples, and the full repo has none in any gen:Space-typed nanopub's assertion. So the inline parsing was dead code in practice too. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SpacesRepoAccess previously built its endpoint from Utils.getMainQueryUrl, which falls back to query.knowledgepixels.com when NANODASH_MAIN_QUERY is unset — and that instance does not (yet) carry the spaces repo, so every SPARQL call 404'd and returned an empty list. Symptom: HomePage rendering "Configured home resource <...> could not be found" because the maintained- resource lookup found nothing. Hardcode query.nanodash.net (the only instance with the spaces repo today) as the default. Override via the new NANODASH_SPACES_REPO_URL env var. Once the repo is widely deployed, the default should fall back to Utils.getMainQueryUrl. Documented in NOTES.md. The legacy NANODASH_MAIN_QUERY / NANOPUB_QUERY_INSTANCES pinning is no longer required for the spaces features to work locally. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
WicketApplication's constructor unconditionally calls Desktop.getDesktop().browse(...) when AWT desktop is available. Every WicketTester-backed test instantiates the application, so a browser window pops to localhost:37373 for each test run. Setting java.awt.headless=true in the surefire argLine flips Desktop.isDesktopSupported() to false, which the existing guard already handles — no source change needed. Real runs (mvn jetty:run) still get the browser-open behaviour. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an INFO-level summary of MaintainedResourceRepository.refresh: row count returned by the SPARQL query, how many rows were kept, how many were skipped because their maintaining space wasn't in SpaceRepository, and the list of missing space IRIs. This will be removed once the live-deployment issue is resolved. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GRAPH ?a { OPTIONAL { ?resource gen:hasNamespace ?namespace } } drops
the entire row when the assertion graph ?a has no triple matching the
inner pattern, even though it's wrapped in OPTIONAL. Many maintained
resources (including the Nanodash home resource) don't declare
gen:hasNamespace, so they were silently filtered out — the spaces repo
returned 21 rows instead of the actual 42.
Pulling the OPTIONAL outside the GRAPH wrapper restores the row even
when the namespace pattern doesn't match; the GRAPH block now only runs
when ?a has the matching triple, while everything else flows through.
Also drops the diagnostic logger.info added in 506841c that pinpointed
the issue.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Defensive twin of 0ab2006. The role-load SPARQL had the same anti-pattern (GRAPH ?role_a { OPTIONAL { ... } } for label/title/name/template) — would drop any role with none of those properties. Unlikely in practice (every known role declares at least one), but worth fixing pre-emptively so we don't get a silent missing-role symptom later. Each property is now its own OPTIONAL { GRAPH ?role_a { ... } } block, matching the safe shape used for maintained-resource namespace lookup. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pre-existing latent NPE risk: findResourcesBySpace returned resourcesBySpace.computeIfAbsent(...) without first calling ensureLoaded(). On a cold start the map is null, so any call site that reached this method before findById/findByNamespace was hit would NPE. In practice SpacePage triggers ensureLoaded via its own panel, so the order worked out — but defensively this should follow the same pattern as the sibling find methods. Also fixed a misleading javadoc parameter note while there. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The spaces-repo state graph (npass:<…>) only contains RIs that the materialiser's per-tier admit query was able to validate. That query times out on some spaces (a RDF4J planner blow-up — observed on fdo-connect, where the observer-tier admit hits a 60s gateway timeout and silently produces zero state-graph rows even though the per-tier candidate set is non-empty). The result: nanodash showed no members for any space whose observer-tier validation didn't complete, even though the data is correctly extracted into npa:spacesGraph. Switch loadMembersFromSpacesRepo to read the universe of candidate RoleInstantiations from the extraction graph and gate them client-side via data.adminPubkeyMap — the same gate the legacy GET_SPACE_MEMBERS query applied pre-migration. The admin closure itself is still sourced from the state graph (admin tier validates reliably; the observer-tier evaluator blow-up is what's broken). Invalidated RIs are filtered via the standard `?inv npx:invalidates ?np` join in npa:graph. Admin-tier RIs are excluded from the member set (they're added separately via loadAdminsFromSpacesRepo). When the materialiser is fixed upstream, the right move is to switch back to querying the state graph here. Tracking the workaround in the javadoc on loadMembersFromSpacesRepo. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five direct-SPARQL call sites in nanodash, all introduced by this PR's spaces-repo migration: - Space.loadAdminsFromSpacesRepo - Space.loadRolesFromSpacesRepo - Space.loadMembersFromSpacesRepo - MaintainedResourceRepository.MAINTAINED_RESOURCES_QUERY - SpaceRepository.SUBSPACE_LINKS_QUERY All Nanopub Query access should eventually flow through published grlc query templates (like the constants in QueryApiAccess) rather than SPARQL strings built in Java. Once those five are republished as grlc nanopubs against /repo/spaces, the SpacesRepoAccess helper itself becomes redundant — TODO on the class javadoc tracks that too. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the UserData-based population of Space.adminPubkeyMap with a
SPARQL query against the spaces repo's mirrored AccountState rows.
Trust-state-validated (agent, pubkey-hash) pairs for the space's admin
RIs now drive the client-side admin gate used by:
- the view-display admin filter (AbstractResourceWithProfile.java:135,
DownloadRdfPage.java:588, ViewDisplayMenu.java)
- the member gate in loadMembersFromSpacesRepo
Stricter than the prior UserData.getPubkeyHashes source: only pubkeys
present in the current trust state count. Matches the trust model the
rest of this PR moves toward.
This is the same programmatic-SPARQL pattern as the other space-data
loads, with the same TODO to migrate to a published grlc query template
later. The ideal end-state is a server-gated get-view-displays-v2 that
joins through the spaces-repo admins, after which adminPubkeyMap can
go away entirely.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the UserData-based admin_pubkey / user_pubkey multi-placeholder
expansion in ViewList and DownloadRdfPage with spaces-repo-sourced sets:
- admin_pubkey → Space.getAdminPubkeyHashes() (adminPubkeyMap keyset,
populated by loadAdminPubkeyHashesFromSpacesRepo from npa:AccountState
rows for admin RIs)
- user_pubkey → Space.getUserPubkeyHashes() (new userPubkeyMap keyset,
populated by a new loadUserPubkeyHashesFromSpacesRepo SPARQL that
looks up AccountState pubkey hashes for every agent already in
data.users — admins + admin-gated members)
Stricter than the prior UserData.getPubkeyHashes(agent, true) source:
only pubkey hashes present in the current trust state are included.
Users with pubkeys not yet trust-validated contribute no hashes to the
expansion; view queries that filter by user_pubkey may return fewer rows
for those users. Matches the trust model the rest of this PR moves
toward.
Both call sites carry TODOs that the long-term direction is to push the
pubkey filtering server-side too (a published grlc query that gates
by user-of-space), at which point nanodash wouldn't have to expand the
placeholder client-side at all.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the legacy GET_SPACES grlc call in SpaceRepository.refresh
with a SPARQL query against npa:spacesGraph that enumerates SpaceRefs,
joins to the declaring nanopub for label and type, filters invalidated
nanopubs, orders by DESC(?date), and dedups by Space IRI in Java
(latest update per space wins).
Public API:
- refresh() is now the SPARQL path (no arg)
- refreshAndInvalidate() retains the prior behaviour of refresh+
setDataNeedsUpdate per space
- refresh(ApiResponse resp) is gone (no external callers)
SpaceListPage's per-type panel switches from the QueryRef-driven
ItemListPanel constructor (which used GET_SPACES purely as an async
trigger and re-fetched the same data via refresh(apiResponse)) to the
ReadyFunction/ResultFunction shape that drives off SpaceRepository
.ensureLoaded directly — saves a roundtrip and matches the pattern
used elsewhere for the MaintainedResource panel.
GET_SPACES constant dropped from QueryApiAccess (no remaining callers).
Same TODO as the other six programmatic-SPARQL spots: ultimately
republish as a grlc nanopub against /repo/spaces.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The member gate in loadMembersFromSpacesRepo was too strict: it required the member RoleInstantiation's signing pubkey hash to be in the admin set of the space. That dropped self-published memberships entirely — e.g., FIP.38.T.8 has 10 self-signed participatedAsParticipantIn nanopubs that the live page shows but the migrated path was filtering out. Pre-migration GET_SPACE_MEMBERS semantic: the admin gate happens server- side on the gen:hasRole attachment (which role predicates are recognised for a space), not on each individual member nanopub. Anyone may then self-publish a triple with one of those predicates; the membership is recognised because the role itself was admin-attached. Match that here by gating on `data.roleMap.containsKey(predicate)` instead. Side effect: extracted RIs whose predicate doesn't correspond to any attached role (in FIP.38.T.8's case: P1344, participatedAsImplementerAspirantIn, participatedAsTrainerAssistantIn — none have a matching gen:RoleAssignment) are dropped, matching legacy behaviour. Verified live: 10 admitted for FIP.38.T.8, matching the live page exactly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2 tasks
github-actions Bot
pushed a commit
to knowledgepixels/nanopub-query
that referenced
this pull request
May 24, 2026
## [1.14.2](nanopub-query-1.14.1...nanopub-query-1.14.2) (2026-05-24) ### Bug Fixes * **SpacesExtractor:** emit inline non-hasAdmin role RIs from gen:Space ([0dd70bc](0dd70bc)), closes [knowledgepixels/nanodash#468](knowledgepixels/nanodash#468) ### Documentation * add sparql-quirks.md with the GRAPH/OPTIONAL gotcha ([1eff821](1eff821)), closes [knowledgepixels/nanodash#468](knowledgepixels/nanodash#468) * **spaces:** correct npa:RoleAssignment → gen:RoleAssignment in prose ([bdc5a77](bdc5a77)) * **sparql-quirks:** note the pointer-pattern planner blow-up ([c63fbc9](c63fbc9)) ### General maintenance * setting next snapshot version [skip ci] ([7b589da](7b589da)) ### Refactoring * **GrlcSpec:** delegate parsing/expansion to QueryTemplate from nanopub-java 1.89.0 ([c7b82a8](c7b82a8))
…y parsing Bump org.nanopub:nanopub 1.88.0 -> 1.89.0 and use the new QueryTemplate class. GrlcQuery now extends QueryTemplate, removing the duplicated SPARQL/ endpoint/label/placeholder parsing that previously lived in nanodash. GrlcQuery keeps only the Nanodash-specific concerns: the instance cache + get() factories, and the QueryParamField form integration (createParamFields, expandQuery(List), allMandatoryFieldsSet). It pre-fetches the nanopub via Utils.getNanopub and uses the QueryTemplate(Nanopub, idOrUri) constructor so fetching stays on Nanodash's registry/cache path. expandQuery(List) keeps its own loop because QueryParamField recognizes the _multi_val convention that the library does not. Local GRLC_* IRI constants replaced by org.nanopub.vocabulary.KPXL_GRLC in ExplorePage and GrlcQueryTest. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Now that all public query instances carry the spaces repo, drop the temporary hardcoding of query.nanodash.net introduced in fe99e16. SpacesRepoAccess again builds its endpoint from Utils.getMainQueryUrl(); removed the DEFAULT_SPACES_REPO_URL constant, the NANODASH_SPACES_REPO_URL override, and the corresponding NOTES.md section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…blished grlc queries Replace the programmatically-built spaces-repo SPARQL in SpaceRepository and MaintainedResourceRepository with the published grlc query templates get-spaces, get-sub-space-links and get-maintained-resources (added as QueryApiAccess constants), fetched via ApiCache/QueryRef like the rest of the query-template access. Preserves the DESC(?date) dedup and the blank-as-absent handling for optional label/namespace columns. Removes 3 of the programmatically-built SPARQL TODO markers; 5 remain in Space.java (the per-space, parameterized queries). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…grlc queries Replace the five programmatically-built spaces-repo SPARQL queries in Space.java (loadAdmins / loadAdminPubkeyHashes / loadRoles / loadMembers / loadUserPubkeyHashes) with the published grlc query templates get-space-admins, get-space-admin-pubkey-hashes, get-space-roles, get-space-members and get-agent-pubkey-hashes (added as QueryApiAccess constants), fetched via ApiCache/QueryRef. Per-space/agent IRI lists are passed as repeated multi-IRI params (space/agent), mirroring ViewList's existing multi-param usage; a runSpacesQuery() helper null-guards the response like the old select() did. Member role-predicate gating, the empty-users short-circuit, and blank-as-absent handling for optional columns are preserved. Completes the grlc migration (0 programmatically-built SPARQL queries remain) and removes SpacesRepoAccess entirely. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the client-side computation of space admins/members/roles, sub-space relations, maintained resources, admin/user pubkey expansion, and the top-level spaces listing with SPARQL queries against
/repo/spaces, exposed by nanopub-query >= 1.11 (seenanopub-query/doc/design-{trust-state-repos,space-repositories}.md). Also removes a couple of deprecated/retired code paths along the way.Migrated in this PR
GET_ADMINS+GET_SPACE_MEMBER_ROLES+GET_SPACE_MEMBERSfixed-point loop inSpace.triggerSpaceDataUpdateSpaceRepository.populateSubspaceRelations?child npa:isSubSpaceOf ?parentfrom the state graph (explicit declarations + URL-prefix fallback)GET_MAINTAINED_RESOURCESinMaintainedResourceRepository?resource npa:isMaintainedBy ?spacefrom the state graph, joined to declaring nanopub for label/namespaceGET_SPACESinSpaceRepository(andSpaceListPage's ItemListPanel async trigger)npa:SpaceRef+npa:SpaceDefinitionfromnpa:spacesGraph, joined to declaring nanopub for label/type, dedup'd by Space IRIUserData.getPubkeyHashes(admin, null)inSpace.addAdminnpa:AccountStatefor the space's admin RIsUserData.getPubkeyHashes(user, true)foradmin_pubkey/user_pubkeymulti-placeholder expansion inViewList/DownloadRdfPagenpa:AccountStatefor each user agent in the spaceGET_PINNED_TEMPLATES+GET_PINNED_QUERIES(deprecated)pinnedResources,pinGroupTags,pinnedResourceMap, their getters, the inlinekpxl:hasPinnedTemplate/kpxl:hasPinnedQuery/nt:hasTagparsing)Projectclass still uses the same predicates via its own code path; not touched.GET_ADMINS,GET_SPACE_MEMBERS,GET_SPACE_MEMBER_ROLES,GET_MAINTAINED_RESOURCES,GET_PINNED_TEMPLATES,GET_PINNED_QUERIES,GET_SPACESQueryApiAccessNew helper:
SpacesRepoAccess(singleton) wraps an RDF4JSPARQLRepository, defaulting tohttps://query.nanodash.net/repo/spaces(the only instance currently exposing the spaces repo). Override via the newNANODASH_SPACES_REPO_URLenv var. The legacyNANODASH_MAIN_QUERY/NANOPUB_QUERY_INSTANCESpinning is no longer required.Stricter trust semantics: anything that needs an admin or user pubkey hash now sources it from the spaces repo's mirrored
npa:AccountStaterows (i.e., trust-state-validated only). The priorUserData.getPubkeyHashessource was broader (all pubkey hashes the user registry knows about for the agent). Practically: view displays and view-query filters that filter by pubkey will only match pubkeys present in the current trust state. For active admins/users this is transparent; for agents using pubkeys not yet trust-validated, their nanopubs may no longer show through these filters.Eight programmatic-SPARQL spots (all TODO-marked)
All Nanopub Query access should eventually flow through published grlc query templates (like the constants in
QueryApiAccess) rather than SPARQL built in Java. Each spot carries a TODO; together they motivate deletingSpacesRepoAccessentirely when the migration completes:Space.javaloadAdminsFromSpacesRepoSpace.javaloadAdminPubkeyHashesFromSpacesRepoSpace.javaloadRolesFromSpacesRepoSpace.javaloadMembersFromSpacesRepoSpace.javaloadUserPubkeyHashesFromSpacesRepoMaintainedResourceRepository.javaMAINTAINED_RESOURCES_QUERYSpaceRepository.javaSUBSPACE_LINKS_QUERYSpaceRepository.javaSPACES_QUERYIncidental fixes (also in this PR)
GRAPH ?x { OPTIONAL { ... } }drops the entire row in RDF4J when the inner pattern has no match. Initially hit onMaintainedResourceRepository(the Nanodash home resource was silently filtered because it doesn't declaregen:hasNamespace). Fixed by pulling the OPTIONAL outside (OPTIONAL { GRAPH ?x { ... } }); same fix applied defensively to the role-loading query inSpace.java.SpacesRepoAccessdefaults directly tohttps://query.nanodash.net/repo/spacesrather than going throughUtils.getMainQueryUrl()(which falls back to instances without the spaces repo). Override viaNANODASH_SPACES_REPO_URL.fdo-connect), leaving member RIs extracted but never validated.loadMembersFromSpacesReporeads the universe fromnpa:spacesGraphand gates client-side viadata.adminPubkeyMap, matching the legacyGET_SPACE_MEMBERSbehaviour. When the materialiser is fixed upstream, this query should switch back to the state graph.MaintainedResourceRepository.findResourcesBySpacenow callsensureLoaded()like its sibling find methods.argLinesets-Djava.awt.headless=true, soDesktop.isDesktopSupported()returnsfalsein tests and the existing guard inWicketApplication's browser-open code skips. Real runs unaffected.Follow-up work (out of scope)
get-view-displays-v2that joins through spaces-repo admins)adminPubkeyMap/userPubkeyMapentirelydesign-space-repositories.mdnpa:RoleAssignmentdoc fix (extractor emitsgen:RoleAssignment)GRAPH ?x { OPTIONAL { ... } }hazard for future queriesDeployment requirement
SpacesRepoAccesshitshttps://query.nanodash.net/repo/spacesby default. Override with theNANODASH_SPACES_REPO_URLenv var if you've pointed nanodash at a different spaces-aware instance. Empty SPARQL results are treated as authoritative; there is no fallback to the legacy code path.Test plan
mvn test— 691 pass, 0 failhttps://query.nanodash.net/repo/spaces🤖 Generated with Claude Code