Consider `trov:accessMode` on `ArrangementBinding`

[tmcphillips]

With trov:ArrangementBinding adopted as the uniform way for TRPs to reference arrangements (#30), we can reconsider how the access role (read vs. write) is expressed. Currently, the role is determined by which property the binding appears under (trov:accessedArrangement vs trov:contributedToArrangement).

An alternative would be to use trov:accessedArrangement for all bindings, with the access mode (via a trov:accessMode property with a value of trov:Read, trov:Write, or both) on the binding itself. This would simplify describing TRPs that might read from and write to the same arrangement; a single binding could list both access modes.

Current approach

Currently (per #30), read-only and write-only arrangements are under separate properties; a read+write arrangement must appear under both:

{
    "@id": "trp/0",
    "@type": "trov:TrustedResearchPerformance",
    "trov:accessedArrangement": [
        {
            "@id": "trp/0/binding/0",
            "@type": "trov:ArrangementBinding",
            "trov:arrangement": { "@id": "arrangement/0" },
            "trov:boundTo": "/data"
        },
        {
            "@id": "trp/0/binding/1",
            "@type": "trov:ArrangementBinding",
            "trov:arrangement": { "@id": "arrangement/1" },
            "trov:boundTo": "/workspace"
        }
    ],
    "trov:contributedToArrangement": [
        {
            "@id": "trp/0/binding/2",
            "@type": "trov:ArrangementBinding",
            "trov:arrangement": { "@id": "arrangement/2" },
            "trov:boundTo": "/output"
        },
        {
            "@id": "trp/0/binding/3",
            "@type": "trov:ArrangementBinding",
            "trov:arrangement": { "@id": "arrangement/1" },
            "trov:boundTo": "/workspace"
        }
    ]
}

Note that arrangement/1 above appears twice with the same trov:boundTo value duplicated. The biggest problem here is that these declarations can be inconsistent, with the same arrangement apparently bound at two different paths in the same TRP.

Proposed alternative

{
    "@id": "trp/0",
    "@type": "trov:TrustedResearchPerformance",
    "trov:accessedArrangement": [
        {
            "@id": "trp/0/binding/0",
            "@type": "trov:ArrangementBinding",
            "trov:arrangement": { "@id": "arrangement/0" },
            "trov:accessMode": { "@id": "trov:Read" },
            "trov:boundTo": "/data"
        },
        {
            "@id": "trp/0/binding/1",
            "@type": "trov:ArrangementBinding",
            "trov:arrangement": { "@id": "arrangement/2" },
            "trov:accessMode": { "@id": "trov:Write" },
            "trov:boundTo": "/output"
        },
        {
            "@id": "trp/0/binding/2",
            "@type": "trov:ArrangementBinding",
            "trov:arrangement": { "@id": "arrangement/1" },
            "trov:accessMode": [
                { "@id": "trov:Read" },
                { "@id": "trov:Write" }
            ],
            "trov:boundTo": "/workspace"
        }
    ]
}

In this scheme, arrangement/1 (read+write) appears once with both access modes declared in one place. No duplication, and it's clearer that the TRP read from /data, wrote to /output, and had dual access to /workspace.

Why dual access modes are useful

Distinguishing access modes (one way or the other) is a cheap way of communicating provenance relationships. A new file produced by a TRP can only have been derived (assuming the appropriate TRS capabilities are available and enabled) from artifacts that the TRP had read access to (because write access does not imply read access).

Other questions to consider

1. Are trov:Read and trov:Write the right names for the modes and cover the scenarios (for now)?

2. Should trov:accessMode be required or optional? If omitted, the binding could imply read/write. But this convention might limit our ability to add more modes in the future.

Related

• How should TRPs reference arrangements when trov:boundTo is empty? #30

[Xarthisius]

Correct me if I'm wrong but arrangements behave like COW (copy on write) objects. You cannot simultaneously have read and write, cause when you modify something you need to register it as a new arrangement anyway. My understanding is/was that "trov:accessedArrangement" and "trov:contributedToArrangement" is more like "before" and "after", rather than "readonly", "writeonly". Moreover:

"trov:contributedToArrangement": [...
        {
            "@id": "trp/0/binding/3",
            "@type": "trov:ArrangementBinding",
            "trov:arrangement": { "@id": "arrangement/1" },
            "trov:boundTo": "/workspace"
        }

from your first example is just a typo, right? arrangement/1 cannot be simultaneously accessed and contributedTo? If I had one file there with checksum A before, and I modified so now it has checksum B. How's that still the same arrangement?

[tmcphillips]

These are important questions, and while my response below describes the scenario I had in mind when bringing up the issue, I agree it's worth considering whether we want to support it at all in TROV. (I also fully admit I may not be seeing a conceptual or technical contradiction that makes the scenario nonsense--this is the thrill and agony of vocabulary design!)

First, you're right that a TRS implementation can treat all arrangements as immutable states, and that the "before/after" reading is the most straightforward perspective both to describe and to implement. For TRPs treated as atomic operations with read-only "before" arrangements and write-only "after" arrangements, this is plenty.

What I had in mind was providing the ability to shed a little more light on multi-step workflows implemented in a single TRP. If earlier steps within a single TRP write files that later steps in that same TRP read, exposing those intermediate files could (cheaply) reveal (or at least constrain speculation about) the finer-grained provenance of files in the "after" arrangement.

For a TRP with...

• /data — read-only input
• /intermediate — read+write working area (analogous to arrangement/1 and the poorly chosen /workspace above)
• /output — write-only output

...one can infer that any file under /output was at most derived from files under /data or under /intermediate. Meanwhile one can also state that any files under /intermediate at most depend on files under /data and any other files under /intermediate. If the TRS further enforces (this could be a TRS capability) that the /intermediate area is empty when the TRP begins, the situation becomes even clearer.

With the potential vocabulary change, a straightforward query can list the possible dependencies of any output (or intermediate) artifact without distinguishing between "before" and "intermediate" arrangements in the query. Given that, say, <artifact/42> is known to be in a writable arrangement of <trp/0>, its potential dependencies are simply all artifacts in arrangements that <trp/0> could read:

PREFIX trov: <https://w3id.org/trace/trov/0.1#>

SELECT ?dependency
WHERE {
    <trp/0> trov:accessedArrangement ?readableBinding .         # get the bindings of trp/0
    ?readableBinding trov:accessMode trov:Read .                # keep only readable bindings
    ?readableBinding trov:arrangement ?readableArrangement .    # get the bound arrangement
    ?readableArrangement trov:hasArtifactLocation ?dependencyLocation .  # get artifact locations in it
    ?dependencyLocation trov:artifact ?dependency .             # return each as a potential dependency
}

With only input and output arrangements, this query returns only input artifacts as potential dependencies. It can't reveal that some outputs were actually derived from other outputs.

But if we add a read+write /intermediate arrangement, the same query, unchanged, also returns intermediate artifacts as potential dependencies. An artifact listed both in the intermediate and output arrangement is now a potential dependency of each of the outputs.

The alternative of splitting multi-step workflows into separate TRPs to capture this provenance is cleaner, but does impose work on the researcher. Dual access mode might let a TRS expose intermediate provenance without requiring the researcher to restructure their workflow.

One question is whether we should require any writable (including intermediate) arrangements to only list files present when the TRP ends. In principle, a TRS that monitors file access, say via logging of user-space filesystem operations, could also list transient files that were created, potentially read by later steps, but then deleted before completion. However, the challenge of hashing transient files may argue against the usefulness of recording them at all.

At any rate, the vocabulary doesn't have to prescribe how thorough the TRS has to be, just provide a way to express what the TRS captures.

Tradeoffs

• If we went this route we'd be removing the trov:contributedToArrangement term but adding the trov:accessMode term, and requiring the latter on each binding.
• For the purely before/after scenario, the cost isn't terrible. An annotation per binding instead of a choice of property, and intermediate arrangements stand out.
• For read+write intermediate arrangements, one binding replaces two duplicated ones.

[Xarthisius]

I'm trying to understand something very basic here: for me an arrangement, despite its JSON complexity, is a very simple, immutable set of (path, checksum) pairs with a unique id.

Arr_1 = {path1: checksum1}

It's not possible to change either the key or the value, or add/remove an element and still call it Arr_1. Do I get that right?

[tmcphillips]

Yes, an arrangement is an immutable set of tuples (ID, path, checksum), with only one element allowed for a particular ID, only one ID associated with a particular path, and only one checksum associated with that (id, path) pair. Nothing in the proposal changes that.

The only difference between an arrangement with a read-write binding on a TRP vs a write-only binding is that the conclusions about provenance of new artifacts in that arrangement are less constrained when the binding is read-write. If an arrangement is writable by a TRP, then the TRP may have created new artifacts in it; if the arrangement is readable as well, then the TRP may have derived these new artifacts from other artifacts (including newly created ones) in that same arrangement. But for any writable arrangement (including readable ones), the final state of the artifacts is what is described.

Thinking about this now reveals a contradiction in my earlier thinking with respect to transient artifacts. This contradiction was likely a source of confusion (especially for me).

Specifically, because we want an arrangement that describes artifacts created by one TRP to be usable as a read-only input arrangement by a later TRP, all the artifacts in that arrangement must still exist when the downstream TRP accesses it via a read-only binding. If a TRS detects transient intermediate files, we'll need another way to express that.