docs: capture fork-gather research and verification plan
This commit is contained in:
+15
@@ -5,6 +5,21 @@ This context defines the core workflow runtime language used by `wf_core`,
|
||||
|
||||
## Language
|
||||
|
||||
**Fork Control Claim**:
|
||||
The outstanding branch obligations carried by a fork/gather continuation in the
|
||||
proposed concurrency model; complete sibling obligations restore their parent
|
||||
claim.
|
||||
_Avoid_: Historical trace, state lineage
|
||||
|
||||
**Referenced Fork Visit**:
|
||||
An unresolved fork occurrence on which at least one component of a control
|
||||
claim depends.
|
||||
_Avoid_: Assuming it encloses the entire claim
|
||||
|
||||
**Enclosing Fork Visit**:
|
||||
An unresolved fork occurrence encompassing every component of a control claim.
|
||||
_Avoid_: Any fork mentioned in historical provenance
|
||||
|
||||
**lda.chat**:
|
||||
An AI agent platform for authoring and executing workspace workflows.
|
||||
_Avoid_: Chatbot, MCP server
|
||||
|
||||
@@ -114,6 +114,10 @@ docs as the active references:
|
||||
concurrent foreach policy and barrier commit semantics.
|
||||
- [`adr/0006-explicit-fork-and-topology-driven-gather.md`](adr/0006-explicit-fork-and-topology-driven-gather.md):
|
||||
proposed explicit fork, gather-slot, and activation-token semantics.
|
||||
- [`superpowers/specs/2026-09-06-fork-gather-design.md`](superpowers/specs/2026-09-06-fork-gather-design.md):
|
||||
draft fork/gather contract, unresolved decisions, and verification gates.
|
||||
- [`historical/research/2026-09-06-fork-gather/README.md`](historical/research/2026-09-06-fork-gather/README.md):
|
||||
copied research reports and runnable reference artifacts, with known defects.
|
||||
- [`superpowers/specs/2026-09-04-foreach-back-edge-design.md`](superpowers/specs/2026-09-04-foreach-back-edge-design.md):
|
||||
approved canonical foreach body-return and validation semantics.
|
||||
- [`superpowers/specs/2026-09-04-structured-runtime-context-design.md`](superpowers/specs/2026-09-04-structured-runtime-context-design.md):
|
||||
|
||||
@@ -10,6 +10,12 @@ gathers rendezvous compatible activation tokens, merge their lineage-local
|
||||
state patches, and create one continuation. This preserves topology-only edges
|
||||
without making multiple matching edges silently mean broadcast.
|
||||
|
||||
The [draft design contract](../superpowers/specs/2026-09-06-fork-gather-design.md)
|
||||
records the current requirements, open decisions, and verification gates.
|
||||
The [research archive](../historical/research/2026-09-06-fork-gather/README.md)
|
||||
preserves supporting evidence and known reference-model defects. This ADR is
|
||||
architectural direction, not an executable implementation plan.
|
||||
|
||||
## Context
|
||||
|
||||
The scheduler, ready queue, blocked frames, lineage-local state views, and
|
||||
@@ -83,8 +89,25 @@ most one deepest shared lineage: the lowest common ancestor is the deterministic
|
||||
merge base. Different scopes or no shared ancestor fail before state mutation.
|
||||
A partial gather creates its intermediate lineage under that merge base and
|
||||
retains multi-input provenance in activation-token metadata. A final gather can
|
||||
merge that lineage with remaining siblings and resume the blocked parent
|
||||
continuation.
|
||||
merge that lineage with remaining siblings and create a continuation token.
|
||||
Completion ownership survives token/frame replacement; foreach and subgraph
|
||||
parents still wait for the corresponding item or call to finish.
|
||||
|
||||
The lineage tree is not sufficient to deduplicate shared pending history after
|
||||
re-forking, cross-gathering, and reconvergence. Supporting that topology requires
|
||||
stable original contribution identities or an equivalent demonstrated scheme;
|
||||
otherwise the topology must be explicitly rejected. New node executions create
|
||||
new contributions even when node and token identities are unchanged. Merging
|
||||
existing contributions preserves their identities.
|
||||
|
||||
Control obligations also need two distinct ancestry queries. Referenced fork
|
||||
visits cover any component of a claim and govern unresolved re-entry checks.
|
||||
Enclosing visits cover every component and govern gather correlation. For a
|
||||
mixed claim `{k.x, r.c}` with `k` nested under `r`, only `r` encloses the whole
|
||||
claim. The proposed V1 matches a compiled anchor to that enclosing dynamic
|
||||
visit, without requiring an authored originating-fork field. One gather may
|
||||
fire at most once per owner/gather/anchor occurrence. Validation must account
|
||||
for past firings as well as currently parked arrivals.
|
||||
|
||||
The first gather merge policy is fail-closed:
|
||||
|
||||
@@ -98,9 +121,12 @@ merges. The gather policy determines what happens when patches cannot be
|
||||
merged; the initial behavior is to fail rather than choose a last writer.
|
||||
|
||||
Gather slots have declaration order, and that order is the canonical reducer
|
||||
replay order. After choosing the merge base, the runtime applies each selected
|
||||
lineage's writes after that base in declared-slot order, never arrival,
|
||||
scheduler, or frame-id order. A bucket accepts exactly one token for each slot;
|
||||
replay order for independent histories, never arrival, scheduler, or frame-id
|
||||
order. Whether an earlier partial gather's serialization becomes a permanent
|
||||
ordering constraint at later reconvergence remains open. Opposite histories
|
||||
`[A,B]` and `[B,A]` can arise in legitimate authored graphs; treating their
|
||||
combination as a merge-order conflict is a policy choice, not a corruption
|
||||
check. A bucket accepts exactly one token for each slot;
|
||||
a second token for the same activation and slot fails the activation instead of
|
||||
making an alternative-path race decide the result. Order-sensitive reducers
|
||||
such as append are therefore deterministic in synchronous and asynchronous
|
||||
@@ -155,9 +181,10 @@ their local rendezvous contract, not one producer.
|
||||
such as `d OR e` would require mutually exclusive arrivals. Named slots provide
|
||||
AND across slots and OR within a slot.
|
||||
|
||||
**Lineage becomes a multi-parent DAG.** Not currently required. Merge provenance
|
||||
belongs to activation tokens; an intermediate merged worldview can remain a
|
||||
child of the compatible inputs' common lineage parent.
|
||||
**Lineage becomes a multi-parent DAG.** Not currently required. An intermediate
|
||||
merged worldview can remain a child of the inputs' common lineage parent, but
|
||||
control provenance alone is not proof of state-write uniqueness. Shared-write
|
||||
identity and ordering require the separate contract described above.
|
||||
|
||||
**Store scope and parent-lineage identity on every frame.** Rejected because
|
||||
those relationships are canonical on the lineage and duplicated frame fields
|
||||
@@ -178,10 +205,13 @@ the node was a pass-through marker with no barrier contract.
|
||||
gather; ordinary edge semantics stay unchanged.
|
||||
- Workflow validation must require every gather-target edge to name exactly one
|
||||
declared slot, reject missing or unknown gather slots, prove that every slot
|
||||
has an incoming edge, reject slots on non-gather targets, and preserve one
|
||||
successor per ordinary `(node, outcome)` pair.
|
||||
has an incoming edge, preserve but ignore ports on ordinary targets, and keep
|
||||
one successor per ordinary `(node, outcome)` pair.
|
||||
- Checkpoints must persist pending gather arrivals and activation provenance so
|
||||
interruption/resume cannot mix loop iterations or subgraph invocations.
|
||||
- Supported checkpoints must not expose partially applied transitions, including
|
||||
when an exception produces a failed stopped run. This does not require a
|
||||
durable transition journal or promise exactly-once external effects.
|
||||
- Runtime operations should resolve a frame, its lineage, and its scope through
|
||||
one internal interface instead of accepting several independently supplied
|
||||
identifiers.
|
||||
@@ -199,11 +229,12 @@ the node was a pass-through marker with no barrier contract.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- The exact serialized edge field and authoring name for a gather slot.
|
||||
- The minimal activation-correlation and provenance representation that
|
||||
supports loops, nested forks, subgraphs, partial gathers, and cross-merges.
|
||||
- Whether a gather resumes an existing blocked frame or creates a dedicated
|
||||
continuation frame in each topology shape.
|
||||
- The supported correlation grammar and production validation algorithm for
|
||||
loops, nested forks, partial gathers, and cross-merges. Destination edges use
|
||||
`target_port`; ordinary destinations ignore that metadata.
|
||||
- Whether prior gather ordering is permanent history or a local state view.
|
||||
- The concrete owner/occurrence representation and integration with blocked
|
||||
foreach and subgraph completion, including failure and checkpoint handling.
|
||||
- The trace representation for waiting and merging without excessive internal
|
||||
scheduler noise.
|
||||
|
||||
|
||||
@@ -56,6 +56,17 @@ independently or repeat ownership walks.
|
||||
Reuse the scheduler, activation, lineage, and reducer-aware barrier machinery:
|
||||
|
||||
- [`ADR-0006: explicit fork and topology-driven gather`](adr/0006-explicit-fork-and-topology-driven-gather.md)
|
||||
- [`Draft contract and verification gates`](superpowers/specs/2026-09-06-fork-gather-design.md)
|
||||
|
||||
Production planning remains gated on correlation analysis and merge-order
|
||||
semantics. The draft records two reproduced reference-model defects and points
|
||||
to archived runnable research; passing that prototype's tests is not approval
|
||||
to copy it into the runtime.
|
||||
|
||||
The isolated
|
||||
[`reference verification plan`](superpowers/plans/2026-09-07-fork-gather-reference-verification.md)
|
||||
closes occurrence regressions and gathers executable decision evidence first;
|
||||
it does not authorize production fork/gather changes.
|
||||
|
||||
Outcomes continue to choose one transition. Forks create concurrent branch
|
||||
activations. Gathers wait on declared incoming topology, merge compatible
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
# Fork/gather research evidence — 2026-09-06
|
||||
|
||||
These are historical research inputs, not approved specifications or production
|
||||
code. The current contract is the
|
||||
[draft fork/gather spec](../../../superpowers/specs/2026-09-06-fork-gather-design.md).
|
||||
The [ADR](../../../adr/0006-explicit-fork-and-topology-driven-gather.md) records
|
||||
the architectural direction. Both remain proposed.
|
||||
|
||||
## Provenance
|
||||
|
||||
Eight files were copied, not moved, from `random shit/research-chatgpt/`.
|
||||
Every copy was checked against its original with SHA-256. The redundant ZIP,
|
||||
older citation-less exports, and other unrelated research were not imported.
|
||||
|
||||
| Archived file under `imported/` | Original relative path |
|
||||
| --- | --- |
|
||||
| `round-2-report.md` | `deep-research-report060926 copy 3.md` |
|
||||
| `round-2-citations.md` | `deep-research-report060926 copy 2.md` |
|
||||
| `audit-1.md` | `research-audit060926.md` |
|
||||
| `audit-2.md` | `research-audit060926-002.md` |
|
||||
|
||||
The remaining four files retain their original basenames and were copied from
|
||||
the `research-audit060926-002/` subdirectory:
|
||||
|
||||
- `fork_gather_reference.py`
|
||||
- `test_fork_gather_reference.py`
|
||||
- `README_fork_gather_reference.md`
|
||||
- `verification_test_results.txt`
|
||||
|
||||
The report's inline citation numbers can be matched to the citations export.
|
||||
Uploaded-file citations concern an older repository snapshot, not necessarily
|
||||
current code. Original `sandbox:` download links are inert historical links;
|
||||
the actual supplied files are alongside the reports here.
|
||||
|
||||
Keep `imported/` byte-identical to these source snapshots. Its local Markdown
|
||||
lint configuration disables style rules solely to preserve the imported text.
|
||||
This README and the live documents use normal lint rules. Future experiments
|
||||
belong outside `imported/`; do not silently repair the evidence.
|
||||
|
||||
## Independent verification and limitations
|
||||
|
||||
On 2026-09-06, the 25 supplied tests passed locally. Reproduce from the repo root:
|
||||
|
||||
```powershell
|
||||
Push-Location 'docs/historical/research/2026-09-06-fork-gather/imported'
|
||||
try { python -B -m unittest -v test_fork_gather_reference.py }
|
||||
finally { Pop-Location }
|
||||
```
|
||||
|
||||
Audit 2 explicitly labels its implementation a reconstruction: the scratch
|
||||
code claimed in audit 1 was not retained. The captured output is the author's
|
||||
record; the local test run is independent verification of the supplied code.
|
||||
Neither is a test of `wf_core`.
|
||||
|
||||
The reference uses a single owner, append-only example contributions,
|
||||
deep-copied transitions, and trusted in-process pickle snapshots. It does not
|
||||
establish production codec validation, durable storage atomicity, arbitrary
|
||||
reducers, external-effect idempotency, foreach, subgraphs, or cancellation.
|
||||
Never load an untrusted pickle checkpoint.
|
||||
|
||||
## Known defects reproduced locally
|
||||
|
||||
### Repeated write execution collides
|
||||
|
||||
```text
|
||||
w: append X -> pick
|
||||
pick.again -> w
|
||||
pick.done -> END
|
||||
```
|
||||
|
||||
`SymbolicAnalyzer` accepts this graph. Execute `w`, choose `again`, then execute
|
||||
`w` using the same token. `_write()` raises
|
||||
`RuntimeError: duplicate semantic contribution w@T0` because its ID uses only
|
||||
node and token identity, not the execution occurrence. This is a valid loop
|
||||
that the simulator must eventually execute correctly.
|
||||
|
||||
### Repeated gather occurrence is not rejected by analysis
|
||||
|
||||
```text
|
||||
g.a -> h.only -> pick
|
||||
pick.again -> h.only
|
||||
pick.done -> final.left
|
||||
g.b -> final.right
|
||||
final -> END
|
||||
```
|
||||
|
||||
The supplied model permits one-port gathers. Analysis accepts the graph with
|
||||
both gather anchors set to `g`. Execute `g`, fire `h`, choose `again`, then fire
|
||||
`h` again: `_fire_gather_impl()` raises
|
||||
`RuntimeError: duplicate merged lineage merge:h@g@T0`.
|
||||
The analyzer tracks current tokens but not past firings under an unresolved
|
||||
anchor. The proposed one-fire-per-occurrence contract requires validation to
|
||||
reject this case. Banning one-port gathers would not prove the general rule.
|
||||
|
||||
Both reproductions were executed independently during review. They are not
|
||||
included in the unchanged imported 25-test suite. Their regression requirements
|
||||
and the next verification gates live in the draft spec, not in this archive.
|
||||
@@ -0,0 +1,3 @@
|
||||
{
|
||||
"default": false
|
||||
}
|
||||
+44
@@ -0,0 +1,44 @@
|
||||
# Fork/Gather Reference Verification Artifacts
|
||||
|
||||
These files are a **reconstruction**, created during the focused verification round. The scratch code implicitly referred to in the earlier audit was not available as a persisted artifact.
|
||||
|
||||
Files:
|
||||
|
||||
- `fork_gather_reference.py` — independent graph model, runtime simulator, checkpoint/recovery, merge policies, symbolic correlation/discharge analyzer, finite re-entry checker, and demo graph builders.
|
||||
- `test_fork_gather_reference.py` — executable `unittest` suite.
|
||||
|
||||
No production repository code is imported.
|
||||
|
||||
## Requirements
|
||||
|
||||
Python 3.11+; standard library only.
|
||||
|
||||
## Run all executed tests
|
||||
|
||||
```bash
|
||||
cd /path/to/files
|
||||
python -m unittest -v test_fork_gather_reference.py
|
||||
```
|
||||
|
||||
## Run focused demos
|
||||
|
||||
```bash
|
||||
python fork_gather_reference.py ordering-strict
|
||||
python fork_gather_reference.py ordering-port
|
||||
python fork_gather_reference.py checkpoint
|
||||
python fork_gather_reference.py analysis
|
||||
```
|
||||
|
||||
`ordering-strict` demonstrates that the authored `hx=[A,B]`, `hy=[B,A]` graph is control-valid but the final merge has a precedence cycle.
|
||||
|
||||
`ordering-port` uses final-gather port order plus stable first-occurrence deduplication and completes as `ROOT,A,B`.
|
||||
|
||||
`checkpoint` snapshots a stable runtime with a ready gather, reconstructs it with `pickle`, then completes the run.
|
||||
|
||||
`analysis` runs the finite symbolic analyzer across the requested graph classes.
|
||||
|
||||
## Persistence contract modeled
|
||||
|
||||
A runtime semantic transition mutates a deep-copied candidate state. The candidate replaces the externally visible state only after the transition succeeds and runtime invariants pass. `checkpoint()` is allowed only between completed semantic transitions. Consequently, an exception during a transition cannot expose its partially mutated candidate to checkpointing.
|
||||
|
||||
This models consistent snapshots and recovery from the last durable checkpoint. It does **not** model recovery from partially persisted internal transitions and does **not** claim exactly-once external side effects.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,700 @@
|
||||
The exact scratch code behind my previous audit was **not persisted and is no longer accessible**. The artifacts below are a **reconstruction** of those semantics, written and executed in this verification round. They do not import or implement against the production repository.
|
||||
|
||||
### Runnable artifacts
|
||||
|
||||
[Download the complete artifact bundle](sandbox:/mnt/data/fork_gather_reference_artifacts.zip)
|
||||
|
||||
Individual files: [simulator + analyzer](sandbox:/mnt/data/fork_gather_reference.py), [25-test suite](sandbox:/mnt/data/test_fork_gather_reference.py), [README / commands](sandbox:/mnt/data/README_fork_gather_reference.md), and [captured test output](sandbox:/mnt/data/verification_test_results.txt).
|
||||
|
||||
Run them with:
|
||||
|
||||
```bash
|
||||
python -m unittest -v test_fork_gather_reference.py
|
||||
|
||||
python fork_gather_reference.py ordering-strict
|
||||
python fork_gather_reference.py ordering-port
|
||||
python fork_gather_reference.py checkpoint
|
||||
python fork_gather_reference.py analysis
|
||||
```
|
||||
|
||||
I executed all four code paths represented by the tests/demos. The final suite result was:
|
||||
|
||||
```text
|
||||
Ran 25 tests in 0.024s
|
||||
|
||||
OK
|
||||
```
|
||||
|
||||
Those are **actual executions of the reconstructed reference implementation**. Nothing below relies on the earlier claimed execution.
|
||||
|
||||
## 1. The ordering counterexample breaks one conclusion from the prior audit
|
||||
|
||||
Your graph is accepted by the reconstructed control analyzer.
|
||||
|
||||
It derives:
|
||||
|
||||
```text
|
||||
hx anchor = r
|
||||
hy anchor = r
|
||||
final anchor = r
|
||||
|
||||
hx discharges: {}
|
||||
hy discharges: {}
|
||||
final discharges: {fa, fb, r}
|
||||
```
|
||||
|
||||
After the two inner gathers, the actual stable contribution IDs were:
|
||||
|
||||
```text
|
||||
hx = [
|
||||
A@r@T0/a,
|
||||
B@r@T0/b,
|
||||
]
|
||||
|
||||
hy = [
|
||||
B@r@T0/b,
|
||||
A@r@T0/a,
|
||||
]
|
||||
```
|
||||
|
||||
or by labels:
|
||||
|
||||
```text
|
||||
hx = [A, B]
|
||||
hy = [B, A]
|
||||
```
|
||||
|
||||
There is no existing control rule in the reconstructed model that rejects this. Every branch obligation is consumed exactly once, and the final normalization restores the owner root.
|
||||
|
||||
So my previous statement that opposite histories imply "corruption" was wrong.
|
||||
|
||||
The correct classification is:
|
||||
|
||||
> **This is a legitimate authored control graph that produces incompatible merge-order constraints under one possible definition of merge-history preservation.**
|
||||
|
||||
The metadata itself is not malformed.
|
||||
|
||||
### Causal order versus gather-imposed order
|
||||
|
||||
This distinction matters.
|
||||
|
||||
In your graph:
|
||||
|
||||
```text
|
||||
r.a -> A -> fa
|
||||
r.b -> B -> fb
|
||||
```
|
||||
|
||||
`A` and `B` are concurrent sibling writes. There is no causal dependency:
|
||||
|
||||
```text
|
||||
A -> B
|
||||
```
|
||||
|
||||
or:
|
||||
|
||||
```text
|
||||
B -> A
|
||||
```
|
||||
|
||||
`hx` later imposes:
|
||||
|
||||
```text
|
||||
A < B
|
||||
```
|
||||
|
||||
because its ports are `(left,right)`.
|
||||
|
||||
`hy` independently imposes:
|
||||
|
||||
```text
|
||||
B < A
|
||||
```
|
||||
|
||||
because its left input happens to originate from `B`.
|
||||
|
||||
Those are **merge serialization orders**, not original happens-before dependencies.
|
||||
|
||||
The architectural question is therefore whether an earlier gather's serialization order becomes a permanent constraint on later reconvergence.
|
||||
|
||||
That is a policy decision, not a provenance-integrity invariant.
|
||||
|
||||
---
|
||||
|
||||
# 2. Policy A: preserve previous merge orders, error on cycles
|
||||
|
||||
The implemented `STRICT_PRECEDENCE` policy treats each materialized history as an ordering constraint.
|
||||
|
||||
Thus:
|
||||
|
||||
```text
|
||||
hx: A, B => A < B
|
||||
hy: B, A => B < A
|
||||
```
|
||||
|
||||
At `final`, their union contains a cycle:
|
||||
|
||||
```text
|
||||
A < B < A
|
||||
```
|
||||
|
||||
The actual simulator result is:
|
||||
|
||||
```text
|
||||
MergeOrderConflict:
|
||||
incompatible contribution-order constraints;
|
||||
cycle involves A@r@T0/a, B@r@T0/b
|
||||
```
|
||||
|
||||
Importantly, `hx` and `hy` both succeed. **Only `final` fails.**
|
||||
|
||||
This policy preserves:
|
||||
|
||||
* exactly-once contribution identity;
|
||||
* every previously established ordering when merging succeeds;
|
||||
* schedule independence;
|
||||
* order-sensitive reducer semantics as a coherent accumulated history;
|
||||
* the property that a later merge never silently reverses an order already visible in an earlier merged lineage.
|
||||
|
||||
It sacrifices composability: a completely legitimate control graph can reach a deterministic merge conflict because independently serialized concurrent histories are incompatible.
|
||||
|
||||
Given your existing preference `conflicts="error"`, I think this is the cleaner first policy. The error should be called something like:
|
||||
|
||||
```text
|
||||
merge-order conflict
|
||||
```
|
||||
|
||||
not corruption.
|
||||
|
||||
## Global cycle detection is necessary
|
||||
|
||||
Checking histories pairwise is insufficient.
|
||||
|
||||
I executed:
|
||||
|
||||
```text
|
||||
H1 = [A, B]
|
||||
H2 = [B, C]
|
||||
H3 = [C, A]
|
||||
```
|
||||
|
||||
A straightforward pairwise shared-subsequence agreement predicate returns:
|
||||
|
||||
```text
|
||||
True
|
||||
```
|
||||
|
||||
because each pair shares only one contribution:
|
||||
|
||||
```text
|
||||
H1 ∩ H2 = {B}
|
||||
H2 ∩ H3 = {C}
|
||||
H3 ∩ H1 = {A}
|
||||
```
|
||||
|
||||
There is no pairwise disagreement to observe.
|
||||
|
||||
But their combined constraints are:
|
||||
|
||||
```text
|
||||
A < B
|
||||
B < C
|
||||
C < A
|
||||
```
|
||||
|
||||
and the implemented global precedence check produces:
|
||||
|
||||
```text
|
||||
MergeOrderConflict:
|
||||
cycle involves A, B, C
|
||||
```
|
||||
|
||||
Therefore:
|
||||
|
||||
> Pairwise agreement on shared IDs is not sufficient. Strict preservation requires building the union precedence graph and checking it globally for cycles.
|
||||
|
||||
The implementation uses adjacent sequence pairs:
|
||||
|
||||
```text
|
||||
[A,B,C] -> A→B, B→C
|
||||
```
|
||||
|
||||
rather than every ordered pair. Transitivity supplies the rest. It then performs deterministic topological sorting.
|
||||
|
||||
---
|
||||
|
||||
# 3. Policy B: final gather port order wins
|
||||
|
||||
The implemented `FINAL_PORT_ORDER` policy performs stable first-occurrence deduplication in the declared final port order.
|
||||
|
||||
For:
|
||||
|
||||
```text
|
||||
final.left = hx = [A, B]
|
||||
final.right = hy = [B, A]
|
||||
```
|
||||
|
||||
the scan is:
|
||||
|
||||
```text
|
||||
left:
|
||||
A -> retain
|
||||
B -> retain
|
||||
|
||||
right:
|
||||
B -> already present
|
||||
A -> already present
|
||||
```
|
||||
|
||||
so:
|
||||
|
||||
```text
|
||||
final = [A, B]
|
||||
```
|
||||
|
||||
The actual simulator commits:
|
||||
|
||||
```text
|
||||
["ROOT", "A", "B"]
|
||||
```
|
||||
|
||||
Reversing the histories supplied to the same primitive produces `[B,A]`, as expected.
|
||||
|
||||
This policy preserves:
|
||||
|
||||
* one occurrence per contribution ID;
|
||||
* deterministic behavior;
|
||||
* schedule independence;
|
||||
* declared **current gather** port precedence;
|
||||
* very simple runtime semantics.
|
||||
|
||||
It sacrifices a stronger property:
|
||||
|
||||
> An ordering already observable in an earlier merged state is not necessarily preserved after later reconvergence.
|
||||
|
||||
A node downstream of `hy` could have observed:
|
||||
|
||||
```text
|
||||
ROOT, B, A
|
||||
```
|
||||
|
||||
while the final continuation observes:
|
||||
|
||||
```text
|
||||
ROOT, A, B
|
||||
```
|
||||
|
||||
That is not a causal contradiction—A and B were originally concurrent—but it means a lineage history is no longer interpretable as a monotonically accumulated global ordering.
|
||||
|
||||
For an order-sensitive reducer such as append, that is a substantial semantic choice.
|
||||
|
||||
### Recommendation between A and B
|
||||
|
||||
For the prototype, I would choose **A: strict precedence + deterministic merge error**.
|
||||
|
||||
That matches `conflicts="error"` without confusing a legitimate authored conflict with malformed metadata. It also avoids making partial-gather ordering retroactively disposable.
|
||||
|
||||
If later experience shows that gather order should be a purely local projection rather than historical ordering, policy B can be introduced explicitly. I would not make that interpretation implicit.
|
||||
|
||||
---
|
||||
|
||||
# 4. Correlation inference is now independently executable
|
||||
|
||||
The reconstructed `SymbolicAnalyzer` does **not** receive manually supplied gather anchors or fork-discharge tables.
|
||||
|
||||
Its supported graph class is intentionally narrow:
|
||||
|
||||
* finite flat graph;
|
||||
* one initial owner token;
|
||||
* concurrency arises only from explicit `ForkNode`;
|
||||
* a fork simultaneously emits all declared branches;
|
||||
* ordinary outcomes have exactly one successor;
|
||||
* `ChoiceNode` alternatives are explored nondeterministically;
|
||||
* gathers have fixed statically declared ports;
|
||||
* each gather firing consumes exactly one arrival per port;
|
||||
* no unresolved re-entry of the same static fork along a token's referenced ancestry;
|
||||
* no foreach/subgraph/failure/interrupt semantics in this verifier.
|
||||
|
||||
That is enough to test the correlation issue in isolation.
|
||||
|
||||
## Abstract state
|
||||
|
||||
A symbolic activation is:
|
||||
|
||||
```python
|
||||
SActivation(
|
||||
fork_id,
|
||||
parent_claim,
|
||||
)
|
||||
```
|
||||
|
||||
A branch obligation is:
|
||||
|
||||
```python
|
||||
SBranch(
|
||||
activation,
|
||||
branch,
|
||||
)
|
||||
```
|
||||
|
||||
So two sibling activations of the same static fork are distinguishable.
|
||||
|
||||
For example:
|
||||
|
||||
```text
|
||||
p.left -> g
|
||||
p.right -> g
|
||||
```
|
||||
|
||||
produces abstract activations equivalent to:
|
||||
|
||||
```text
|
||||
g<[p.left]>
|
||||
g<[p.right]>
|
||||
```
|
||||
|
||||
rather than merely:
|
||||
|
||||
```text
|
||||
g
|
||||
g
|
||||
```
|
||||
|
||||
A marking is a multiset of symbolic tokens located either:
|
||||
|
||||
```text
|
||||
at node N
|
||||
```
|
||||
|
||||
or:
|
||||
|
||||
```text
|
||||
parked at gather H.port
|
||||
```
|
||||
|
||||
with their complete current control claims.
|
||||
|
||||
## Fork transfer
|
||||
|
||||
Executing `g` with claim `C` creates:
|
||||
|
||||
```text
|
||||
activation = g<C>
|
||||
```
|
||||
|
||||
and emits:
|
||||
|
||||
```text
|
||||
g<C>.x
|
||||
g<C>.y
|
||||
...
|
||||
```
|
||||
|
||||
Before doing so, the analyzer recursively examines the incoming claim. If static `g` already occurs in any referenced unresolved obligation, that transition is rejected as:
|
||||
|
||||
```text
|
||||
unresolved_reentry:g
|
||||
```
|
||||
|
||||
It does not unroll another `g`.
|
||||
|
||||
## Gather inference
|
||||
|
||||
When one arrival exists for every port, the analyzer enumerates complete port combinations.
|
||||
|
||||
For each combination it computes the **deepest common enclosing symbolic activation**.
|
||||
|
||||
That dynamically observed activation is the candidate correlation anchor.
|
||||
|
||||
The analyzer does not need to know in advance what the gather discharges.
|
||||
|
||||
It then unions the input claims and repeatedly performs:
|
||||
|
||||
```text
|
||||
all branches of activation X present
|
||||
->
|
||||
replace them with X.parent_claim
|
||||
```
|
||||
|
||||
The activations removed by that normalization are the observed discharges for this firing.
|
||||
|
||||
Only after exploring all reachable markings does it require that each static gather have one static anchor fork across every possible firing.
|
||||
|
||||
That avoids the previous circular construction:
|
||||
|
||||
```text
|
||||
need anchor to infer firing
|
||||
need firing to infer discharge
|
||||
need discharge to infer anchor
|
||||
```
|
||||
|
||||
For this supported class, firing combinations are enumerated from the actual symbolic marking first; anchor and discharge are observations of each combination.
|
||||
|
||||
---
|
||||
|
||||
# 5. Actual analyzer results
|
||||
|
||||
| Graph | Result | Derived anchor(s) | Observed discharge |
|
||||
| ----------------------------------------------------------- | ---------- | ------------------------- | ---------------------------- |
|
||||
| Ordinary `g -> a,b -> h` | accept | `h → g` | `h: {g}` |
|
||||
| Direct `g -> h.left,h.right` | accept | `h → g` | `h: {g}` |
|
||||
| Partial `a+b`, then result+`c` | accept | both → `g` | partial `{}`, final `{g}` |
|
||||
| Sibling nested cross-gathers | accept | `h1,h2,final → r` | final `{fa,fd,r}` |
|
||||
| Your A/B ordering graph | **accept** | `hx,hy,final → r` | final `{fa,fb,r}` |
|
||||
| Conditional `d OR e → h.right` | accept | `h → g` | `{g}` |
|
||||
| Simultaneous two arrivals to `h.right` | **reject** | `h → g` observed | duplicate compatible arrival |
|
||||
| Completed round `g→h→repeat→g` | accept | `h → g` | `{g}` |
|
||||
| Unresolved `g...→g` recursion | **reject** | — | unresolved re-entry |
|
||||
| Two sibling activations of same static `g` feeding same `h` | **reject** | both `g` and `p` observed | ambiguous correlation |
|
||||
|
||||
The completed-round analysis reached a fixed point after **8 symbolic markings**. The unresolved-recursion graph rejected after **9**.
|
||||
|
||||
The separate simpler `(node, open_static_forks)` re-entry checker examined six states in each of its two executed cases:
|
||||
|
||||
```text
|
||||
completed round:
|
||||
accepted
|
||||
6 states
|
||||
|
||||
unresolved recursion:
|
||||
rejected: unresolved_reentry:g
|
||||
6 states
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# 6. The two-sibling-same-static-fork case reveals the real conservative boundary
|
||||
|
||||
Consider:
|
||||
|
||||
```text
|
||||
Fork p(branches=l,r)
|
||||
|
||||
p.l -> g
|
||||
p.r -> g
|
||||
|
||||
Fork g(branches=x,y)
|
||||
|
||||
g.x -> h.left
|
||||
g.y -> h.right
|
||||
```
|
||||
|
||||
The symbolic state contains:
|
||||
|
||||
```text
|
||||
g<[p.l]>.x
|
||||
g<[p.l]>.y
|
||||
|
||||
g<[p.r]>.x
|
||||
g<[p.r]>.y
|
||||
```
|
||||
|
||||
There are sensible same-activation combinations:
|
||||
|
||||
```text
|
||||
g<[p.l]>.x + g<[p.l]>.y
|
||||
anchor = g<[p.l]>
|
||||
|
||||
g<[p.r]>.x + g<[p.r]>.y
|
||||
anchor = g<[p.r]>
|
||||
```
|
||||
|
||||
But ports alone also permit:
|
||||
|
||||
```text
|
||||
g<[p.l]>.x + g<[p.r]>.y
|
||||
```
|
||||
|
||||
whose deepest common encloser is:
|
||||
|
||||
```text
|
||||
p
|
||||
```
|
||||
|
||||
and similarly for the opposite cross-pair.
|
||||
|
||||
The executed analyzer therefore observes:
|
||||
|
||||
```text
|
||||
h anchors = {g, p}
|
||||
```
|
||||
|
||||
and rejects:
|
||||
|
||||
```text
|
||||
non_unique_static_anchor:h:['g', 'p']
|
||||
```
|
||||
|
||||
It also observes duplicate compatible arrivals under the `p` interpretation.
|
||||
|
||||
This is a useful result because it demonstrates that:
|
||||
|
||||
> Static port identity + provenance + deepest-common-encloser does **not** solve general correlation.
|
||||
|
||||
For this prototype grammar, the exact restriction is:
|
||||
|
||||
> A gather is accepted only if every reachable complete port combination yields the same static anchor fork, and each dynamic anchor can have at most one compatible arrival per port.
|
||||
|
||||
An explicit correlation declaration could make more graphs expressible later. The prototype does not need one yet.
|
||||
|
||||
---
|
||||
|
||||
# 7. Why the symbolic analysis terminates
|
||||
|
||||
This is a demonstrated property of the implemented restricted analysis, not a claim about unrestricted workflow graphs.
|
||||
|
||||
An activation is recursively:
|
||||
|
||||
```text
|
||||
(static fork ID, parent claim)
|
||||
```
|
||||
|
||||
and unresolved re-entry of an already referenced static fork is rejected.
|
||||
|
||||
Therefore a single obligation ancestry cannot contain the same static fork twice, so nesting depth is bounded by the finite number of fork nodes.
|
||||
|
||||
Branch sets and static node positions are finite. Forking can create exponentially many combinations, but without same-fork unresolved recursion it cannot create an unbounded activation ancestry. Ordinary cycles eventually revisit an existing marking. Fully converged rounds similarly restore their parent claim, so another round produces the same symbolic abstract state and hits the visited set.
|
||||
|
||||
Thus the state space is finite for this supported class.
|
||||
|
||||
The practical cost is still poor in the worst case: explicit concurrent marking exploration is combinatorial. The implementation has a `max_markings` guard. I would treat this as a correctness prototype, not the intended production validator algorithm.
|
||||
|
||||
---
|
||||
|
||||
# 8. Durability correction: no journal is justified by the stated contract
|
||||
|
||||
My previous recommendation overreached here.
|
||||
|
||||
The current persistence contract you supplied is:
|
||||
|
||||
> externally stopped states are persisted: interrupted, completed, failed.
|
||||
|
||||
It does **not** promise that arbitrary internal fork/gather mutations are individually durable.
|
||||
|
||||
Under that contract, a gather transition journal is not required merely because an internal transition has several logical substeps.
|
||||
|
||||
The reconstructed runtime instead models:
|
||||
|
||||
```python
|
||||
candidate = deepcopy(stable_state)
|
||||
|
||||
perform_complete_semantic_transition(candidate)
|
||||
validate(candidate)
|
||||
|
||||
# publish only after successful completion
|
||||
stable_state = candidate
|
||||
```
|
||||
|
||||
`checkpoint()` serializes only `stable_state`.
|
||||
|
||||
That is deliberately illustrative rather than a production implementation prescription.
|
||||
|
||||
### What was actually tested
|
||||
|
||||
The 25-test run includes:
|
||||
|
||||
**Checkpoint round trip.** A complete state containing live tokens, a ready gather, fork visits, lineages, contributions, and gather buckets was serialized and reconstructed exactly. This is actually *stronger* than your currently promised checkpoint surface because the reference test snapshots an internally runnable state.
|
||||
|
||||
**Recovery from last durable checkpoint.** The simulator checkpointed, performed additional uncheckpointed gather/write progress, discarded that process state, reconstructed the old checkpoint, re-executed from there, and completed as:
|
||||
|
||||
```text
|
||||
ROOT, S, C, D
|
||||
```
|
||||
|
||||
**Exception isolation.** Under strict ordering, `final` raises `MergeOrderConflict`. Because it executes against a candidate copy, the published runtime state after the exception is byte-for-semantic-state identical to the state immediately before attempting the merge.
|
||||
|
||||
**Shared-write recovery.** After checkpoint/recovery, reconvergence still produced:
|
||||
|
||||
```text
|
||||
S, C, D
|
||||
```
|
||||
|
||||
rather than:
|
||||
|
||||
```text
|
||||
S, C, S, D
|
||||
```
|
||||
|
||||
because stable contribution IDs survived the checkpoint.
|
||||
|
||||
### Four durability questions should remain separate
|
||||
|
||||
| Problem | Required here? | Mechanism |
|
||||
| ----------------------------------------------------- | ----------------------------- | ---------------------------------------------------------------- |
|
||||
| Consistent supported checkpoint | **Yes** | snapshot only a stable semantic state |
|
||||
| Process crash after uncheckpointed progress | **Yes** | recover last durable state; later work may rerun |
|
||||
| Recovery from half-persisted internal gather mutation | **No, given stated contract** | don't persist such states; use atomic snapshot storage |
|
||||
| Exactly-once external side effect | **Not solved here** | requires idempotency/dedup/activity semantics at effect boundary |
|
||||
|
||||
There is one implementation consequence worth retaining.
|
||||
|
||||
If an operational exception causes the runtime to persist a **failed** stopped run, that failed snapshot should be derived from:
|
||||
|
||||
```text
|
||||
last stable pre-transition state
|
||||
+ structured failure information
|
||||
```
|
||||
|
||||
not from an in-place `RunState` that may have been partially mutated before the exception.
|
||||
|
||||
That can be achieved with a transition patch, copy-on-write, rollback, or another commit boundary. It does **not** imply a durable transition journal.
|
||||
|
||||
Exactly-once external effects are different. If:
|
||||
|
||||
```text
|
||||
checkpoint
|
||||
-> call payment API
|
||||
-> process crashes before next checkpoint
|
||||
```
|
||||
|
||||
recovery can invoke the API again. Fork/gather contribution IDs cannot undo or intrinsically deduplicate that external effect.
|
||||
|
||||
---
|
||||
|
||||
# 9. What the artifacts now demonstrate—and what they don't
|
||||
|
||||
Demonstrated by execution:
|
||||
|
||||
* direct fork→gather preserves destination ports;
|
||||
* partial gathers can retain unresolved obligations and later complete them;
|
||||
* cross-gathers can be correlated for the tested sibling-nested topology;
|
||||
* your ordering graph is control-valid;
|
||||
* `[A,B]` and `[B,A]` are legitimate intermediate histories;
|
||||
* strict precedence rejects their final reconvergence;
|
||||
* final-port-order deterministically accepts it;
|
||||
* global precedence cycles can exist despite pairwise agreement;
|
||||
* stable contribution IDs deduplicate shared history;
|
||||
* two alternate schedules of the shared-write adversarial graph both commit `ROOT,S,C,D`;
|
||||
* checkpoint/recovery preserves tokens, gathers, lineages, contributions and correlation state;
|
||||
* unresolved same-static-fork re-entry can be rejected with a terminating finite analysis;
|
||||
* completed rounds are not falsely rejected;
|
||||
* two sibling activations of one static fork expose a real correlation ambiguity.
|
||||
|
||||
Still assumptions or intentionally unsupported:
|
||||
|
||||
* scalability of explicit marking exploration;
|
||||
* foreach/subgraph owner correlation;
|
||||
* interrupts and cancellation;
|
||||
* operational failures during concurrent branches;
|
||||
* correlation for graphs outside the stated grammar;
|
||||
* general workflow soundness or termination;
|
||||
* exactly-once external effects;
|
||||
* whether earlier gather serialization should semantically constrain all later reconvergences.
|
||||
|
||||
---
|
||||
|
||||
# 10. Narrow recommendation for the next prototype
|
||||
|
||||
I would prototype exactly this much:
|
||||
|
||||
1. **Keep explicit forks, destination gather ports, and token replacement.**
|
||||
2. Represent symbolic/runtime fork activations by fork identity plus the consumed parent control claim; don't use static fork ID alone for correlation.
|
||||
3. Initially accept a gather only when finite analysis proves one static deepest-common-encloser across all reachable firing combinations and at most one compatible arrival per port.
|
||||
4. Reject unresolved same-static-fork re-entry. Permit loops after normalization fully discharges that fork.
|
||||
5. Treat the two-sibling-same-static-fork/shared-gather topology as unsupported rather than inventing a correlation heuristic.
|
||||
6. If shared pending histories can reconverge, retain stable contribution identities.
|
||||
7. For noncommutative reducers, prototype **strict global precedence checking** and report cycles as ordinary merge conflicts.
|
||||
8. Keep persistence at atomic, consistent externally supported stop boundaries. **Do not add a transition journal** for fork/gather at this stage.
|
||||
9. Keep exactly-once external effect semantics out of this prototype.
|
||||
|
||||
The principal remaining semantic decision is now much narrower than before: **does a partial gather's deterministic ordering become durable history, or is it merely that continuation's local state view?** Policy A and B correspond almost exactly to those two interpretations. The control/correlation model no longer needs to answer that question for you.
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
+289
@@ -0,0 +1,289 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
|
||||
from fork_gather_reference import (
|
||||
MergeOrderConflict,
|
||||
MergePolicy,
|
||||
Runtime,
|
||||
SymbolicAnalyzer,
|
||||
finite_reentry_check,
|
||||
graph_adversarial_shared_reconvergence,
|
||||
graph_basic,
|
||||
graph_completed_round_loop,
|
||||
graph_conditional_alternative,
|
||||
graph_cross_gather,
|
||||
graph_direct_fork_gather,
|
||||
graph_duplicate_arrival,
|
||||
graph_ordering_counterexample,
|
||||
graph_partial,
|
||||
graph_three_history_order_cycle,
|
||||
graph_two_sibling_same_fork_analysis_only,
|
||||
graph_unresolved_reentry,
|
||||
precedence_merge,
|
||||
pairwise_shared_order_agrees,
|
||||
stable_ordered_dedup,
|
||||
)
|
||||
|
||||
|
||||
class CorrelationAnalysisTests(unittest.TestCase):
|
||||
def assertAcceptedAnchor(self, graph, expected):
|
||||
result = SymbolicAnalyzer(graph).analyze()
|
||||
self.assertTrue(result.accepted, result.errors)
|
||||
self.assertEqual(result.unique_anchor_plan(), expected)
|
||||
return result
|
||||
|
||||
def test_ordinary_fork_gather(self):
|
||||
r = self.assertAcceptedAnchor(graph_basic(), {"h": "g"})
|
||||
self.assertEqual(r.always_discharged_by("h"), {"g"})
|
||||
|
||||
def test_direct_fork_to_gather(self):
|
||||
r = self.assertAcceptedAnchor(graph_direct_fork_gather(), {"h": "g"})
|
||||
self.assertEqual(r.always_discharged_by("h"), {"g"})
|
||||
|
||||
def test_partial_gather_then_final(self):
|
||||
r = self.assertAcceptedAnchor(graph_partial(), {"h_ab": "g", "h_final": "g"})
|
||||
self.assertEqual(r.always_discharged_by("h_ab"), set())
|
||||
self.assertEqual(r.always_discharged_by("h_final"), {"g"})
|
||||
|
||||
def test_cross_gathers(self):
|
||||
r = self.assertAcceptedAnchor(graph_cross_gather(), {"h1": "r", "h2": "r", "final": "r"})
|
||||
self.assertEqual(r.always_discharged_by("h1"), set())
|
||||
self.assertEqual(r.always_discharged_by("h2"), set())
|
||||
self.assertEqual(r.always_discharged_by("final"), {"fa", "fd", "r"})
|
||||
|
||||
def test_ordering_counterexample_control_is_accepted(self):
|
||||
r = self.assertAcceptedAnchor(
|
||||
graph_ordering_counterexample(),
|
||||
{"hx": "r", "hy": "r", "final": "r"},
|
||||
)
|
||||
self.assertEqual(r.always_discharged_by("final"), {"fa", "fb", "r"})
|
||||
|
||||
def test_mutually_exclusive_alternatives_same_port(self):
|
||||
self.assertAcceptedAnchor(graph_conditional_alternative(), {"h": "g"})
|
||||
|
||||
def test_simultaneous_duplicate_arrivals_same_port_are_rejected(self):
|
||||
r = SymbolicAnalyzer(graph_duplicate_arrival()).analyze()
|
||||
self.assertFalse(r.accepted)
|
||||
self.assertTrue(any(e.startswith("duplicate_compatible_arrivals:h.right") for e in r.errors), r.errors)
|
||||
|
||||
def test_completed_round_loop_terminates_and_is_accepted(self):
|
||||
r = self.assertAcceptedAnchor(graph_completed_round_loop(), {"h": "g"})
|
||||
self.assertLess(r.explored_markings, 100)
|
||||
|
||||
def test_unresolved_same_fork_reentry_is_rejected(self):
|
||||
r = SymbolicAnalyzer(graph_unresolved_reentry()).analyze()
|
||||
self.assertFalse(r.accepted)
|
||||
self.assertIn("unresolved_reentry:g", r.errors)
|
||||
self.assertLess(r.explored_markings, 100)
|
||||
|
||||
def test_two_sibling_activations_same_static_fork_are_ambiguous(self):
|
||||
r = SymbolicAnalyzer(graph_two_sibling_same_fork_analysis_only()).analyze()
|
||||
self.assertFalse(r.accepted)
|
||||
# Same-activation pairs correlate to g; cross-pairs correlate to p.
|
||||
self.assertIn("non_unique_static_anchor:h:['g', 'p']", r.errors)
|
||||
|
||||
|
||||
class ReentryCheckerTests(unittest.TestCase):
|
||||
def test_finite_checker_accepts_completed_round_loop(self):
|
||||
g = graph_completed_round_loop()
|
||||
analysis = SymbolicAnalyzer(g).analyze()
|
||||
discharge = {"h": analysis.always_discharged_by("h")}
|
||||
r = finite_reentry_check(g, discharge)
|
||||
self.assertTrue(r.accepted, r.errors)
|
||||
self.assertLess(r.explored_states, 100)
|
||||
|
||||
def test_finite_checker_rejects_unresolved_reentry(self):
|
||||
g = graph_unresolved_reentry()
|
||||
# h would discharge g if reached normally; recurse reaches g before h.
|
||||
r = finite_reentry_check(g, {"h": {"g"}})
|
||||
self.assertFalse(r.accepted)
|
||||
self.assertIn("unresolved_reentry:g", r.errors)
|
||||
self.assertLess(r.explored_states, 100)
|
||||
|
||||
|
||||
class MergeOrderingTests(unittest.TestCase):
|
||||
def test_counterexample_has_legitimate_opposite_intermediate_orders(self):
|
||||
graph = graph_ordering_counterexample()
|
||||
analysis = SymbolicAnalyzer(graph).analyze()
|
||||
self.assertTrue(analysis.accepted, analysis.errors)
|
||||
rt = Runtime(graph, analysis.unique_anchor_plan(), merge_policy=MergePolicy.FINAL_PORT_ORDER)
|
||||
rt.step_token("T0")
|
||||
rt.step_token(rt.token_at("A")[0])
|
||||
rt.step_token(rt.token_at("B")[0])
|
||||
rt.step_token(rt.token_at("fa")[0])
|
||||
rt.step_token(rt.token_at("fb")[0])
|
||||
rt.fire_gather("hx")
|
||||
rt.fire_gather("hy")
|
||||
hx = [tid for tid in rt.state.tokens if tid.startswith("out:hx@")][0]
|
||||
hy = [tid for tid in rt.state.tokens if tid.startswith("out:hy@")][0]
|
||||
self.assertEqual([rt.state.contributions[c].value for c in rt.token_history(hx)], ["A", "B"])
|
||||
self.assertEqual([rt.state.contributions[c].value for c in rt.token_history(hy)], ["B", "A"])
|
||||
|
||||
def test_policy_a_rejects_opposite_order_constraints(self):
|
||||
with self.assertRaises(MergeOrderConflict):
|
||||
precedence_merge([["A", "B"], ["B", "A"]])
|
||||
|
||||
def test_policy_b_final_port_order_resolves_by_first_occurrence(self):
|
||||
self.assertEqual(stable_ordered_dedup([["A", "B"], ["B", "A"]]), ["A", "B"])
|
||||
self.assertEqual(stable_ordered_dedup([["B", "A"], ["A", "B"]]), ["B", "A"])
|
||||
|
||||
def test_pairwise_shared_pair_agreement_is_not_sufficient_collectively(self):
|
||||
histories, exc = graph_three_history_order_cycle()
|
||||
self.assertEqual(histories, [["A", "B"], ["B", "C"], ["C", "A"]])
|
||||
self.assertTrue(pairwise_shared_order_agrees(histories))
|
||||
self.assertIsNotNone(exc)
|
||||
|
||||
def test_counterexample_strict_runtime_raises_at_final_not_intermediate(self):
|
||||
graph = graph_ordering_counterexample()
|
||||
analysis = SymbolicAnalyzer(graph).analyze()
|
||||
rt = Runtime(graph, analysis.unique_anchor_plan(), merge_policy=MergePolicy.STRICT_PRECEDENCE)
|
||||
rt.step_token("T0")
|
||||
for node in ("A", "B", "fa", "fb"):
|
||||
rt.step_token(rt.token_at(node)[0])
|
||||
rt.fire_gather("hx")
|
||||
rt.fire_gather("hy")
|
||||
with self.assertRaises(MergeOrderConflict):
|
||||
rt.fire_gather("final")
|
||||
|
||||
def test_counterexample_port_order_runtime_commits_A_B(self):
|
||||
graph = graph_ordering_counterexample()
|
||||
analysis = SymbolicAnalyzer(graph).analyze()
|
||||
rt = Runtime(graph, analysis.unique_anchor_plan(), merge_policy=MergePolicy.FINAL_PORT_ORDER)
|
||||
rt.step_token("T0")
|
||||
for node in ("A", "B", "fa", "fb"):
|
||||
rt.step_token(rt.token_at(node)[0])
|
||||
rt.fire_gather("hx")
|
||||
rt.fire_gather("hy")
|
||||
rt.fire_gather("final")
|
||||
end = rt.token_at("END")[0]
|
||||
self.assertEqual([rt.state.contributions[c].value for c in rt.token_history(end)], ["A", "B"])
|
||||
rt.step_token(end)
|
||||
self.assertEqual(rt.state.committed_values, ["ROOT", "A", "B"])
|
||||
|
||||
|
||||
class RuntimeScheduleTests(unittest.TestCase):
|
||||
def test_direct_fork_to_gather_preserves_ports_at_runtime(self):
|
||||
graph = graph_direct_fork_gather()
|
||||
analysis = SymbolicAnalyzer(graph).analyze()
|
||||
rt = Runtime(graph, analysis.unique_anchor_plan())
|
||||
rt.step_token("T0")
|
||||
self.assertEqual(rt.token_at("h"), [])
|
||||
self.assertEqual(len(rt.ready_gathers()), 1)
|
||||
key = rt.ready_gathers()[0]
|
||||
bucket = rt.state.gather_buckets[key]
|
||||
self.assertEqual(set(bucket.arrivals), {"left", "right"})
|
||||
|
||||
def _finish_adversarial(self, schedule):
|
||||
graph = graph_adversarial_shared_reconvergence()
|
||||
analysis = SymbolicAnalyzer(graph).analyze()
|
||||
rt = Runtime(graph, analysis.unique_anchor_plan(), merge_policy=MergePolicy.FINAL_PORT_ORDER)
|
||||
rt.step_token("T0")
|
||||
for action in schedule:
|
||||
if action in {"S", "C", "D", "k"}:
|
||||
rt.step_token(rt.token_at(action)[0])
|
||||
else:
|
||||
rt.fire_gather(action)
|
||||
rt.step_token(rt.token_at("END")[0])
|
||||
return rt
|
||||
|
||||
def test_adversarial_two_schedules_same_semantic_result(self):
|
||||
first = self._finish_adversarial(["S", "k", "C", "hx", "D", "hy", "final"])
|
||||
second = self._finish_adversarial(["D", "C", "S", "k", "hy", "hx", "final"])
|
||||
self.assertEqual(first.state.committed_values, ["ROOT", "S", "C", "D"])
|
||||
self.assertEqual(second.state.committed_values, ["ROOT", "S", "C", "D"])
|
||||
self.assertEqual(first.state.committed_values, second.state.committed_values)
|
||||
|
||||
|
||||
class CheckpointTests(unittest.TestCase):
|
||||
def _prepared_runtime(self):
|
||||
graph = graph_adversarial_shared_reconvergence()
|
||||
analysis = SymbolicAnalyzer(graph).analyze()
|
||||
self.assertTrue(analysis.accepted, analysis.errors)
|
||||
rt = Runtime(graph, analysis.unique_anchor_plan(), merge_policy=MergePolicy.FINAL_PORT_ORDER)
|
||||
rt.step_token("T0")
|
||||
rt.step_token(rt.token_at("S")[0])
|
||||
rt.step_token(rt.token_at("k")[0])
|
||||
rt.step_token(rt.token_at("C")[0])
|
||||
return graph, analysis, rt
|
||||
|
||||
def test_checkpoint_round_trip_ready_gather(self):
|
||||
graph, analysis, rt = self._prepared_runtime()
|
||||
before = rt.snapshot_dict()
|
||||
blob = rt.checkpoint()
|
||||
recovered = Runtime.recover(
|
||||
graph,
|
||||
analysis.unique_anchor_plan(),
|
||||
blob,
|
||||
merge_policy=MergePolicy.FINAL_PORT_ORDER,
|
||||
)
|
||||
self.assertEqual(recovered.snapshot_dict(), before)
|
||||
self.assertEqual([g for g, _ in recovered.ready_gathers()], ["hx"])
|
||||
|
||||
def test_process_crash_discards_uncheckpointed_progress_and_recovers_last_snapshot(self):
|
||||
graph, analysis, rt = self._prepared_runtime()
|
||||
blob = rt.checkpoint()
|
||||
|
||||
# Progress after the durable checkpoint is intentionally lost.
|
||||
rt.fire_gather("hx")
|
||||
rt.step_token(rt.token_at("D")[0])
|
||||
|
||||
recovered = Runtime.recover(
|
||||
graph,
|
||||
analysis.unique_anchor_plan(),
|
||||
blob,
|
||||
merge_policy=MergePolicy.FINAL_PORT_ORDER,
|
||||
)
|
||||
# Re-execute from the last durable semantic snapshot.
|
||||
recovered.fire_gather("hx")
|
||||
recovered.step_token(recovered.token_at("D")[0])
|
||||
recovered.fire_gather("hy")
|
||||
recovered.fire_gather("final")
|
||||
recovered.step_token(recovered.token_at("END")[0])
|
||||
self.assertEqual(recovered.state.committed_values, ["ROOT", "S", "C", "D"])
|
||||
|
||||
def test_resume_from_checkpoint_matches_no_checkpoint_run(self):
|
||||
graph, analysis, rt = self._prepared_runtime()
|
||||
blob = rt.checkpoint()
|
||||
recovered = Runtime.recover(
|
||||
graph,
|
||||
analysis.unique_anchor_plan(), blob,
|
||||
merge_policy=MergePolicy.FINAL_PORT_ORDER,
|
||||
)
|
||||
for current in (rt, recovered):
|
||||
current.fire_gather("hx")
|
||||
current.step_token(current.token_at("D")[0])
|
||||
current.fire_gather("hy")
|
||||
current.fire_gather("final")
|
||||
current.step_token(current.token_at("END")[0])
|
||||
self.assertEqual(recovered.state.committed_values, rt.state.committed_values)
|
||||
self.assertEqual(recovered.snapshot_dict(), rt.snapshot_dict())
|
||||
|
||||
def test_exception_inside_transition_does_not_publish_partial_mutation(self):
|
||||
graph = graph_ordering_counterexample()
|
||||
analysis = SymbolicAnalyzer(graph).analyze()
|
||||
rt = Runtime(graph, analysis.unique_anchor_plan(), merge_policy=MergePolicy.STRICT_PRECEDENCE)
|
||||
rt.step_token("T0")
|
||||
for node in ("A", "B", "fa", "fb"):
|
||||
rt.step_token(rt.token_at(node)[0])
|
||||
rt.fire_gather("hx")
|
||||
rt.fire_gather("hy")
|
||||
before = rt.snapshot_dict()
|
||||
with self.assertRaises(MergeOrderConflict):
|
||||
rt.fire_gather("final")
|
||||
self.assertEqual(rt.snapshot_dict(), before)
|
||||
|
||||
def test_shared_write_is_deduplicated_after_checkpoint(self):
|
||||
graph, analysis, rt = self._prepared_runtime()
|
||||
blob = rt.checkpoint()
|
||||
rt = Runtime.recover(graph, analysis.unique_anchor_plan(), blob, merge_policy=MergePolicy.FINAL_PORT_ORDER)
|
||||
rt.fire_gather("hx")
|
||||
rt.step_token(rt.token_at("D")[0])
|
||||
rt.fire_gather("hy")
|
||||
rt.fire_gather("final")
|
||||
end = rt.token_at("END")[0]
|
||||
self.assertEqual([rt.state.contributions[c].value for c in rt.token_history(end)], ["S", "C", "D"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main(verbosity=2)
|
||||
+30
@@ -0,0 +1,30 @@
|
||||
test_checkpoint_round_trip_ready_gather (test_fork_gather_reference.CheckpointTests.test_checkpoint_round_trip_ready_gather) ... ok
|
||||
test_exception_inside_transition_does_not_publish_partial_mutation (test_fork_gather_reference.CheckpointTests.test_exception_inside_transition_does_not_publish_partial_mutation) ... ok
|
||||
test_process_crash_discards_uncheckpointed_progress_and_recovers_last_snapshot (test_fork_gather_reference.CheckpointTests.test_process_crash_discards_uncheckpointed_progress_and_recovers_last_snapshot) ... ok
|
||||
test_resume_from_checkpoint_matches_no_checkpoint_run (test_fork_gather_reference.CheckpointTests.test_resume_from_checkpoint_matches_no_checkpoint_run) ... ok
|
||||
test_shared_write_is_deduplicated_after_checkpoint (test_fork_gather_reference.CheckpointTests.test_shared_write_is_deduplicated_after_checkpoint) ... ok
|
||||
test_completed_round_loop_terminates_and_is_accepted (test_fork_gather_reference.CorrelationAnalysisTests.test_completed_round_loop_terminates_and_is_accepted) ... ok
|
||||
test_cross_gathers (test_fork_gather_reference.CorrelationAnalysisTests.test_cross_gathers) ... ok
|
||||
test_direct_fork_to_gather (test_fork_gather_reference.CorrelationAnalysisTests.test_direct_fork_to_gather) ... ok
|
||||
test_mutually_exclusive_alternatives_same_port (test_fork_gather_reference.CorrelationAnalysisTests.test_mutually_exclusive_alternatives_same_port) ... ok
|
||||
test_ordering_counterexample_control_is_accepted (test_fork_gather_reference.CorrelationAnalysisTests.test_ordering_counterexample_control_is_accepted) ... ok
|
||||
test_ordinary_fork_gather (test_fork_gather_reference.CorrelationAnalysisTests.test_ordinary_fork_gather) ... ok
|
||||
test_partial_gather_then_final (test_fork_gather_reference.CorrelationAnalysisTests.test_partial_gather_then_final) ... ok
|
||||
test_simultaneous_duplicate_arrivals_same_port_are_rejected (test_fork_gather_reference.CorrelationAnalysisTests.test_simultaneous_duplicate_arrivals_same_port_are_rejected) ... ok
|
||||
test_two_sibling_activations_same_static_fork_are_ambiguous (test_fork_gather_reference.CorrelationAnalysisTests.test_two_sibling_activations_same_static_fork_are_ambiguous) ... ok
|
||||
test_unresolved_same_fork_reentry_is_rejected (test_fork_gather_reference.CorrelationAnalysisTests.test_unresolved_same_fork_reentry_is_rejected) ... ok
|
||||
test_counterexample_has_legitimate_opposite_intermediate_orders (test_fork_gather_reference.MergeOrderingTests.test_counterexample_has_legitimate_opposite_intermediate_orders) ... ok
|
||||
test_counterexample_port_order_runtime_commits_A_B (test_fork_gather_reference.MergeOrderingTests.test_counterexample_port_order_runtime_commits_A_B) ... ok
|
||||
test_counterexample_strict_runtime_raises_at_final_not_intermediate (test_fork_gather_reference.MergeOrderingTests.test_counterexample_strict_runtime_raises_at_final_not_intermediate) ... ok
|
||||
test_pairwise_shared_pair_agreement_is_not_sufficient_collectively (test_fork_gather_reference.MergeOrderingTests.test_pairwise_shared_pair_agreement_is_not_sufficient_collectively) ... ok
|
||||
test_policy_a_rejects_opposite_order_constraints (test_fork_gather_reference.MergeOrderingTests.test_policy_a_rejects_opposite_order_constraints) ... ok
|
||||
test_policy_b_final_port_order_resolves_by_first_occurrence (test_fork_gather_reference.MergeOrderingTests.test_policy_b_final_port_order_resolves_by_first_occurrence) ... ok
|
||||
test_finite_checker_accepts_completed_round_loop (test_fork_gather_reference.ReentryCheckerTests.test_finite_checker_accepts_completed_round_loop) ... ok
|
||||
test_finite_checker_rejects_unresolved_reentry (test_fork_gather_reference.ReentryCheckerTests.test_finite_checker_rejects_unresolved_reentry) ... ok
|
||||
test_adversarial_two_schedules_same_semantic_result (test_fork_gather_reference.RuntimeScheduleTests.test_adversarial_two_schedules_same_semantic_result) ... ok
|
||||
test_direct_fork_to_gather_preserves_ports_at_runtime (test_fork_gather_reference.RuntimeScheduleTests.test_direct_fork_to_gather_preserves_ports_at_runtime) ... ok
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 25 tests in 0.024s
|
||||
|
||||
OK
|
||||
@@ -0,0 +1,298 @@
|
||||
# Fork/Gather Reference Verification Implementation Plan
|
||||
|
||||
> **For agentic workers:** Use `subagent-driven-development` or
|
||||
> `executing-plans` to implement this plan task-by-task. OpenCode should use
|
||||
> one fresh implementer per task, sequentially, with separate spec and quality
|
||||
> reviews before proceeding. Do not launch production implementation.
|
||||
|
||||
**Goal:** Close two reproduced reference-model defects and deliver executable
|
||||
evidence for the next fork/gather design decision.
|
||||
|
||||
**Architecture:** Copy the archived reference into an isolated experimental
|
||||
directory. Preserve the original evidence, repair occurrence tracking, and
|
||||
exercise compiler/runtime agreement. This produces a verification prototype,
|
||||
not a new production runtime or approval of the entire graph model.
|
||||
|
||||
**Tech Stack:** Repository Python 3.14, standard-library `unittest`, existing
|
||||
Ruff tooling, Markdown lint. No new dependencies or lockfile changes.
|
||||
|
||||
**Spec:** [Draft fork/gather contract](../specs/2026-09-06-fork-gather-design.md).
|
||||
Only its reference-verification gates are implemented here. Production owner
|
||||
integration, step-budget dispatch, and merge-policy selection remain open.
|
||||
|
||||
## Global constraints
|
||||
|
||||
- Do not modify `src/`, production tests, public APIs, or dependencies.
|
||||
- Do not modify `docs/historical/research/2026-09-06-fork-gather/imported/`.
|
||||
- Do not choose strict precedence or final-port order as product policy.
|
||||
- Do not add a transition journal, external-effect guarantees, or compatibility
|
||||
for this throwaway prototype's old checkpoints.
|
||||
- New writes get new occurrence identities; merges retain original identities.
|
||||
- One gather firing per owner/gather/anchor visit; resolved rounds may repeat.
|
||||
- Report analysis bounds as inability to prove, never acceptance.
|
||||
- Work in an isolated worktree; no main-worktree edits, merge, push, or squash.
|
||||
- Do not remove untracked files or reset somebody else's changes.
|
||||
|
||||
## Handoff preflight
|
||||
|
||||
The planning inputs may still be uncommitted in the primary workspace. A new
|
||||
worktree does not inherit those files automatically. Before dispatching a
|
||||
worker, copy the following planning inputs from the primary workspace into the
|
||||
isolated worktree if they are absent or differ from the clean branch baseline:
|
||||
|
||||
- `CONTEXT.md`, `docs/README.md`, and `docs/current_roadmap.md`;
|
||||
- `docs/adr/0006-explicit-fork-and-topology-driven-gather.md`;
|
||||
- `docs/superpowers/specs/2026-09-06-fork-gather-design.md`;
|
||||
- this plan;
|
||||
- `docs/historical/research/2026-09-06-fork-gather/` in full.
|
||||
|
||||
Use read-only access to the primary workspace and copy, never move. Verify
|
||||
SHA-256 equality before starting work. Do not overwrite unrelated dirty files
|
||||
in the destination; stop on a conflict or if the primary source cannot be
|
||||
identified. Commit transferred inputs separately on the isolated branch and
|
||||
record the resulting task base. Do not substitute older docs from branch HEAD.
|
||||
|
||||
Record the worktree path, branch, initial commit, and initial dirty-file list.
|
||||
Read root `AGENTS.md`, `docs/AGENTS.md`, the spec, and the archive README. The
|
||||
archive README is enough to understand the prior findings; reading both long
|
||||
reports is optional unless resolving a specific discrepancy.
|
||||
|
||||
## File ownership
|
||||
|
||||
Create `experiments/fork_gather/` with these files:
|
||||
|
||||
- `fork_gather_reference.py`: editable copy of the reference simulator/analyzer.
|
||||
- `test_fork_gather_reference.py`: copy of the original 25 tests.
|
||||
- `test_occurrences.py`: repeated writes, gather firings, recovery regressions.
|
||||
- `test_agreement.py`: bounded schedules, inference agreement, merge evidence.
|
||||
- `README.md`: commands, supported grammar, experimental status, limitations.
|
||||
- `VERIFICATION.md`: actual commands/results, counterexamples, decision report.
|
||||
|
||||
The copied flat module is an explicitly disposable research artifact, not a
|
||||
production package-design precedent. Avoid a gratuitous module split in this
|
||||
verification wave; document non-obvious changes near their implementation.
|
||||
|
||||
## Task 1: Distinguish write executions from token identity
|
||||
|
||||
**Files:** Create the experimental directory, copy the two Python files above,
|
||||
create `test_occurrences.py` and `README.md`. Edit only the experimental copies.
|
||||
|
||||
**Consumes:** Archived `Graph`, `Runtime`, `RuntimeState`, `_write`, `_atomic`,
|
||||
`checkpoint`, and `recover` interfaces.
|
||||
|
||||
**Produces:** Existing public interfaces unchanged; persisted per-token
|
||||
execution counters in `RuntimeState`. Write identity includes the node-execution
|
||||
occurrence, not only the static node and token.
|
||||
|
||||
- [ ] Copy the two Python sources with `Copy-Item`, not move. Record SHA-256
|
||||
hashes of the originals before starting. Run the copied 25-test baseline:
|
||||
|
||||
```powershell
|
||||
Push-Location experiments/fork_gather
|
||||
try { python -B -m unittest -v test_fork_gather_reference.py }
|
||||
finally { Pop-Location }
|
||||
```
|
||||
|
||||
- [ ] Add this graph and a regression that executes two writes, then ends:
|
||||
|
||||
```python
|
||||
graph = Graph("w", {
|
||||
"w": WriteNode("w", "X"),
|
||||
"pick": ChoiceNode("pick", ("again", "done")),
|
||||
"end": EndNode("end"),
|
||||
}, (
|
||||
Edge("w", "ok", "pick"),
|
||||
Edge("pick", "again", "w"),
|
||||
Edge("pick", "done", "end"),
|
||||
))
|
||||
rt = Runtime(graph, {})
|
||||
rt.step_token("T0")
|
||||
rt.step_token("T0", outcome="again")
|
||||
rt.step_token("T0")
|
||||
rt.step_token("T0", outcome="done")
|
||||
rt.step_token("T0")
|
||||
assert rt.state.committed_values == ["ROOT", "X", "X"]
|
||||
assert len(rt.state.contributions) == 2
|
||||
```
|
||||
|
||||
- [ ] Run `python -B -m unittest -v test_occurrences.py` from the experimental
|
||||
directory. Confirm failure is `duplicate semantic contribution w@T0`.
|
||||
- [ ] Add `execution_counts: dict[str, int]` with a default factory to
|
||||
`RuntimeState`. Increment only inside the candidate-state transaction for a
|
||||
valid node dispatch; use the token-local count in each fresh write ID:
|
||||
|
||||
```python
|
||||
occurrence = s.execution_counts.get(token.id, 0) + 1
|
||||
s.execution_counts[token.id] = occurrence
|
||||
# _write reads this count after _step_token_impl admits the execution.
|
||||
contribution_id = f"{node.id}@{token.id}:execution:{occurrence}:write:0"
|
||||
```
|
||||
|
||||
The reference has one write per WriteNode execution. Document that limitation;
|
||||
do not invent multi-write capabilities. Failed candidate transitions must
|
||||
leave the published counter unchanged. Do not use a global scheduling counter
|
||||
that changes contribution IDs when sibling execution order changes.
|
||||
- [ ] Add checkpoint coverage after the first write and before the second.
|
||||
Continue both original and recovered states through the same choices. Assert
|
||||
equal contribution ID sets and two final X values. Keep shared-history
|
||||
deduplication tests passing so fresh IDs are not minted during a merge.
|
||||
- [ ] Run the baseline and occurrence suites. Document commands and the identity
|
||||
rule in the experiment README. Commit only Task 1 files, then obtain separate
|
||||
spec and quality reviews. Close concrete findings before Task 2.
|
||||
|
||||
## Task 2: Track fired gathers through unresolved activations
|
||||
|
||||
**Files:** Modify the experimental reference and `test_occurrences.py`.
|
||||
|
||||
**Consumes:** Existing `SActivation`, `SToken`, claim normalization/enclosure,
|
||||
`SymbolicAnalyzer.analyze()`, and runtime gather transitions.
|
||||
|
||||
**Produces:** Same `AnalysisResult` interface, with a structured diagnostic
|
||||
string prefix `repeated_gather:` for the forbidden repeated occurrence.
|
||||
Runtime raises `RepeatedGatherError`, not incidental duplicate-lineage errors.
|
||||
|
||||
- [ ] Add the following fixture to `test_occurrences.py`:
|
||||
|
||||
```python
|
||||
def repeated_gather_graph():
|
||||
nodes = (
|
||||
ForkNode("g", ("a", "b")),
|
||||
GatherNode("h", ("only",)),
|
||||
ChoiceNode("pick", ("again", "done")),
|
||||
GatherNode("final", ("left", "right")), EndNode("end"),
|
||||
)
|
||||
edges = (
|
||||
Edge("g", "a", "h", "only"),
|
||||
Edge("g", "b", "final", "right"),
|
||||
Edge("h", "ok", "pick"),
|
||||
Edge("pick", "again", "h", "only"),
|
||||
Edge("pick", "done", "final", "left"),
|
||||
Edge("final", "ok", "end"),
|
||||
)
|
||||
return Graph("g", {n.id: n for n in nodes}, edges)
|
||||
```
|
||||
|
||||
Assert analysis rejects with `repeated_gather:`. Independently supply runtime
|
||||
anchors `{"h": "g", "final": "g"}` to exercise runtime defense despite
|
||||
invalid authoring. Fire `h`, checkpoint, recover, choose `again`; assert
|
||||
`RepeatedGatherError` and unchanged published state. It is acceptable to
|
||||
reject on deposit before the second firing; test that explicit boundary.
|
||||
- [ ] Confirm the rejection test fails against the Task 1 analyzer, which
|
||||
incorrectly accepts. Do not make it pass by banning one-port gathers.
|
||||
- [ ] Introduce immutable symbolic marking state:
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True)
|
||||
class SMarking:
|
||||
tokens: tuple[SToken, ...]
|
||||
fired: frozenset[tuple[str, SActivation]] = frozenset()
|
||||
```
|
||||
|
||||
Preserve the canonical token ordering from `_marking`. Queue and visited
|
||||
membership must include `fired`. An inferred firing key is `(gather_id,
|
||||
anchor_activation)`. If already present, record `repeated_gather:` and do not
|
||||
explore that invalid transition. Otherwise insert it before continuation.
|
||||
- [ ] Keep firing keys only while their anchor remains referenced by at least
|
||||
one token in the resulting marking, including references through saved parent
|
||||
claims. Add a helper returning symbolic activation objects, not merely static
|
||||
fork IDs. This garbage collection lets fully resolved rounds reach the same
|
||||
abstract state again. Test retention through partial claims and cleanup only
|
||||
after complete discharge. Do not drop markers based on enclosure alone.
|
||||
- [ ] Add runtime `fired_gathers: set[tuple[str, str]]` to `RuntimeState`, using
|
||||
the existing single-owner `(gather_id, dynamic_anchor_id)` keys. Add
|
||||
`RepeatedGatherError(RuntimeError)`. Check before deposit and firing mutation;
|
||||
insert the marker within the successful candidate transition before routing
|
||||
its continuation. Persist the set in reference checkpoints. Distinct dynamic
|
||||
visits must remain distinct; this is not a serialized production format.
|
||||
- [ ] Verify the completed-round loop remains accepted within a fixed bound
|
||||
(100 markings for the supplied tiny graph). Execute two complete rounds in
|
||||
the runtime and assert distinct visit and continuation IDs. Force a tiny
|
||||
`max_markings` bound and assert rejection with `analysis_limit_exceeded:`.
|
||||
- [ ] Run all copied and new tests. Explain the revised finite-state argument,
|
||||
including token multiplicity and retained markers, without claiming general
|
||||
workflow soundness. If the argument fails, report a counterexample and stop
|
||||
that claim; do not waive it because the sample tests pass. Commit Task 2 and
|
||||
obtain separate spec and quality reviews before Task 3.
|
||||
|
||||
## Task 3: Adversarial agreement and decision handoff
|
||||
|
||||
**Files:** Create `test_agreement.py` and `VERIFICATION.md`; update experiment
|
||||
README and only evidence-backed reference gates in the live spec.
|
||||
|
||||
**Consumes:** Repaired occurrence tracking and existing graph fixtures,
|
||||
`AnalysisResult.unique_anchor_plan()`, runtime transitions, and both merge
|
||||
policies. **Produces:** Executable bounded agreement tests and a short report;
|
||||
no selected production policy and no production implementation plan.
|
||||
|
||||
- [ ] For the supplied basic, direct, partial, cross, conditional, and ordering
|
||||
graph fixtures, derive plans through the analyzer. Do not hardcode anchors
|
||||
in happy-path tests. Execute accepted acyclic examples to completion under
|
||||
all schedules within an explicit per-fixture bound, branching over ready
|
||||
token/gather actions and declared choices. Fail the test if the bound is
|
||||
exceeded rather than silently omitting paths. Keep loops in separate bounded
|
||||
tests so this is not a termination test.
|
||||
- [ ] Use semantic comparisons, not trace order or counts alone:
|
||||
|
||||
```python
|
||||
semantic_result = (
|
||||
tuple(rt.state.committed_values),
|
||||
frozenset(rt.state.committed_contribution_ids),
|
||||
rt.state.completed,
|
||||
)
|
||||
assert not rt.live_token_ids()
|
||||
assert not rt.parked_token_ids()
|
||||
assert not rt.state.gather_buckets
|
||||
```
|
||||
|
||||
For choice fixtures compare schedules for the same outcome choices. Treat
|
||||
deliberate strict-policy merge conflicts as expected results, not completion.
|
||||
Report schedule counts and bounds by fixture.
|
||||
- [ ] Exercise the two new occurrence regressions alongside simultaneous
|
||||
duplicate ports, unresolved re-entry, and sibling-same-static-fork ambiguity.
|
||||
Record conservative unsupported cases separately from invalid runtime state.
|
||||
Add a missing-arrival/deadlock probe and report what the analyzer does; do
|
||||
not relabel `accepted` as complete workflow soundness.
|
||||
- [ ] Preserve the A/B opposite-history and three-history precedence-cycle
|
||||
tests under both policies. Show the concrete final values or merge error.
|
||||
Add a pure contribution-replay example with numeric addition to demonstrate
|
||||
that deduplication is distinct from append ordering. This is evidence for
|
||||
one more reducer, not proof of generic state-path merge semantics.
|
||||
- [ ] For a ready-gather snapshot, resume and compare semantic results with
|
||||
uninterrupted execution. For a failing strict-order transition, compare the
|
||||
complete published dataclass state (not only `snapshot_dict`, which omits
|
||||
fields) before/after; include occurrence counters and fired markers.
|
||||
- [ ] Run `python -B -m unittest discover -v` inside the experimental directory.
|
||||
Run Ruff check and format-check on the exact changed experimental Python
|
||||
files. Document inherited diagnostics before fixing; keep formatting changes
|
||||
confined to editable copies. Lint the exact changed Markdown files and run
|
||||
`git diff --check`. Recheck archived hashes and forbidden-path diffs.
|
||||
- [ ] Write `VERIFICATION.md` with the exact base/HEAD, commands, pass/fail
|
||||
counts, explored-state/schedule counts, unsupported cases, and new findings.
|
||||
Include a short user-facing choice: preserve previous merge orders and fail
|
||||
on cycles, or let final-port order define the local merged view. Recommend
|
||||
one with tradeoffs, but leave the spec's policy undecided.
|
||||
- [ ] Update only closed spec gates. Leave ordering selection, owner integration,
|
||||
and production-validator selection open. Once this plan's work is complete,
|
||||
move it under `docs/historical/superpowers/plans/` and repair its relative
|
||||
links and live references per `docs/AGENTS.md`. Commit Task 3 and obtain a
|
||||
final whole-branch review. Stop without merging or pushing.
|
||||
|
||||
## Controller and review contract
|
||||
|
||||
The controller owns scope, baseline, integration, and verdict triage; it should
|
||||
not implement tasks behind its reviewers. Use a fresh implementer per task and
|
||||
separate spec/quality reviewers. Give each worker exact file ownership and the
|
||||
preceding commit. Do not run concurrent writers against the same files.
|
||||
|
||||
Reviewers must execute the two regression cases and inspect occurrence state,
|
||||
not merely repeat the implementer's test counts. Distinguish a demonstrated
|
||||
bug, a conservative limitation, and a policy preference. Reproduce findings
|
||||
before fixing; commission focused fix waves without expanding into production.
|
||||
|
||||
If subagents are unavailable, report that fact and request a workflow decision;
|
||||
do not invent review verdicts. The model-coordination guide is optional context,
|
||||
not a dependency on its ignored filesystem path or its model availability claims.
|
||||
|
||||
Final handoff: branch/worktree, commits, verification evidence, outstanding
|
||||
findings, and the one merge-order policy choice in plain language. Do not label
|
||||
fork/gather implemented or production-ready because this plan is complete.
|
||||
@@ -0,0 +1,154 @@
|
||||
# Explicit fork/gather design
|
||||
|
||||
Status: draft, not implemented or approved for production execution.
|
||||
|
||||
This is the current fork/gather contract under development. It refines
|
||||
[ADR-0006](../../adr/0006-explicit-fork-and-topology-driven-gather.md).
|
||||
The [research archive](../../historical/research/2026-09-06-fork-gather/README.md)
|
||||
preserves the reports, reconstructed reference code, and independent review.
|
||||
Research recommendations are not requirements unless adopted below.
|
||||
|
||||
## Settled direction
|
||||
|
||||
### Emission and destination ports
|
||||
|
||||
Ordinary outcomes select exactly one transition. An explicit fork emits one
|
||||
branch token for every statically declared branch. Gathers declare ordered,
|
||||
named input ports; every incoming edge must name a declared `target_port`.
|
||||
Several edges may be mutually exclusive alternatives for the same port.
|
||||
Ports may be derived from node configuration before execution, not invented
|
||||
by runtime data. On ordinary destinations, `target_port` is preserved metadata
|
||||
and is ignored for execution, not rejected.
|
||||
|
||||
All emissions use the same destination handling, including direct fork-to-gather
|
||||
edges. An arrival retains the destination port; a cursor located only at the
|
||||
gather node is insufficient. A ready gather's execution must respect the
|
||||
existing run-wide step budget before publishing its transition.
|
||||
|
||||
### Control obligations and correlation
|
||||
|
||||
A fork consumes its incoming token, saves its parent claim, and creates branch
|
||||
obligations. A partial gather consumes its arrivals and produces one token with
|
||||
their combined obligations. Complete sibling obligations normalize to their
|
||||
saved parent claim. Re-forking a partial claim saves that claim on the new visit;
|
||||
it does not duplicate ownership of that claim onto every new branch.
|
||||
|
||||
Keep these relations distinct:
|
||||
|
||||
- Referenced visits: unresolved visits on which any part of the claim depends.
|
||||
Use this relation to reject re-entry of an unresolved static fork.
|
||||
- Enclosing visits: unresolved visits encompassing every part of the claim.
|
||||
Use this relation for gather correlation, not the union of referenced visits.
|
||||
|
||||
For `{k.x, r.c}`, where `k` is inside `r`, referenced visits are `{k, r}` but
|
||||
enclosing visits are only `{r}`. The mixed token cannot be correlated under `k`.
|
||||
|
||||
Gather declarations do not name the originating fork. The proposed V1 uses a
|
||||
compiled static anchor resolved to a unique enclosing dynamic visit. Matching
|
||||
also requires the same completion owner and runtime scope; scheduling order
|
||||
must never decide matching. The supported inference grammar remains open.
|
||||
|
||||
### Occurrences and completion
|
||||
|
||||
The proposed V1 permits at most one arrival per port and one gather firing per
|
||||
`(owner, gather, anchor visit)`. Validation must account for earlier firings,
|
||||
not only the current marking of runnable and parked tokens. Runtime checks
|
||||
remain necessary. Do not turn a known illegal second firing into an incidental
|
||||
duplicate-lineage-ID exception. Fully converged rounds may loop again under
|
||||
a fresh dynamic visit; ordinary same-region loops remain valid.
|
||||
|
||||
Token identity is not node-execution occurrence identity. A repeated execution
|
||||
of a writing node creates fresh contribution identities even when it uses the
|
||||
same token. Replaying or merging an existing contribution preserves its
|
||||
identity. A recovered occurrence and a genuinely new execution must not be
|
||||
conflated. The concrete ID encoding is not selected here.
|
||||
|
||||
Completion ownership must survive token/frame replacement. Existing foreach
|
||||
and subgraph blocking responsibilities do not disappear: the parent waits for
|
||||
item/call completion, not merely the original cursor's retirement. Branches
|
||||
must converge before owner completion or foreach return. No branch may use
|
||||
`END` to silently discard outstanding siblings.
|
||||
|
||||
### State and ordering
|
||||
|
||||
A single-parent lineage tree may remain the physical state-view structure.
|
||||
Its lowest common ancestor supplies a merge base, not proof that pending
|
||||
contributions are unique. Accepting shared-history re-fork/cross/reconvergence
|
||||
requires stable original contribution identity or an equivalent demonstrated
|
||||
deduplication mechanism. Otherwise that topology must be rejected explicitly.
|
||||
|
||||
Reducers and `conflicts="error"` remain the merge foundation. Results must not
|
||||
depend on arrival or scheduling order. Declared port order determines replay
|
||||
for independent histories. It does not yet settle conflicting earlier merge
|
||||
orders at later reconvergence:
|
||||
|
||||
```text
|
||||
r.a -> write A -> fa(x,y)
|
||||
r.b -> write B -> fb(x,y)
|
||||
hx(left=fa.x, right=fb.x) -> [A, B]
|
||||
hy(left=fb.y, right=fa.y) -> [B, A]
|
||||
final(left=hx, right=hy)
|
||||
```
|
||||
|
||||
This is a legitimate authored control graph, not inherently corrupt metadata.
|
||||
Strict preservation of previous orders would fail at `final`; final-port
|
||||
precedence would choose `[A, B]`. Neither policy is adopted yet. Strict ordering
|
||||
would require global cycle detection: `[A,B]`, `[B,C]`, `[C,A]` can pass pairwise
|
||||
shared-subsequence checks while jointly imposing a cycle.
|
||||
|
||||
### Persistence boundary
|
||||
|
||||
Supported stopped checkpoints must contain a consistent semantic state,
|
||||
including any pending arrivals and occurrence information needed on resume.
|
||||
An exception must not publish a partially applied transition as the failed
|
||||
checkpoint: retain stable state plus failure information, or provide an
|
||||
equivalent rollback/commit boundary. Whole-run deepcopy is a reference-model
|
||||
technique, not a production requirement.
|
||||
|
||||
Do not add a transition journal merely because a transition has several
|
||||
in-memory steps. Arbitrary mid-transition persistence and exactly-once external
|
||||
effects are not added by this design. A process crash can discard progress since
|
||||
the last durable checkpoint; contribution IDs alone do not deduplicate an
|
||||
external API call. Existing store transaction contracts still apply.
|
||||
|
||||
## Open decisions before production planning
|
||||
|
||||
1. Does an earlier gather's serialization order constrain later merges, or is
|
||||
it a local state view? Select and document the reducer conflict semantics.
|
||||
2. Which correlation grammar can be validated soundly with acceptable cost and
|
||||
conservative rejection? The imported explicit marking explorer is evidence,
|
||||
not an approved production algorithm or complete soundness validator.
|
||||
3. How are owner completion, failure cleanup, scope boundaries, occurrence
|
||||
state, and checkpoint validation integrated with foreach and subgraphs?
|
||||
The single-owner reference does not verify these integrations.
|
||||
|
||||
## Verification gates before an executable implementation plan
|
||||
|
||||
The [reference verification plan](../plans/2026-09-07-fork-gather-reference-verification.md)
|
||||
executes the experimental subset below. A production implementation plan remains
|
||||
gated on the open decisions; completing the reference plan does not close them.
|
||||
|
||||
Do not copy the imported simulator into production or modify the archived
|
||||
evidence. Develop any revised experiment separately. Gates remain unchecked
|
||||
until new executable evidence closes them.
|
||||
|
||||
- [ ] Add a repeated-write regression: `w -> pick.again -> w`, then exit.
|
||||
Two executions append twice with distinct IDs; merging the same contribution
|
||||
through two descendants still applies it once. Include checkpoint/resume.
|
||||
- [ ] Add the repeated-gather regression: `g.a -> h.only -> pick`,
|
||||
`pick.again -> h.only`, `pick.done -> final.left`, `g.b -> final.right`.
|
||||
Reject the second firing under the same anchor during validation. Preserve
|
||||
acceptance of fully resolved rounds that revisit the same static nodes.
|
||||
- [ ] Compare inferred plans with runtime behavior for basic, direct, partial,
|
||||
cross, conditional-alternative, duplicate-port, unresolved-reentry, and
|
||||
sibling-same-static-fork graphs. Define unsupported cases explicitly.
|
||||
- [ ] Include already-fired occurrence state in the analysis argument without
|
||||
preventing completed rounds from reaching a finite abstract fixed point.
|
||||
Bound exploration and report inability to prove, not silent acceptance.
|
||||
- [ ] Resolve ordering with the A/B graph and the three-history cycle above.
|
||||
Test more than append-only examples before claiming general reducer support.
|
||||
- [ ] Test schedule permutations and consistent stopped checkpoint/resume,
|
||||
including transition failures. Distinguish this from external-effect safety.
|
||||
- [ ] Design and test owner integration, then produce a scoped production plan
|
||||
against the chosen grammar and merge policy. Keep its code/test paths and
|
||||
compatibility decisions grounded in current runtime seams.
|
||||
Reference in New Issue
Block a user