This document states and proves the concurrency invariants of
shm_gen_cache, the lock-free shared-memory generational cache, at
shared layout version 12 (LAYOUT_VERSION,
lib.rs:89).
Two artefacts back the claims made here:
A Lean 4 model of these events, in lean_model, machine-checks most of the proofs.
File and line references (store.rs:264) are to src/ and
link to it at commit 920285f3b6.
| Covered | File |
|---|---|
| Insert: pin, reservation, chunks, record store, publication, unpin | cache/store.rs |
| Rotation, rotation ownership, takeover, post-seal fence (R1F), pin scan, slot reaping, readiness certificate (R3C) | cache/rotation.rs |
| Lookup and its validation; promotion readiness (certificate load, recheck, reprobe) | cache/lookup.rs, arena/probe.rs, cache/promotion.rs |
| Index publication and key check | arena/put.rs |
| Record payload loads and stores | arena/records.rs, util.rs (emit_*) |
| Participant slots: claim, release, steal | participant.rs |
| Occupancy accounting (W6, the B0 target check) | occupancy.rs |
| Word encodings | arena/ctl.rs, arena/index.rs, arena/records.rs |
Initialisation, registration, from_raw |
cache/mod.rs |
Not covered:
ffi.rs), except as the place that enforces A2.linux_pid.rs, windows_pid.rs). Their
contract is assumed: A3 for safety, A9 for robustness.wait.rs and futex wake-up delivery. Its loads only decide whether
to sleep; they are liveness, outside the safety model.test_access::stage_interrupted_reap
(test_access.rs:188-210): test-only, not modelled.GetPid test hooks (reservation_epoch_hook,
reservation_retry_hook, after_epoch_publication,
after_arena_reset,
after_record_publication, promotion_hook;
pid.rs:30-58). They are called only under feature
verify, default to empty functions and make no shared-memory access
of their own; promotion_hook tells a GenMC program which way a lookup
came to promote (section 15).PromotionSampler (cache/promotion.rs:13-38), which decides whether a
lookup rechecks readiness. Every proof quantifies over both of its answers;
it affects hit rate and liveness only.insert(). (HT2, HT3.)Ok after publishing in epoch .
A lookup of whose final pass's V1 happens after
that return may miss. Any hit originates in or a same-key insert
later than or concurrent with , never one that
precedes (its return happens before 's call),
regardless of promotions or rotations (FR3).
The supporting property is that a promotion copies the last publication
of its key in the source generation (FR2).
If, besides, the lookup's final pass reads epoch
or and reads no store of incarnation or later, it does not
miss: it returns a hit, InsufficientCapacity or Corrupt, never Ok(None)
(FR4). (RD1, RD2, FR1-FR4.)insert() that
returns an error other than Corrupt makes no I2: no index entry is a
publication of any of its records. So no lookup ever returns a record
written by a store attempt that descends from it, directly or as a promoted copy,
in any generation. Corrupt returns are outside this guarantee.
Limits: a failed insert can still have side effects: the rotations it
performs or triggers, which evict the oldest generation, so an older
value of the key may stop being returned; slot reaping; and record-area
space consumed by bytes reserved, or written but never indexed. So a
lookup after a failed insert is not guaranteed to return the previous
value, only not to return the failed insert's. (FR5.)Non-goals:
| Goal | Rests on |
|---|---|
| G1 | HT2, HT3 (through HT1, IX1, SP5, SP4, QU2, EP1) |
| G2 | BD1, BD2, SP4 |
| G3 | QU1, QU2, OW2, EP1-EP4 (through PN1-PN3, OW1, SL1) |
| G4 | RD1, RD2, FR1-FR4 (through RB3, RB4, HT1, HT2, SP5, QU2, PN1-PN3, EP1-EP3, OW2, A6, A7, A8) |
| G5 | FR5 (through HT2, F10, F15) |
Section 6 ends with the converse: which invariants and assumptions are necessary, and what goes wrong without each.
The proofs use the C++20 memory model that Rust atomics adopt ([intro.races], [atomics.order], [atomics.fences]). Read in RC11 terms, each argument is the acyclicity of or of . GenMC checks the code under RC11.
Threads are written θ, fences , a particular O3 event , and
the epoch in which a returning insert published (G4, FR3, FR4).
F1-F15 always denote code facts, never fences. is the single total
order of SeqCst operations; to avoid a clash, the scan load is written
and the liveness verdict . Like Z for the seal of ,
Cert denotes any readiness certificate for (section 4.2).
| Symbol | Meaning |
|---|---|
| Sequenced before: program order within one thread. | |
| Load (or RMW) takes its value from write . | |
| Modification order: the total order of the writes to word . says precedes . | |
| Release sequence of a release write : and the maximal run of read-modify-writes that follow it in , each reading its predecessor (C++20: no same-thread plain store continues it). | |
| Synchronises with: rules REL-ACQ, FENCE-ACQ and FENCE-FENCE below. | |
| Additional synchronisation outside the memory model: thread creation and join, and the liveness backend's edge of A3. | |
| Happens before: the transitive closure of , and . | |
| Coherence-ordered before ([atomics.order]/3; RC11 ): if ; if ; if reads a write -before and (excluding an RMW's self-edge); and transitively. | |
The SeqCst-fence part of RC11's : . It is acyclic, and extends it. |
The rules cited by name:
| Rule | Statement |
|---|---|
| COH-WW | If writes on one word, then . |
| COH-WR | If write load on one word, reads or a write -after it. |
| COH-RW | If load write on one word, reads a write -before . |
| COH-RR | If loads on one word, reads the write read or a later one. |
| RMW | A read-modify-write immediately follows, in , the write it read. |
| REL-ACQ | A release write an acquire load that reads from . |
| FENCE-ACQ | A release write an acquire fence when a load reads from . |
| FENCE-FENCE | A release fence an acquire fence when a write -after is read (through its release sequence) by a load . A SeqCst fence is both a release and an acquire fence. |
| SC-4.4 | If SeqCst fences , and , then precedes in ([atomics.order]/4.4; in RC11 a edge). |
The remaining rule of [atomics.fences] (a release fence an acquire load that reads a write -after it) is not used by any proof here.
A0 (memory model). Executions are RC11-consistent, with the
transitive closure of , and . is well-founded (has
no infinite descending chain), and every read reads exactly one
write. SeqCst fences are ordered by an acyclic . The proofs use
only the C++20 release sequence, continued by RMWs only. That
definition is contained in C++11's, so the proofs hold under both. With
the A3 edge in , the model is RC11 plus , not pure RC11:
without it, the steal cases of SL1 and OW1 would have no edge.
A set of actions is serial when totally orders it.
Serial-reads lemma. On a serial set of writes to one word, agrees with (COH-WW). A load that is -ordered against every write to the word reads exactly the latest write that happens before it (COH-WR, COH-RW).
Chains such as O4 P3 P1 read left to right and compose into .
The mapping holds a header, one cache-line slot per participant, and
three arenas. Each arena has a control word , a readiness word (the
last epoch certified as a promotion source), an occupancy counter
(exact mode only), an index of stamp:32, ref:32 entries and a record
area. Incarnation lives in arena ; while the global epoch
is , lookups probe incarnations and , inserts write into
, and the third arena waits to be reused as .
Registration. A thread claims a FREE slot (P1, P2) with a fresh registration id, and releases it at the end (P3). When every slot is taken, it may steal the slot of a dead participant (P0-P6). The steal asks the liveness backend for a verdict about the claim it replaces.
Insert. A pass of reserve() loads the epoch (W1), pins it (W2),
fences (W3) and prechecks the arena (B0). It then cuts the record from
its private chunk (H1), or reserves with a fetch_add on the bump (B1),
possibly claiming a new chunk (H2). The insert then stores the payload
and the header (W4, W5), publishes the index entry (I0, I2), updates the
occupancy (W6) and unpins (W7). A pass that finds the arena sealed or
full unpins and rotates instead.
Rotation. The rotator takes ownership of the rotation-owner word
(O1, O2), rechecks the epoch (R0) and seals the current arena (R1).
Right after the seal loop it runs a SeqCst fence (R1F). It then scans
every pin (R3). An observation at least two epochs old enters the wait
and possible slot-reaping path (R3A, reap_pinner). An observation of
the epoch it has just sealed disables readiness and takes neither path.
A reap triggered by an older observation may clear a pin that changed meanwhile
(RD1). If the scan observed no pin of
, it certifies as a promotion source (R3C). It
then resets the arena to be reused (R4), publishes the next epoch (R5)
and releases ownership (O4).
Takeover. A rotator that finds a dead owner examines the owner's
slot (reap_owner()) and, unless the owner is live, takes ownership
over with a CAS (O3).
Lookup. A lookup loads the epoch (V1) and probes , then .
It loads index entries (I1), fences (I1F), loads the record (D4, D5),
fences again (D6) and reloads the epoch (V2), returning Retry when the
probed incarnation is two or more epochs old. Before probing it
loads that arena's readiness word (C0). Probes do not pin and write
nothing, except a recheck's certificate (C4, a fetch_max made
unpinned); a promotion pins and writes through store().
Promotion. Copying a hit from into is hazardous while a writer
of is still in flight: an insert of pinned to may publish
after a lookup has already read the older there and return, and the
promotion would then republish in , shadowing the completed insert
for every later lookup (section 16, item 2). A certificate for
says that every store attempt for has finished, and makes their writes
visible to whoever reads the certificate (RD1, RD2). A probe ordered after
it therefore reads the key's last publication in (FR1, FR2). The order
matters: a certificate learned after the probe's loads does not order
them, hence C0 precedes the first I1, and a certificate obtained later
leads to a fresh probe. The mechanism: a hit in after a miss in is stored again,
into only (only_epoch), through the same insert path, but only if
is certified: either C0 read the certificate for before the
probe read anything, or a sampled recheck (C1-C4) certifies it now
and a fresh probe of (the reprobe) supplies the copy that is promoted.
Otherwise the hit is returned unpromoted. The recheck neither waits nor
reaps. The sampler rechecks with probability min(1, 8 / bucket_count),
drawn from a per-registration SplitMix64 stream. A promotion is
also dropped if the epoch moved on or the arena has no room, and when no
index slot accepts it, without rotating. Once published, it triggers the
same maintenance as an insert: a rotation at the occupancy target.
The C API's lookup documentation (ffi.rs:496-504, and the
generated shm_gen_cache.h) states the same contract, including that the
reprobe's result, possibly a miss or InsufficientCapacity, replaces the hit.
Event labels are those of section 3.1; section 4.2 tabulates them.
reap_pinner() briefly holds a
second registration: its events from that P4 to the P3 that releases
it (T2, P5, P6, H0) are on behalf of the second registration, not of
the rotator's own ; being made by the rotator's thread between
's claim and P3, they are also -ordered with 's actions. The
register of is its P2 or P5. The ender of is the state
write that ends it: its own P3, or a P4 stealing it.reserve()'s loop, from W1 to the
return of a reservation or to W7. A store attempt is a pass
whose B0 saw , extended through its record
stores, I2 and W6 to its W7 (store.rs:175, or store.rs:352 for an
attempt that stops). Its actions from W2 to W7 are the events of the
pass from W2 on, and of the store that follows it, up to that W7 (or to
the end, if the thread dies). An attempt may stop at the target check
or fail at B1; if it succeeds, it writes and publishes one record.ROTATION_WAITING flag in a pin it waits on. The head of a pin
write is the last non-announce write at or before in
(or ): lies in its release sequence (PN1).insert(), from its call
to its return; its store attempts lie in between (F15). Insert
precedes insert when 's return happens before 's call. A
store attempt descends from insert if it is an attempt of ,
or a promotion attempt whose promoted hit's I1 read an entry published by
an attempt that descends from . Each attempt descends from exactly one
insert, its root. By induction on the attempt's epoch: an insert
attempt belongs to one insert, and a promotion attempt to none (F15); a
promotion attempt copies one hit (F10), whose I1 reads one entry, whose
publication belongs to one attempt (F2) of the epoch before.fetch_max with argument
on the readiness word of arena : the R3C of a tenure with
R0 , or the C4 of a recheck whose C1 read (Cert).| Word | Writes (event, ordering) | Loads (event, ordering) |
|---|---|---|
global_epoch |
init (0, relaxed; then 2, release); R5 (release) | W1 (acquire); V1, V2, R0, O0 (relaxed); C1 (acquire) |
rotation_owner |
init (relaxed); O2 CAS (AcqRel/Relaxed); O3 CAS (Release/Relaxed); O4 store (release) |
O1 (acquire); wait loads (relaxed, liveness) |
registration_counter |
init (relaxed); id allocation fetch_add (relaxed) |
the same fetch_add |
magic |
init CAS (SeqCst), init store (release) |
the init CAS reads the header word (SeqCst; cache/mod.rs:191); A1 abstracts completed initialization for concurrency proofs |
| header parameters | init (plain) | never read (A7) |
slot.state |
init (relaxed); P1 CAS (acquire); P2, P3, P5 CAS (release); P4 CAS (release; failure acquire) | P0 (acquire); is_free precheck, P3's own load, own-word loads (relaxed) |
slot.thread_disambiguation |
init; T1, T2 (relaxed) | T0 (relaxed) |
slot.pinned_epoch |
init (relaxed); W2 store (release); W7 swap (release); P6 store (release); R3A CAS (relaxed) | R3 (acquire; also the readiness test), C3 (acquire); own-word loads, wait loads (relaxed) |
slot.chunk_epoch, chunk_cursor, chunk_end |
init, H0, H2 (relaxed); H1 (cursor only, relaxed) | H1 (relaxed) |
arena.ctl (stamp:32, sealed:1, bump:31) |
init (relaxed); B1 fetch_add, R1 CAS, R4 store (relaxed) |
B0, R1's loads (relaxed) |
arena.promotion_ready_epoch (full 64-bit epoch, same line as ctl) |
init (0, relaxed); R3C, C4 fetch_max (release) |
C0 (acquire); test code only otherwise (the unit tests in cache/promotion.rs, and test_access::promotion_ready_epoch, feature verify) |
arena.exact_occupancy (exact mode) |
init, R4 (relaxed); W6 fetch_add (relaxed) |
B0's target check, W6 (relaxed) |
| index slot | init zero fill; I2 CAS or store (release) | I1 (relaxed) then I1F; I0 (acquire) |
| record words | init zero fill; W4, W5 (relaxed) | D4, D5 (relaxed) |
Labels are those of the source comments, except C0, the readiness
load in probe(), which the source does not label. The readiness labels
R1F and R3C are in rotation.rs (rotate_owned()), and C1-C4 in
promotion.rs (recheck_promotion_ready()). Section 4.3 gives the
conventions and the renames.
Insert (store(), reserve(), publish(), table_put()):
| Label | Access | Ordering | Code |
|---|---|---|---|
| W1 | Load global_epoch, value , at the start of every pass |
acquire | store.rs:264 |
| W2 | Pin: pinned_epoch := e |
release | store.rs:272 |
| W3 | Fence of the pass, in every pass, a rotator's included (the writer role): store-buffering partner of R1F (reuse and readiness scans) and of C2 (PN3), and the release fence D6 synchronises with (HT1). The scan side of PN3 is R1F or C2, never a W3 | SeqCst |
store.rs:284 (comment 274-283) |
| B0 | Precheck: load , plus the occupancy target check (relaxed exact_occupancy load, or the estimator). The pass goes on only on and target not reached |
relaxed | store.rs:288 (checks 296-301); occupancy.rs:63 |
| H1 | Chunk cut: load chunk_epoch, chunk_cursor, chunk_end; store chunk_cursor := cursor + n after the u32 room check |
relaxed | store.rs:303-308 |
| B1 | Reservation: fetch_add on ctl.bump (bits 33-63; a carry out of bit 63 is discarded). Succeeds iff the old value is |
relaxed | store.rs:327-331 (check 332) |
| H2 | Chunk claim after a successful B1 that claimed a chunk: chunk_epoch := e, chunk_cursor := bump + n, chunk_end := min(bump + claim, capacity) |
relaxed | store.rs:333-341 |
| W4 | Payload stores: lengths word, padded key words, padded value words | relaxed | store.rs:152, 154, 155 -> util.rs:129, 138, 161 |
| W5 | Record header (e as u32, len), after the payload |
relaxed | store.rs:158 |
| I0 | table_put()'s load of an index slot, on which the I2 CAS or store is predicated (a failed CAS re-reads the same slot) |
acquire | put.rs:60 |
| I2 | Publication: CAS over an entry without the attempt's stamp, or (never for a promotion) store over an entry with it for the same key | release | put.rs:86-88 (CAS), put.rs:78 (store) |
| W6 | End of publication: occupancy accounting entry_published() after I2 (exact counter fetch_add for a new entry, or a relaxed load of that same counter for an existing entry; the estimator touches no shared word). Both exact-mode accesses are W6 events at the fixed occupancy word. The verify hook after_record_publication follows, before W7. No safety proof depends on its value; QU1 orders it before the reusing R4 |
relaxed | store.rs:230-240 -> occupancy.rs:67-85; hook store.rs:173-174 |
| W7 | Unpin: pinned_epoch.swap(UNPINNED). Ends the pass, after publication or when the pass is abandoned before rotate(). The swap serves only the ROTATION_WAITING wake |
release | participant.rs:164-168; called at store.rs:175, store.rs:352 |
Registration (register_participant(), claim(),
release_claim()):
| Label | Access | Ordering | Code |
|---|---|---|---|
| id allocation | next_registration_id(): fetch_add on registration_counter, skipping 0 (u32, wraps); the source of the fresh ids A4 relies on |
relaxed | participant.rs:21-28; callers cache/mod.rs:268, rotation.rs:210, rotation.rs:239 |
is_free precheck |
Load a slot's state before trying P1; no proof needs it | relaxed | participant.rs:353 via cache/mod.rs:271 |
| P1 | Claim: CAS FREE -> INITIALIZING(pid, id) |
acquire | participant.rs:201-203 |
| T1 | Store the claimant's start time | relaxed | participant.rs:211 |
| H0 | Chunk reset reset_chunk(): store 0 to the three chunk words; in claim() between P1 and P2, in steal() after P6, and at slot initialisation. Only the chunk_epoch store matters |
relaxed | participant.rs:189-193; calls at 182, 213, 346 |
| P2 | Register: CAS INITIALIZING -> REGISTERED |
release | participant.rs:216-218 |
| P3 | Release the slot: load the own state, then CAS REGISTERED -> FREE |
relaxed load; release CAS | participant.rs:231 (load), 239-241 (CAS) |
Steal (steal(); used by reap_dead_participant(),
reap_pinner() and reap_owner()):
| Label | Access | Ordering | Code |
|---|---|---|---|
| P0 | Load the slot state | acquire | participant.rs:280 |
| T0 | Load the start time paired with the claim P0 read | relaxed | participant.rs:295 |
Backend verdict is_live(pid) (INITIALIZING) or is_live_since(pid, st) (REGISTERED) returning Ok(false). Not a memory event: the target of A3's edge |
- | participant.rs:297-302 |
|
| P4 | Steal: CAS current -> INITIALIZING(me); a failure loads the newer state |
release; failure acquire | participant.rs:314-316 |
| T2 | Store the stealer's start time | relaxed | participant.rs:322 |
| P5 | Register: CAS INITIALIZING -> REGISTERED |
release | participant.rs:324-326 |
| P6 | Clear the dead pin: pinned_epoch := UNPINNED, standing in for the dead W7; then wake and H0 |
release | participant.rs:338 |
Ownership (acquire_rotation(), release_rotation()):
| Label | Access | Ordering | Code |
|---|---|---|---|
| O0 | Epoch precheck before each ownership attempt; no proof needs it | relaxed | rotation.rs:283, 303 |
| O1 | Load the owner word (WaitForOwner only) |
acquire | rotation.rs:307 |
| O2 | Take ownership from NONE: CAS |
AcqRel; failure relaxed |
rotation.rs:289-294 (SkipBusyOwner, no O1), 310-312 |
| O3 | Take over a dead owner: CAS owner -> me after reap_owner() returned true |
Release; failure relaxed |
rotation.rs:336-338 |
| O4 | Release ownership: store NONE |
release | rotation.rs:260-262 |
Rotation (rotate(), rotate_owned(), reap_pinner()):
| Label | Access | Ordering | Code |
|---|---|---|---|
| R0 | Recheck the epoch under ownership; the tenure goes on only if it reads | relaxed | rotation.rs:89 |
| R1 | Seal loop on the current arena's : initial load, then CAS setting SEALED and keeping stamp and bump. Exits on its own seal or on a sealed word of stamp |
relaxed | rotation.rs:94 (load), 96-123 (CAS 115-120) |
| Z | Notation, not a label: the successful R1 on the word of stamp , by whichever tenure (unique, EP2) | - | rotation.rs:114-119 |
| R1F | Post-seal fence. It runs once the seal loop has exited, on the tenure's own successful R1 or on a load (the initial load or a failed CAS) that returned stamp , sealed, and before the first R3 load; no return lies in between. Store-buffering partner of every writer's W3, for the reuse scan and for the readiness scan of (PN3) | SeqCst |
rotation.rs:130 (comment 125-129), in rotate_owned() |
| R3 | Scan: load every slot's pin; passes on UNPINNED or (flag masked); reaps the slot after the wait budget. Every load, reloads in the wait and reap loop included, also tests whether the pinned epoch (flag masked) is , and then clears the local readiness flag; an observation of takes neither the wait nor the reap edge. A reap triggered by an older observation may clear a pin that changed meanwhile (RD1) |
acquire | rotation.rs:132-180 (flag 131, load 141, readiness test 143-147, pass test 148-150) |
| R3A | Announce, inside R3's wait loop: CAS pin -> pin | ROTATION_WAITING before blocking; an RMW keeping the pinned epoch |
relaxed | rotation.rs:157-161 |
reap_pinner() |
Procedure, not a label: steal() then P3 of the stolen slot; the rotator briefly holds a second registration |
- | rotation.rs:208-215 |
| R3C | Rotation certificate: promotion_ready_epoch.fetch_max(e, Release) on arena . It runs only if no R3 load after R1F returned pinned epoch ; after the scan, before R4 and R5, with no error path in between |
release RMW | rotation.rs:181-185 -> promotion.rs:74-78 (mark_promotion_ready()) |
| R4 | Reset arena : exact_occupancy := 0 (exact mode), |
relaxed | rotation.rs:189-194 |
| R5 | Publish global_epoch := e + 1 |
release | rotation.rs:199 |
Takeover: reap_owner() (rotation.rs:234-247) is a procedure, not
a label. It runs steal() restricted to the owner's registration id (P0,
T0, , P4, T2, P5, P6, H0), then P3 of a stolen slot; O3 follows
unless the outcome is Live.
Lookup (lookup(), probe(), reprobe_for_promotion(),
recheck_promotion_ready()):
| Label | Access | Ordering | Code |
|---|---|---|---|
| V1 | Load global_epoch, value ; probe then |
relaxed | lookup.rs:77 |
| C0 | Readiness load, in probe::<true> only, which only the previous-generation probe of lookup() uses: load the probed arena's promotion_ready_epoch and compare it exactly, as a full 64-bit value, with the probed epoch . Sequenced before the probe's home I1, so before every load of the probe. Its result is the only thing that sets the hit's promotion_ready flag |
acquire | probe.rs:74; called from lookup.rs:102-113 |
| I1 | Index load: home slot, group scan, sequential scan | relaxed | probe.rs:99, 269, 220 |
| I1F | Fence once I1 selected a candidate | acquire | probe.rs:151 |
| D4 | Record header load, in probe() and in entry_has_key() |
relaxed | probe.rs:159; put.rs:125 |
| D5 | Payload loads: lengths word, key words, and in probe() the value copy |
relaxed | probe.rs:171 (lengths), 181 -> records.rs:99, 108 (key), probe.rs:191 (value); put.rs:135 (lengths), 146 -> records.rs (key) |
| D6 | Fence after the copy; probe() only |
acquire | probe.rs:199 |
| V2 | Reload the epoch; Retry when the u32 age of the probed stamp exceeds 1 |
relaxed | probe.rs:200-204 |
| C1 | Recheck epoch load. Reached only after a validated hit in whose C0 did not certify, and only when PromotionSampler::should_recheck returns true. The recheck goes on only if it reads , V1's value |
acquire | promotion.rs:56, via lookup.rs:182-209 |
| C2 | Recheck fence, after C1 and before C3 | SeqCst |
promotion.rs:59 |
| C3 | Recheck pin loads: one load of every slot's pin (0..participant_capacity). The recheck fails, without waiting or reaping, if any returns pinned epoch (flag masked) |
acquire | promotion.rs:60-65 |
| C4 | Recheck certificate: promotion_ready_epoch.fetch_max(e - 1, Release) on arena , after every C3 load passed; sequenced before the reprobe |
release RMW | promotion.rs:66 -> promotion.rs:74-78 |
reprobe_for_promotion() |
Procedure, not a label (the reprobe): the sampler, C1-C4, then a probe::<false> of , which repeats I1, I1F, D4-D6 and V2 without C0. Its result replaces the hit, the output words and the value length: Ok makes the lookup promote; Retry restarts the pass; Miss returns a miss; InsufficientCapacity goes to the caller. Out of line (#[cold], #[inline(never)]) but on the lookup's thread, so sequenced after V1 and the first probes |
- | lookup.rs:182-209, and the match in lookup(), 127-147 |
| Cert | Notation, not a label: any certificate for , either the R3C of a tenure with R0 or the C4 of a recheck whose C1 read . Each is a release fetch_max with argument on the readiness word of arena |
- | mark_promotion_ready() (promotion.rs:74-78), called at rotation.rs:184 and promotion.rs:66 |
Initialisation (, notation): Cache::initialize and
initialize_storage (cache/mod.rs:172-258). Magic CAS, plain header
parameters, registration_counter := 1, global_epoch := 0,
rotation_owner := 0, slots (FREE, UNPINNED, start time 0, H0;
participant.rs:178-183), words (arena 2: ;
the others ), promotion_ready_epoch := 0 for each
arena (relaxed; cache/mod.rs:246; never reset afterwards, as R4
does not write it), exact_occupancy := 0, zero fill of
index and records, global_epoch := 2 (release, cache/mod.rs:256),
magic (release, cache/mod.rs:257). By A1 it is abstracted as one write
per word.
Accesses omitted from the concurrency event graph. The production-assertion loads of the caller's own
pin (participant.rs:212, 238; rotation.rs:76;
lock.rs:150-155) and the unconditional own-pin load in UnpinOnExit::drop
(lock.rs:175), and registration_id() building the owner word
(participant.rs:360, via rotation.rs:280) read words that only the
caller writes: they are sequential. The WordHalf::read loads of BoundedWait
(wait.rs:81) only decide whether to sleep. Their addresses, and those of the own-slot
loads above (the claim assertion uses the registration scan index), are nevertheless covered by BD2: the wait watches a range-loop
participant pin or the header rotation-owner word, and own-slot loads use
the slot retained from registration, a scan slot, or the slot decoded from
a nonempty owner word when a reaper releases its temporary registration.
The omitted is_free precheck also uses the registration scan slot.
The initialization magic CAS reads the header word at byte 0 before the
A1 abstraction. On AArch64, arm_wfet's
ldxr reads the low half of a scan pin or the high half of the
header rotation-owner word at byte 60
(wait.rs:1045-1057).
O0 and the is_free precheck
only short-cut: R0 and P1 are the accesses that decide.
Besides C0, only test code reads the readiness word: the unit tests in
cache/promotion.rs and test_access::promotion_ready_epoch (feature
verify).
table_put() (insert), I1
and I1F to probe() (lookup), so its numbers are not one program
order.reap_owner()
and reap_pinner(); so are Cert and reprobe_for_promotion().Assumptions are environmental or contractual; code facts are properties of the source, checked by reading (and, where noted, by GenMC).
| Id | Assumption | Source | Used by |
|---|---|---|---|
| A0 | Memory model: RC11-consistent executions with ; well-founded; one write per read; acyclic. The C++20 release sequence, contained in C++11's. | Section 2 | all |
| A1 | Initialisation precedes use: every action of Cache::initialize happens before every other action on the mapping, the init store of 0 to the epoch included, so is one write per word. |
Safety contract of initialize, from_raw (cache/mod.rs:168-171, 370-374) |
SL1, PN1, OW2, EP1, EP2 |
| A2 | Registrations are sequential: all actions on behalf of a registration are made by the claiming thread and totally ordered by . Never concurrent, never from a signal handler that interrupted the thread, never in a forked child; none follows its P3. In particular an owner acts under ownership only before its P3, and a rotator's actions under a second registration taken in reap_pinner() are on behalf of that registration (and -ordered with the rotator's own; section 3.2). |
ParticipantLock is !Send; ffi.rs:139-141; fork guard ffi.rs:150-169; crate README "Limits" |
SL1, OW1, PN1, PN2, SP3 |
| A3 | Dead-verdict soundness: if is_live(pid) or is_live_since(pid, st) returns Ok(false) at , each matching thread incarnation already observed before the verdict has stopped and all its actions (an edge). Matching means the same pid and, when , the same start time. An incarnation is observed through a chain of edges from one of its actions to a read -before the verdict. Later incarnations reusing the pid are outside that verdict's scope. Thread ids are non-zero and below 2^31. |
GetPid contract (pid.rs:8-13); Linux: /proc/<tid>/stat and membarrier(GLOBAL); ids: linux_pid.rs:28, windows_pid.rs:59 |
SL1, OW1 |
| A4 | Per-operation non-recurrence: every claim's state word is unique in its slot (registration ids from the u32 counter are assumed not to wrap onto a word still held; a taker never reuses an id for a slot), so an owner word names one registration. Within one operation a 32-bit stamp stands for one full epoch. The epoch stays below 2^63 (bit 63 of a pin is ROTATION_WAITING). |
Fresh ids | SL1, OW1, EP2, HT2 |
| A5 | Configuration: the parameters passed Config::resolve: reservation_adds_fit_bump (record area plus one largest claim per slot below 2^31), a bucket count that is a power-of-two multiple of 8, a record area that is a multiple of 8 and below 2^31, and a largest record max_record_size = 16 + align8(max_key_size) + align8(max_value_size) and a chunk both at most the record area (so the area is at least 16 bytes). |
config.rs:81, 116, 123-134, 241-256 |
SP2, SP4, BD1, BD2 |
| A6 | Lifetime non-recurrence: an index load ordered after an epoch load returning n may observe an entry published at epoch e only if n < e + 2^32; the zero fill counts as epoch 0. A lookup's final validation must read an epoch less than e + 2^32, where e is the epoch it probes. Both bounds are 2^32 epoch advances, independent of which arena holds the entry. |
Crate README "Limits"; probe.rs:138-143 |
HT1, HT2, RB3, RB4, FR2-FR4 (through HT2, and the exact stamps of FR3 step 6) |
| A7 | Shared parameters: every handle on a mapping uses the initialiser's parameters (participant_capacity, bucket count, record area size). The header copies are never read, so this holds only by the from_raw contract. |
cache/mod.rs:370-376; crate README "Limits" |
QU1 (R3 covers every slot), RD1 (R3's readiness test and C3 cover every slot), BD1, BD2 |
| A8 | Hash contract: equal keys are passed equal hashes. | API contract | RB4, hence FR2-FR4 (G4); FR3 step 6 (one key, one tag) |
| A9 | Verdict completeness: a running thread is never reported dead under its own identity. | GetPid contract |
RB1, RB2 |
No assumption covers fairness or eventual promotion.
PromotionSampler::should_recheck may return anything, every proof
quantifies over both answers, and the sampler is outside the model
(section 1). RD1-FR4 need no new axiom either: only the
rules of section 2 under A0.
Record headers need no lifetime clause: no safety proof uses their stamps (D4's stamp check is redundant with the entry stamp, A6 and V2).
| Id | Code fact | Code | Checked |
|---|---|---|---|
| F1 | Pin discipline, in two parts. (quiet) The registrant makes no write to its pin strictly between a W2 and that pass's W7: from W2, reserve() reaches either a reservation exit (after H1 or a successful B1), which returns to store() still pinned, or the W7 that ends an abandoned attempt; store() then runs W4, W5, publish() (I2, W6) and its W7, none of which but the last writes the pin. (exit) If the registrant writes its pin again after a W2, that pass has run its W7. Per exit: a reservation exit returns pinned to store(), whose every return and repetition (Ok, the rotation after Full, the FullRejected promotion drop, the FullRejected retry) follows its W7; an abandoned attempt runs W7 before the promotion Ok(None) exit, the rotation, the rotation's error return and the next pass's W2; the promotion Ok(None) exit for a moved epoch returns before W2, so its pass has none; and store() turns a None from reserve() into Ok(()), unpinned. The registrant writes its pin only at W2 and W7 (lookup() only through store()); rotate() and release_claim() are entered unpinned. |
(quiet) store.rs:272 (W2), 284-346 (reservation exits 308, 342), 352 (W7); 147-175. (exit) store.rs:142-146, 175-203, 269-271 (before W2), 352-359 |
reading; production_asserts at rotation.rs:76, participant.rs:238, lock.rs:150-155 |
| F2 | Attempt shape: W1 (acquire) W2 W3 (SeqCst) B0; B1, H1 and H2 only after a B0 that read the epoch's stamp; each event belongs to one attempt and each pin store to one slot. rotate(e) is called unpinned. "Each event belongs to one attempt" is also a premise of FR3. |
store.rs:264-343, 185, 198, 356; test_access.rs:216-224 |
reading; GenMC |
| F3 | Record stores: an attempt stores each word of its reservation once, inside the bytes it reserved (lengths and key/value W4, then header W5), and only then calls table_put (I2) with its own offset, epoch and key. Sizes: insert() rejects a key or value above max_key_size or max_value_size (store.rs:90-95); a promotion's key is bounded at lookup.rs:56 and its value at 154. So the u32 arithmetic of payload_size() (records.rs:75-78) does not wrap, the record takes exactly 8 + align8(8 + align8(key_len) + value_len) bytes, and every record is at most max_record_size, hence at most the record area (A5). |
store.rs:90-95, 147-169; lookup.rs:56, 154; records.rs:75-78; util.rs:120-166 |
reading |
| F4 | Chunk words: written only by , H0, H1 and H2. Every registration that pins has run H0 first: in claim() between P1 and P2, in steal() after P6 (which covers a registrant that keeps a stolen slot). |
participant.rs:182, 213, 346; store.rs:303-308, 333-341 |
reading; GenMC slot_reaping_resets_reservation_chunk |
| F5 | Rotation shape: under ownership, R0 once; R1 an RMW keeping stamp and bump, sealing the arena of R0's epoch. The seal loop exits only on the tenure's own successful R1 or on a load (the initial load or a failed CAS) returning stamp R0, sealed, so a tenure reaches R1F, hence R4, only after its own seal or a sealed word of stamp R0; R1F (SeqCst) immediately follows the loop, with no return in between. R3 loads each slot at least once after R1F (acquire, passing only on UNPINNED or R0 ; every load also tests whether the pinned epoch, flag masked, is R0, and clears the readiness flag on a match) R3C (release fetch_max with argument R0 on the arena of R0, only if no R3 load showed R0) R4 R5, one each per tenure; no R3C on an error path; R5 writes R0 + 1, release; R3A an RMW keeping the pinned epoch; written only by , B1, R1, R4; the epoch only by and R5. |
rotation.rs:89-203 (seal loop 96-123, R1F 130, scan 132-180, R3C 181-185) |
reading |
| F6 | Takeover control flow: before an O3, reap_owner() returned true for the slot decoded from the owner value loaded at O1, whose registration id it passes as the expected id; O3 expects that same owner value. The slot's acquire P0 is sequenced after O1 and before O3. The true return paths are Free (P0 read zero), Other (the read state's high u32 differs from the expected id), Stolen (the id matched and, after a dead verdict, P4 succeeded with the full P0 word as its expected value), or Lost (the id matched and that strong P4 failed, reading a different word with acquire ordering). Both P4 outcomes precede O3; Stolen also completes the steal and releases its claim before returning true. Live and errors do not proceed to this O3. |
rotation.rs:232-245, 305, 328-336; participant.rs:275-348 |
reading; GenMC |
| F7 | Steal shape: P0 (acquire) T0 P4 (release; failure acquire) T2 P5 (release) P6 (release) H0. The verdict asks is_live for INITIALIZING and is_live_since(st from T0) for REGISTERED. |
participant.rs:275-348 |
reading |
| F8 | Index writes: only the zero fill and I2 write index slots; a publication lies in its attempt's arena, encodes its own reservation's offset (a multiple of 8) and its epoch's 32-bit stamp; a CAS replaces only an entry without its stamp. | put.rs:53, 61, 78, 86-88; index.rs:88-92 |
reading |
| F9 | table_put probing: it advances past a slot only after I0 found an entry with its stamp for another key there (a failed CAS re-reads the same slot); the probe visits each slot once; a store replaces only an entry with its stamp for which entry_has_key held. A promotion never stores: its I2 is always a CAS (the store, put.rs:78, is guarded by if !promotion; the empty-slot CAS, put.rs:86-88, is shared by inserts and promotions). Used by RB4(d), hence by G4. |
put.rs:57-99 |
reading |
| F10 | Values: an insert stores the caller's key and value. A promotion pins only V1's epoch (only_epoch) and stores the caller's key and the copied padded value words of one of two hits, both in epoch V1 - 1, after a miss of the same pass in V1's epoch: (a) a hit of probe::<true> whose C0 returned V1 - 1, with C0 sequenced before its first I1; or (b) the hit of a reprobe sequenced after a recheck that executed C4. A hit in the current arena is never promoted (probe::<false> never sets the flag). |
store.rs:96-104, 269-271; lookup.rs:76-171, 182-209, probe.rs:74 |
reading; GenMC reaches both promotion paths (PROMOTED_VIA_CERTIFICATE, PROMOTED_VIA_RECHECK in promotion_keeps_newer_stale_epoch_insert) |
| F11 | Reader checks: probe() and entry_has_key() load the header first, and load the lengths, key and value words only at addresses derived from the entry's offset and from a header and lengths word that passed the length checks (u32 arithmetic exact under A5). The probed entry carries the probe's stamp, and the probe's record loads lie between I1F and D6. |
probe.rs:144-212; put.rs:111-147; records.rs:92-114 |
reading |
| F12 | Arena words: index slots and record words are written only by the zero fill and by store attempts (W4, W5, I2), each after its W1 and W3 and before its W7; arena serves epoch , so two attempts writing one word have equal epochs or epochs at least 3 apart. exact_occupancy is not an arena word in this sense, and neither is the readiness word (F13). |
store.rs:147-175; cache/mod.rs:359-361 |
reading |
| F13 | Readiness word: arena.promotion_ready_epoch (the second word of ArenaCtlLine) is written only by (0, relaxed); by R3C (fetch_max with argument R0, release, after F5's scan conditions); and by C4 (fetch_max(e - 1), release; before it, C1 acquire-read , then C2, then C3 acquire-loaded all participant_capacity slots and found none pinned to ). Each fetch_max targets the arena of its argument (arena_of(epoch)). R4 never writes the word. Production code reads it only at C0 (acquire, exact comparison of the full 64-bit value with the probed epoch); Otherwise only test code reads it: the unit tests in cache/promotion.rs and test_access::promotion_ready_epoch (feature verify). |
ArenaCtlLine (arena/mod.rs:36-41); promotion.rs:44-78; rotation.rs:130-185; probe.rs:74; cache/mod.rs:246 |
reading; unit test delayed_certificate_does_not_regress_or_truncate_the_epoch |
| F14 | Probe outcomes, stated sequentially. Let be an index slot of the probed arena and its distance from 's home slot along the probe sequence. Suppose that, in a probe of key at epoch : (1) every I1 of a slot at distance that the probe makes returned an entry with stamp ; (2) for each such entry with 's tag, the candidate's record loads (D4, and the D5 lengths and key loads, after I1F) read the record of the attempt that published that entry; (3) the I1 of returned an -stamped entry with 's tag, and its record loads read that entry's publisher's record, which is for . Then the probe does not return Miss: it returns a hit, Retry or InsufficientCapacity, possibly at an earlier candidate. Contrapositive: a Miss implies that some I1 at distance , or some record load of a candidate at distance , read another value. A correctly read candidate whose key differs (a tag collision) is skipped, never a cause of Miss. All loads of a probe are sequenced before any later probe of the same pass, in particular before the D6 of the hit the pass returns. |
probe.rs: home check 99-108; group scans 109-135 (find_candidate returns the first candidate of its group, even past a terminator, so terminators in 's group do not stop it); candidate validation 144-212 (Miss only from a D4 stamp mismatch, 159-162; key and length mismatches skip the candidate or give Retry); sequential continuation 214-230 (Miss only on a stamp mismatch or a distance beyond the mask) |
reading |
| F15 | Insert operations: an insert's store attempts lie, in , between its call and its return. insert() returns Ok only after one of its attempts made an I2 (PutStatus::Ok or Full), and that I2 is sequenced before the return; ordinary errors after publication do not turn into failure, Corrupt does. insert() returns an error only: (1) before store() (KeyTooLarge, ValueTooLarge), or from reserve(), whose passes make no I2; (2) after a pass whose table_put returned FullRejected, when the following rotate(.., WaitForOwner) fails with an error other than ConcurrentOperation; (3) Corrupt after Full, that is, after the pass published. store()'s loop repeats only after FullRejected, and table_put returns FullRejected only after its probe loop, each of whose I2s leaves the loop with Ok or Full (a failed CAS writes nothing): a FullRejected pass made no I2. Exit (3) is outside the failed-insert guarantee. A promotion attempt belongs to a lookup, not to an insert. |
store.rs:176-203, 111; (1) 90-95, 142-143, 356-359; (2) 190-202; (3) 178-189; put.rs:57-99 (I2 at 78, 86-91; FullRejected at 99) |
reading |
Invariants are grouped in layers; each rests only on earlier layers and on the assumptions and code facts:
SL slots -> OW ownership -> EP epochs/seals -> PN pins -> QU quiescence
-> SP record space -> IX index -> HT hits -> BD bounds
-> RB robustness, index shape -> RD readiness -> FR freshness
layer invariants rests on (besides A0-A9, F1-F15)
SL SL1 -
OW OW1 OW2 SL
EP EP1 EP2 EP3 EP4 OW
PN PN1 PN2 PN3 SL, EP
QU QU1 QU2 PN, EP, OW
SP SP1 SP2 SP3 SP4 SP5 QU, PN, EP, OW, SL
IX IX1 -
HT HT1 HT2 HT3 IX, SP, QU, EP
BD BD1 BD2 IX, SP
RB RB1-RB5 SL, OW, IX, SP, QU (RB3, RB4: needed by G4 only;
RB1, RB2, RB5: no guarantee)
RD RD1 RD2 PN, QU, EP, OW
FR FR1-FR5 RD, HT, RB3, RB4, SP, QU
Slots and ownership.
| Id | Title | Statement (short) | Rests on |
|---|---|---|---|
| SL1 | Slot serialisation | A slot's claims form blocks in , and everything a claim does happens before every later claim. | A0-A4, F7 |
| OW1 | Takeover handoff | Everything the taken-over tenure's registration did happens before the O3; that tenure has no O4. | SL1, A2, A3, A4, F6, F7 |
| OW2 | Ownership is a lock under | Tenures are totally ordered, and everything under one happens before the next. | OW1, A0, A1 |
Epochs and seals.
| Id | Title | Statement (short) | Rests on |
|---|---|---|---|
| EP1 | Epoch sequence | The epoch takes 2, 3, 4, ... once each; R0 R0 implies before . | OW2, A1, F5 |
| EP2 | Seal persistence | After Z every value of stamp is sealed, and a sealed value of stamp is Z or -after it; a load seeing is -before Z. | OW2, EP1, A1, A4, F5 |
| EP3 | Seal before reuse | Z happens before the R4 and R5 of a publishing tenure with R0 , the R1F of every tenure with R0 , everything under the tenure with R0 , everything after an acquire load of the epoch returning , and the caller's W3 of a rotator with R0 . | OW2, EP1, EP2, F2, F5 |
| EP4 | Reset before use | A publishing tenure's R4 happens before every W1 that reads its R5. | F5 |
Pins and quiescence.
| Id | Title | Statement (short) | Rests on |
|---|---|---|---|
| PN1 | Pin writes are release-serial | Non-announce pin writes are serial releases; an acquire read at or after one happens after it. | SL1, A1, A2, F5 |
| PN2 | Attempt closure | An attempt's actions from W2 to W7 are, or happen before, every later non-announce write to its pin. | PN1, SL1, A2, F1 |
| PN3 | Pin visibility | A load after a SeqCst fence that the seal happens before reads the attempt's W2 or later. |
EP2, EP3, A0, F2 |
| QU1 | Quiescence before reuse | An attempt for happens before the R4 of every tenure with R0 . | PN1-PN3, EP1, EP3, OW2, A7, F5 |
| QU2 | Incarnations are serial blocks | Attempts three or more epochs apart are -ordered; no attempt reads a later incarnation. | QU1, EP1, EP4 |
Record space.
| Id | Title | Statement (short) | Rests on |
|---|---|---|---|
| SP1 | B1 lands in its incarnation | Every B1 for reads stamp , after every R4 stamping and before every R4 stamping . | QU1, EP1, EP2, EP4, OW2, F2 |
| SP2 | Bounded bump, one failure per slot | Each slot fails at most one B1 per incarnation; no B1 carries out of the bump. | SP1, PN2, SL1, EP2, A0, A5, F1, F2, F3 |
| SP3 | Chunk words are sequential | Each H1 load reads its own registration's latest store; a chunk is never inherited. | SL1, PN2, A2, F1, F4 |
| SP4 | Disjoint, in-area reservations | Reservations of one incarnation are disjoint and inside the record area. | SP2, SP3, A5, F4 |
| SP5 | Write-once records | Each record word is written at most once per incarnation, and in epoch order across them. | SP4, QU2, F3 |
Index, hits and bounds.
| Id | Title | Statement (short) | Rests on |
|---|---|---|---|
| IX1 | Publication follows the record | An I2 carries its attempt's stamp and offset, after all its record stores. | F3, F8 |
| HT1 | Recycled reads are caught | A lookup load reading an attempt of epoch makes V2 read at least . | EP1, A6, F2, F12 |
| HT2 | Hit integrity | A hit read exactly the record of one attempt of the probed epoch. | HT1, IX1, SP4, SP5, QU2, EP1, A4, A6, F11, F12 |
| HT3 | Value provenance | Every published pair was passed together to insert(). |
HT2, IX1, F10 |
| BD1 | Stores stay in the mapping | Every shared store is in bounds. | SP4, A5, A7, F3 |
| BD2 | Loads stay in the mapping | Every shared load is in bounds, whatever it read. | IX1, SP4, A5, A7, F8, F11 |
Robustness and index shape. No guarantee G1-G3 needs these; they are separated so that the safety argument visibly does not depend on them. G4 needs RB3 and RB4.
| Id | Title | Role |
|---|---|---|
| RB1 | Live registrant never displaced | No guarantee: SL1 holds whenever a steal happens, whether or not the victim was dead. |
| RB2 | Live owner never taken over | No guarantee: OW1 holds whenever a takeover happens. |
| RB3 | Key-check exactness | G1-G3: probe() re-checks the key of every hit. G4: without it a key could get a second slot in one incarnation (RB4). |
| RB4 | Index shape | G4 only: (a), (b) and (d) for FR2 and FR3, (c) for FR3's case and for FR4. |
| RB5 | Pins balanced on return | No guarantee: a leaked pin stalls reuse (R3 timeout or reaping), never breaks safety. |
Readiness and recency. These carry G4; FR5 carries G5.
| Id | Title | Statement (short) | Rests on |
|---|---|---|---|
| RD1 | Certificate soundness | Every store attempt for happens before every certificate for . | PN1-PN3, EP1, EP3, OW2, A7, F5, F13 |
| RD2 | Certificate visibility | An acquire load of the readiness word that returns synchronises with a certificate for . | RD1, A0, F13 |
| FR1 | Fresh source | Every attempt for happens before the I1 that read the entry of the hit a lookup with V1 promotes. | RD1, RD2, F10 |
| FR2 | Promotion freshness | A promotion copies the -last publication of its key in the source incarnation. | FR1, HT2, HT3, RB3, RB4, A6, A8 |
| FR3 | Read-after-insert: no resurrection | A lookup whose final pass happens after a successful insert may miss, but never returns a record rooted in an insert that precedes , regardless of promotions or rotations. | FR2, RB4, HT1, HT2, SP5, QU2, EP1, A6, A8, F2, F9, F10, F12, F14, F15 |
| FR4 | Retention: the lookup does not miss | If, besides, the final pass reads or and nothing of incarnation or later, the lookup does not miss. | FR3, RB4, F14, EP1, OW2 |
| FR5 | Failed inserts are invisible | No store attempt that descends from an insert that returned an error other than Corrupt makes an I2; no lookup returns its record. |
HT2, F8, F10, F15 |
Necessity. The "Rests on" columns give sufficiency. Conversely, each of the following is necessary: dropping it admits the bad execution shown. The evidence column says where a mutant was actually built. Without such evidence, the row is an argument only. Section 14 gives the same analysis for each atomic ordering.
| Dropped | Bad execution | Evidence |
|---|---|---|
| SL1 with A3 (a sound verdict) | A live writer's slot is stolen: its record stores land after the stealer's P6 and the reuse of the arena, so a later hit reads a torn record. | argument |
| A2 (one flow of control per registration) | A signal handler's W2' and W7' on the interrupted registration end its pin early, so the arena is reused under the interrupted writer: a torn hit (section 16, item 5). | argument |
| OW1, OW2 (one owner at a time) | Two owners both read epoch at R0, or a taker misses the dead owner's R5 and publishes an epoch already published: the epoch is rewound and an arena is reset twice under live writers. | GenMC: O1 relaxed gives two owners in competing_dead_rotation_takeovers; P4's failure ordering relaxed fails live_rotation_owner_identity_handoff and owner_slot_reaping_preserves_epoch_publication (section 15) |
| EP2, EP3 (the seal R1, and its persistence) | A writer's B0 sees the arena unsealed after the reusing scan passed its slot; nothing then relates its W3 to the scan, and it writes into the next incarnation. | argument (PN3 step 1 needs B0 Z) |
PN3 for reuse (SeqCst W3 on the writer's side, R1F on the rotator's) |
Store buffering: the scan misses the writer's W2 while the writer's B0 misses the seal; the arena is reset under the writer. | W3 has two roles. Its writer role is needed by every reservation pass, a rotator's included: it is the writer half of the store-buffering pair and HT1's release fence. The scan role, the SeqCst fence that orders a rotation's pin scan after the seal, is R1F's. GenMC: with R1F removed, promotion_keeps_newer_stale_epoch_insert fails. With R1F removed and, besides, W3 skipped in the rotators' passes only (a test hook; the writer's W3 is kept), chunked_insert_blocks_reuse fails (unordered writes between a reusing rotator's reset and the writer's record store). The mutant that skips only the rotators' W3, with R1F kept, passes. That shows only that the scan does not rely on the rotator's W3; it does not show that a rotator's pass can skip W3 in its writer role. |
| R1F (the readiness scan's fence) | Store buffering: the readiness scan misses a writer still pinned to while the writer's B0 misses the seal; R3C certifies falsely and a stale copy is promoted over the writer's newer publication. | GenMC: removing R1F fails promotion_keeps_newer_stale_epoch_insert |
| R3's test for a pin of | The rotation certifies while a writer of is still pinned; a lookup promotes a copy older than that writer's publication. | GenMC: ignoring the test fails promotion_keeps_newer_stale_epoch_insert |
| The certificates' release (R3C, C4) | A C0 that reads the certificate imports none of the attempts' publications; the probe reads an older entry and promotes it. | GenMC: mark_promotion_ready() relaxed, which weakens R3C and C4 together, fails promotion_keeps_newer_stale_epoch_insert. Weakening only one of the two was not measured |
| C0's acquire | The certificate is read but imports nothing: the same stale promotion. | GenMC: C0 relaxed fails promotion_keeps_newer_stale_epoch_insert |
| C0 before I1 | A certificate learned after the probe's loads does not order them after the attempts: a stale copy is promoted (FR1 step 4). | GenMC: moving C0 after the probe's reads fails promotion_keeps_newer_stale_epoch_insert |
| C3's acquire | The recheck sees an unpin without importing the attempt's publication; the reprobe may read an older entry and promote it. | GenMC: C3 relaxed fails promotion_keeps_newer_stale_epoch_insert |
| The reprobe | After a successful recheck, the first copy, read before the recheck, is promoted; it may be older than the last publication. | GenMC: promoting the first copy after a successful recheck fails promotion_keeps_newer_stale_epoch_insert |
| RB3, RB4, A8, I0's acquire (for G4) | A key gets a second slot in one incarnation, or a probe stops before the key's slot: the -last write in the hit's slot is not the key's last publication, and a promotion or a lookup returns an older value (FR2, FR3). | argument |
| PN1, W2's release | The scan reads the registrant's next W2, which does not carry the previous attempt's record stores; the arena is reset before they land. | argument |
| SP3's H0 reset | A registrant that keeps a stolen slot inherits a torn chunk (current epoch, an older chunk's cursor and end) covering another participant's bytes, and writes its first record over that participant's. | GenMC slot_reaping_resets_reservation_chunk stages exactly this state; no mutant run |
| SP2 with A5 (no bump carry) | The bump carries out of bit 63 and restarts low: a later B1 succeeds over bytes already reserved, giving overlapping records. | argument |
| SP5 (write-once records, F3) | Two attempts of one incarnation store into one record word: a torn record, whatever the reader's checks. | argument |
| A4 (non-recurring claim words) | A P4 whose expected word recurred replaces a newer claim than its P0 and T0 examined: SL1 step 3 breaks, and with it OW1. | argument |
| A6 (lifetime non-recurrence) | An entry or lookup that outlives epochs makes a stamp alias: HT2 step 1 accepts a recycled entry. | argument |
| A7 (shared parameters) | An attacher with a smaller participant_capacity skips slots in R3: QU1 fails for the skipped writers (section 16, item 6). It also skips them in R3's readiness test or in C3 and certifies falsely: a stale promotion (RD1 step 2). |
argument |
| A1, A5 | Without A1 no load is ordered against initialisation; without A5 the bump carries (row SP2) or a reader's bounds check is inexact (BD2). | argument |
Not needed for G1-G3: RB1-RB5, A8, A9, I0's acquire. Of these, G4 needs RB3, RB4, A8 and I0's acquire (through RB3).
Needed by no guarantee: RB1, RB2 and RB5; the C1's acquire and C2, redundant on the reachable path (the remark after RD1); the acquire half of O2 when it reads ; any acquire on O3; record-header stamps (D4's stamp check); W7's swap, as opposed to a store. Section 14 lists the orderings that are stronger than needed.
This section tells the whole story without citing rules; sections 8-13 make each step precise. There, each invariant has an Outline before its Proof; where the proof is itself one or two sentences, the two are merged as Outline and proof.
Slots behave sequentially. A slot's state word changes only by compare-and-swap, so its history is a chain: a claim, its register, then the write that ends it, and the next claim. A normal hand-over goes through a release (P3) that the next claim (P1) acquires. A steal goes through the liveness verdict: a "dead" answer orders everything the dead thread did before it, and the steal's CAS confirms that the verdict was about the claim being replaced. So everything one registration does happens before the next registration of the slot starts. Its pin and its chunk words then behave like private variables passed down a chain of owners.
Ownership is a lock, takeovers included. The rotation-owner word works as a lock: O4 releases, O2 acquires. A takeover O3 does not read the owner's release. Instead the taker learns from the owner's slot, with acquire loads, that the owner's registration is over: either it stole that registration itself after a dead verdict, or it saw a later state of the slot, which carries the release of whatever ended the owner. Either way, everything the old owner did happens before the takeover. So rotations run one after another, each starting from the epoch its predecessor published: the epoch climbs 2, 3, 4, ... and every rotation seals its arena before publishing.
Writers are seen by the rotation that reuses their arena. A writer
pins its epoch, fences, and only then checks that its incarnation is
unsealed. The rotation that will reuse that arena, two epochs later,
fences right after its own sealing step, under ownership, and that fence
comes after the seal of the writer's incarnation through the chain of
ownership handoffs. Two SeqCst fences on
opposite sides of the seal cannot both miss each other: either the
writer saw the seal, and stops, or the rotation's scan sees the pin.
The rotation then waits until the pin moves on (an unpin, a new pin, or
a reaper's clear). Each of those is a release made after the writer's
last write, so the reset of the arena happens after everything the
writer did there. Incarnations of an arena are therefore serial, and a
pinned writer never sees a later incarnation.
Within an incarnation, space is handed out once. The bump is a
fetch_add that lands inside its incarnation, so successful
reservations are disjoint intervals. A slot fails at most one add per
incarnation, so the bump stays below 2^31. Chunks are private to their
registration and are reset on every claim, so cuts are disjoint too.
Each attempt writes each word of its own reservation once.
A reader that follows a publication reads that record. The index
publication is a release after the record stores, and the reader fences
with acquire after loading the entry. Every record word it then loads
holds that attempt's store, unless the arena was reused meanwhile. In
that case the reader read a store made after some writer's SeqCst
fence, its acquire fence D6 synchronises with that fence, and the epoch
reload V2 sees the newer epoch and makes the lookup retry. The 32-bit
stamps do not alias because no entry or lookup lives for 2^32 epochs.
Every published pair comes from an insert. An insert stores the
pair its caller passed. A promotion stores its caller's key and the value
of a validated hit in epoch , which, by the previous paragraph, is
exactly the pair some earlier attempt of wrote for that key. That
attempt is an insert or, again, a promotion from an even older epoch, so
an induction on the epoch traces every pair back to an insert().
A promotion copies a finished generation. The rotation that
seals fences right after sealing and then sees every writer still
pinned to : a writer whose fence came first in the SeqCst order is
in the scan, and one whose fence came later sees the seal and writes
nothing. If no slot is pinned to , every writer of has unpinned, each
unpin is a release made after all its writes, and the scan acquires it;
the rotation then publishes a certificate for with a release. A lookup
acquires that certificate before reading anything in , or rechecks the
pins itself and then reads again. Either way every write of happens
before the reads that supply the promoted copy, so each index load sees
the last entry of its slot, and the promotion copies the key's last
publication in . A copy read before the certificate was learned may be
stale; it is returned but never promoted.
Promotions cannot resurrect a superseded insert. Suppose
insert(k, v) returned Ok after publishing in epoch ,
and a lookup's final pass starts happens-after that return. It
reads an epoch . In incarnation , any probe reads the insert's
entry in the key's slot, or a later entry for from an insert that did
not come before. Later incarnations hold only inserts that did not come
before, and promotions, which copy the last value of the generation before
theirs. If and the lookup falls back to , its probe of
missed although the key's chain in happened before it; that requires
reading a store of incarnation or later, and the validation of
whatever the lookup then hits forces a retry. So the lookup never returns
a value whose origin is an insert that came before, regardless of
promotions or rotations. It may miss; no retention is assumed.
If, besides, its final
pass reads or and nothing of incarnation or later, it does not
miss: it hits, or fails with InsufficientCapacity or Corrupt.
A failed insert leaves nothing to find. An insert that returns
an error other than Corrupt publishes nothing: its input was too
large, its reservation or rotation failed, or its record found no bucket
and the rotation that should make room failed. Corrupt returns
are outside this guarantee. A promotion copies a hit, and a hit is always
a publication, so nothing copied from the failed insert can exist either.
Its side effects remain:
its rotations may evict the key's older value.
Bounds checks hold whatever was read. Every index value, however stale, encodes an offset inside the record area. The readers bound the header's length against the area before loading anything else, and derive every further address from values that passed those checks.
Statement. For one slot:
Outline. All writes after are RMWs, so is a chain in which each write reads its predecessor; unique claim words (A4) leave room for only the shape in (a). Ownership passes either by release and acquire (P3 then P1) or through the verdict (A3), which orders a dead thread's actions before , and is -before P4. The start time is loaded relaxed; P4's success is what proves that it belonged to the claim being replaced. Ruling out an earlier claim's start time uses (b) for earlier claims, hence an induction on .
Proof. By well-founded induction on the position of in (A0).
participant.rs:201-241). A successful CAS reads its expected value
and immediately follows that write (RMW). P1 expects FREE; P2 and P5
expect their own claim's INITIALIZING word; P3 expects its own
REGISTERED word; P4 expects the word its P0 loaded, which is never
FREE (steal() returns Free first, participant.rs:282). By A4
each value INITIALIZING() and REGISTERED() is written once in
the slot, by and by its register. So after INITIALIZING()
comes 's register or a P4 stealing ; after REGISTERED()
comes 's P3 or a P4 stealing ; after FREE only a P1. A block
may be the last one, so each part is "at most".is_live(pid_c) = Ok(false) (F7). The opening P0
read before the verdict, identifying an already existing incarnation.
By A3 with , every
action of 's thread, which has pid_c, happens before
P4.is_live_since(pid_c, st), with loaded at T0. We show that T0
read 's own start-time store (T1 or T2). The register CAS reads ,
and P0 reads that register before the verdict, so the incarnation was already
observed. Then A3 applies as above (or with ).
A reaper that dies between P4 and P6 leaves an ordinary claim of its own, which the next reaper steals under the same rules: the chain of claims in (a) covers it.
Statement. Let a successful O3 by thread θ read the acquiring CAS of tenure , held by registration . Then every action on behalf of , and every action under , happens before . Consequently has no O4.
Outline. O3 is a Release CAS: the taker does not synchronise
through the owner word. It synchronises through 's slot instead. O1's
acquire load of an owner word naming imports 's register, so the
steal's first acquire load P0 sees REGISTERED() or a later state.
Then either θ steals itself after a dead verdict, and SL1 gives the
order, or it reads a later state. That later state lies in the release
sequence of the write that ended : 's P3, after all of 's
actions, or a reaper's P4, after 's death. Without this handoff, an O3
ordered before the owner's O4 in would let a second rotator acquire
ownership; and the new owner, which builds on the dead owner's R1 and R5,
would not see them.
Proof.
ω = (slot_G, id_G) at O1 (acquire,
rotation.rs:307), and O3 expects (rotation.rs:336-338). O1
read some acquiring CAS writing . By A4, names one
registration, ; but may be or an earlier tenure of
(owner-word ABA: released and reacquired), so the evidence is
about , not about . is made by 's thread, after 's
register (only a registered participant rotates:
ParticipantLock is constructed after a successful P2/P5 and
retains that registration until drop; its methods forward the slot and
index into rotation. registration_id() reads that registration, participant.rs:359-364). is a
release (O2 is AcqRel, O3 Release), so O1 P0,
and P0 reads or a later state (COH-WR).Live: no O3 follows.Stolen, when P0 read : θ's successful P4 read the same full word.
Init and P3 write zero; P1/P4 set INITIALIZING.
Thus its source is a register with id_G; A4 identifies the claim and SL1 identifies its unique register .
This proves that P4 steals .
SL1(b): every action of happens before that P4, which is
-before . The dead verdict also orders every action of 's thread,
including work for a second registration during reap_pinner(), before P4 (A3).Free and Other return, since their guards exclude 's nonzero word with id_G, and also covers Stolen and Lost when their P0 source differs from .
For Lost when P0 read , P4 failed; a failed CAS reads a value other
than its expected one, and COH-RR (P0 P4) puts that write
after ; P4's failure ordering is acquire. Each of these cases supplies an
acquire read of a write -after , with . By
SL1(c), is at or after the ender of . All state writes
after are RMWs, so lies in , and is a release:
. If is 's P3, every action of is -before it
(A2), as is every action under : the holder releases its original registration only after its tenure. If is a reaper's P4,
SL1(b) orders every action of before it, and the thread-wide dead verdict orders every action under before it (A3).In the code the P3 case cannot precede a successful O3: rotate()
installs a release guard immediately after acquiring ownership (rotation.rs:78).
Its destructor runs O4 on ordinary return and unwinding before the registration can execute P3, so O4 P3, and
step 3 makes O3 fail. The proof does not need this. Nor does it use any
acquire on O3: the taker's edges come from O1, from reap_owner()'s
acquire reads and from A3. O3's release serves the next taker's O1
(step 1 for a later takeover).
Statement.
mo(rotation_owner) is followed by blocks, each an
acquiring CAS (O2 or O3), optionally followed immediately by the same
tenure's O4. An O2 reads or the O4 closing the previous block;
an O3 reads the acquiring CAS just before it.Outline. Once we know that an O4 always sits right after its own tenure's acquiring CAS, every block looks like a lock acquisition and release. The normal handoff is O4's release read by O2's acquire; the takeover handoff is OW1. O3 needs no acquire: it reads an acquiring CAS, and OW1 supplies the order.
Proof.
NONE), O2 CAS(NONE -> me) (AcqRel), O3
CAS(ω -> me) (Release) and O4 store(NONE) (release), one O4 per
tenure, -after its acquiring CAS.NONE value (RMW), so it is an O3, and OW1 says has
no O4. If is the O4 of another tenure , then is
before ( O4t0, COH-WW), and by the induction
hypothesis O4t0 immediately follows , not :
contradiction.NONE from or from an
O4, which closes the previous block; an O3 reads a non-NONE value,
which is an acquiring CAS, immediately before it.O2 must acquire exactly when it reads an O4; O2 and O3 must release (for O1 in OW1 step 1); O4 must release.
Statement. The writes to global_epoch are (abstracted as
one write of 2, A1) and R5s; an R5 of tenure writes R0.
Along the values are 2, 3, 4, ..., each exactly once. If
precedes and has an R5, then R0 R0; hence
R0 R0 implies that precedes . Every thread's
successive loads of the epoch are non-decreasing.
Outline. R0 and R5 run only under ownership, so OW2 makes them serial; each R0 then reads the latest earlier R5, and each R5 adds one.
Proof.
cache/mod.rs:216 and of 2
at , one write by A1) and R5 (rotation.rs:199), which writes
after R0 read (rotation.rs:89) (F5). R0 and R5 are
under ownership, at most one each per tenure.Statement. For stamp (arena ):
No exact shape of is claimed: a taker may repeat R4, and after a death between R4 and R5 the reset has left arena unsealed while the epoch is still .
Outline. Only R4 changes a word's stamp. B1 adds a multiple of 2^33 (a carry out of bit 63 is discarded, so bits 0-32 never change), and R1 sets the sealed bit keeping stamp and bump. So after the seal of stamp , the stamp- values form an RMW chain that keeps the bit, until an R4 writes another stamp. An R4 stamping again would need a tenure with R0 after the sealing tenure (R0 ), which EP1 rules out.
Proof.
rotation.rs:97, 114-119). By (i)
and OW2(b), every R4 stamping happens before Z, so it precedes
Z in (COH-WW); so does (A1). Walk along from Z:
each later write with stamp is a B1 or an R1, an RMW reading its
immediate predecessor; the stamp of the predecessor is then as well, so
by induction it descends from Z through writes that keep the bit.
Each step of that walk goes back along (RMW), so the value is Z
or -after it (transitivity); EP3(ii) needs this order.
Likewise every sealed value with a non-zero stamp is a successful R1 or
-after one, on the same word and with the same stamp (the sealed
values have stamp 0; R4 writes unsealed).
Two successful R1s on stamp are impossible: the later one in would
read a sealed value, but a successful R1 reads the unsealed value it expects.Statement. A successful Z has and happens before:
Form (iii), at R1F, is what the reuse scan relies on; forms (ii) and (iv) are what the readiness scans rely on (PN3).
Outline. A tenure leaves the seal loop only after the seal: its own R1, or a sealed word it loaded, which an earlier tenure sealed (OW2). That holds whether or not the tenure publishes, so the fence it runs right after the loop follows the seal. Two publications later, the seal precedes everything the next-but-one tenure does under ownership. A load that acquires acquires an R5 that followed the seal.
Proof.
rotation.rs:96-123) either on its own successful R1,
which is Z and sequenced before R1F, or on a load under ,
the initial load or a failed CAS, returning , with
R1F (F5). In the second case is Z or -after it
(EP2(ii)), where Z is made by a tenure with R0 . If
preceded , would happen before Z (OW2(b)) and would read a
write -before it (COH-RW): a contradiction. If , Z is
sequenced before R1F. Otherwise precedes (OW2(c)), and
Z happens before everything under , R1F included (OW2(b)).Form (ii) does not assume publication because a tenure may certify (R3C) and then die before R5. Form (i) stays limited to publishing tenures: QU1 and SP1 cover every R4 directly.
Statement. The R4 of a publishing tenure happens before every acquire load (W1) that reads the epoch its R5 wrote.
Outline and proof. R4 R5 (F5, one R4 per tenure), R5 is a release and W1 an acquire reading it: R4 R5 W1 (REL-ACQ).
Statement. The non-announce writes to a pin (W2, W7, P6) are release writes, each made on behalf of a claim, and they are serial ( totally orders them; agrees by COH-WW). Every pin write lies in the release sequence of its head, and has the head's pinned epoch. Hence if an acquire load of the pin reads a non-announce write or any write -after , then .
Outline. W2 and W7 are the registrant's; P6 is a stealer's, made as the new registrant. So SL1 orders them all. R3A is the only other writer, and it is an RMW keeping the pinned epoch, so it never breaks a release sequence.
Proof.
ROTATION_WAITING, F5).W2's release is essential: a scan may read the registrant's next W2, which must carry the previous attempt's record writes.
Statement. Let W2 be the pin store of a store attempt of registration . Every action of from W2 to W7 is, or happens before, every non-announce write to that pin that is -after W2.
The next non-announce write after W2 need not be 's W7 or the P6 of 's own reaper. If never executes its W7 and that reaper dies after its P4 but before its P6, any next non-announce write is the P6 of a later descendant in the steal chain. A consistent schedule can have two successive reapers die before clearing and a third reaper execute that P6.
Outline. If is 's own, F1 makes the attempt's W7 come first. If belongs to a later claim, SL1(b) orders all of before it. An earlier claim cannot write after W2.
Proof. is made on behalf of a claim (PN1).
As a corollary, two store attempts in one slot are -ordered
(att_order).
Statement. Let be a store attempt in slot for epoch :
W2 W3 B0, and B0 read . Let be a
SeqCst fence with Z, and any load of slot 's pin
with . Then reads W2 or a write -after it.
No ordering is needed on W2, B0 or . The production instances are:
The attempt ranges over every store attempt, those of a participant that has just rotated included: W3 is the writer side of every pass. The scan side is , R1F or C2, never a W3.
Outline. Store buffering: two SeqCst fences, one on each side. The
writer's B0 saw the arena unsealed, so it is coherence-before the seal,
and the seal happens before : the writer's W3 comes before in .
A scan missing W2 would put before the writer's W3. A writer cutting
from its chunk does no RMW on , so only these fences relate it to
the rotation.
Proof.
Statement. Let be a store attempt for . For every tenure with R0 that executes R4, every action of from W2 to W7 happens before R4.
This covers every R4, a repeated one too. If no tenure with R0 executes R4, the statement is vacuous.
Outline. The tenure with R0 scans 's slot after its own R1F, which follows Z (EP3(iii)). So the scan sees W2 or later (PN3). It passes only on a different pinned epoch, so it read a write whose head is a later non-announce write : 's W7, a later W2, or a reaper's P6. PN2 puts before , PN1 puts before the scan, and the scan is before R4. Later tenures inherit the order through ownership.
Proof.
Case R0.
participant_capacity slots) with an acquire load after its
R1F , and the final load passed: pinned epoch UNPINNED
or at least R0 (flag masked) (F5;
rotation.rs: load 141, pass test 148). After a reap, the loop reloads, so the final
load is the one that passed.Case R0. R0 read a value at least , so by EP1 (no gaps) some tenure published , with R0; it has an R4 (F5). The first case gives R4. R0 R0, so precedes (EP1), and R4 R4 (OW2(b)).
For a writer that died pinned, the only route from its record writes to
the reuse is: its writes, the verdict (A3), the reaper's P6 (release),
R3's acquire of the cleared pin, R4. The release sequence of W2 does not
cover the record writes, which follow W2. W6 lies between W2 and W7, so
the counter reset at R4 also follows every count of the previous
incarnation; GenMC's chunked_insert_blocks_reuse checks exactly that
pair.
Statement.
Outline. Through the tenure that published : QU1 for its R4, then EP4 to 's W1. Going through tenures, not through an induction over incarnations, also handles incarnations with no attempts.
Proof.
Statement. Every B1 of a store attempt for :
Outline. Every R4 stamping belongs to a tenure at or before the one that published , so it happens before the attempt's W1 (EP4, OW2). Every R4 stamping comes after the attempt (QU1). Between them, only RMWs that keep the stamp touch the word.
Proof.
Statement. Within incarnation :
Outline. A failure leaves the arena sealed or the bump above capacity, and both persist for the rest of the incarnation, so no success follows a failure and the failing slot stops at its next B0. The last success leaves the bump at most , and at most one failure per slot follows it.
Proof. By induction along from the base of (A0), for both parts together: a failure's new bump exceeds capacity only because the bump did not wrap.
Every claim is at most : a chunk is or less by definition,
and a record claim is at most max_record_size (F3).
By SP1, B1 reads a value of stamp . It succeeds iff that value is
unsealed with (store.rs:332); otherwise it
fails, leaving the value sealed, or with bump
(no wrap, by the induction
hypothesis for (b)).
Later writes in are B1s (bump never decreases) and R1s (set the seal), so "sealed or bump > capacity" persists to the next R4. No success follows a failure.
(a). A later attempt of the same slot happens after the failed B1
(att_order, from PN2), so its B0 reads that B1 or a later write
(COH-WR). Seeing another stamp, or sealed, it stops before H1 and B1;
seeing , it may still cut from its chunk at H1, but
stops at store.rs:311-313, before B1.
(b). The base has bump 0; R1 keeps the bump (seal_bump); B1 adds
its claim, so the bump is the sum of the claims. A success needs
and adds at most ; let the last success, by
slot , leave the bump at most . Every failure
comes after it, at most one per slot (a).
Either way it stays at most . A5 checks
(omitting the slack,
config.rs:241-256), so every B1 has .
A failed B1's bytes are lost (an unindexed hole beyond capacity or in a sealed arena); nothing rolls them back.
Statement. Every H1 load of chunk_epoch, chunk_cursor or
chunk_end reads the latest store to that word by its own registration.
The epoch load may read this registration's H0 reset (zero), or an
H2 from an earlier attempt. If its epoch matches the current attempt,
the epoch and end loads read the H2 stores of that chunk's claim; the
cursor load reads that H2's store or the last H1 cut since. For these matching probes, the cursor
stays within [bump + n_0, min(bump + claim, capacity)], where ,
and are those of the claiming B1. Only the chunk_epoch
reset (H0) before a registration's first W2 is needed:
these guarantees also hold if registration and steal omit the cursor and end
reset stores while retaining the epoch store in the same position. Thus a chunk left
by an earlier registration, or in a stolen slot, is never used.
Outline. The chunk words are touched only by the slot's current
registrant, so SL1 makes them sequential. H0 runs before the first pin of
every registration and stores epoch 0, which no epoch equals, so a
matching chunk_epoch can only come from this registration's own H2.
The epoch stored is the full 64-bit epoch: no stamp aliasing.
Proof.
claim() and
steal() after the claim; in attempts), and loaded only by H1. By
SL1 and A2 they are serial, and every H1 load is -ordered against
every store: it reads the latest one before it (serial-reads lemma).chunk_epoch store before an H1 of registration is
's H0 (value 0, which differs from any ) or an H2 of
in an earlier attempt (F4: 's H0 precedes its first W2). If H1
found chunk_epoch == e, it is an H2 of an earlier attempt of
for the same epoch (att_order orders the attempts).
chunk_end's latest store is the same H2: any later H0 or H2 would
also be the latest chunk_epoch store.u32 is exact, because
(A5), so .Statement. In incarnation the byte ranges of successful
reservations, whether won at B1 or cut from a chunk at H1, are pairwise
disjoint and lie in [0, record_area_size). The u32 room check of H1
is exact.
Outline. Successful B1s are RMWs on a bump that never wraps, so their claims are disjoint intervals. A record takes the first bytes of its claim; a chunk is the rest, clipped to capacity, and is cut by one registration with an advancing cursor.
Proof.
[bump + n_0, min(bump + claim, capacity)) of the B1 claim that made the chunk (SP3), so inside that
claim, after its own record, and below capacity.Statement. Each record word is written at most once per incarnation, by the attempt that reserved it (W4 or W5). In the of a record word, the writes of different incarnations appear in epoch order.
Outline and proof. An attempt stores each word of its reservation once, inside its reservation (F3); reservations of one incarnation are disjoint (SP4). Across incarnations, QU2(a) orders the writes.
Statement (code-level). Only (zero fill: stamp 0, offset 0) and I2 write index slots. An I2 of attempt lies in an index slot of 's arena and writes an entry whose stamp is and whose offset is 's own reservation offset, a multiple of 8. Every word of 's record, the lengths word holding 's lengths included, is stored by -before the I2.
Outline. A writer stores its whole record, then publishes an entry naming it; nothing else writes the index. There is no concurrency argument: this restates F3 and F8 as the one index fact that hit integrity needs. The shape of the index (one slot per key, no gap) is not needed for G1-G3; it is RB4, which G4 needs.
Statement. Let a lookup make an arena load (index or record word)
D6 that reads a store of an attempt for epoch . Then V2,
-after D6, reads an epoch . With A6, if for a
probe of epoch , V2's u32 age exceeds 1 and the probe returns
Retry.
Outline. This is the seqlock-style validation. The writer's W3 is a release fence before its store; D6 is an acquire fence after the load that read the store. So the writer's W1 happens before V2, which therefore sees at least the writer's epoch.
Proof.
age_of(V2, E as u32) = V2 - E >= 3 > 1 (util.rs:42-44,
probe.rs:200): Retry.Statement. If probe() returns Ok for key at epoch (
is V1's or ), then: the entry its I1 read was published by an
attempt with ; V2 read an epoch in ; and every
record load of the hit (D4, the lengths, key words and value words D5)
read a store of . So the output is exactly the value wrote for
, and was re-checked against 's key. No torn, mixed or recycled
hit is returned.
Outline. First identify the record: the entry carries 's stamp, so it is a publication of an epoch congruent to modulo 2^32. A6 rules out the zero fill and anything 2^32 older, and HT1 rules out anything newer than V2, so it is exactly 's. Then each record load reads 's store: I2's release and I1F's acquire order 's stores before the loads; earlier incarnations precede them in (QU2), the same incarnation has no other store (SP5), and later incarnations are caught (HT1). The loads address 's record because each address is computed from values already known to be 's.
Proof.
entry_life) says an index load after V1 reads an
entry of an epoch above V1 , the zero fill counting as
epoch 0, so V1 ; then V1 (V1 is at
least 2 by EP1, and is V1 or V1 ), so .
Excluded.age_of(V2, E) <= 1 with V2 , so
V2 .Retry (HT1). F12 leaves no other
writer.8 + align8(key_len) + value_len == len (probe.rs:173) then places
the key words and the padded value words inside
[offset, offset + 8 + align8(len)), which is 's reservation
(F3). Each of them reads 's store by step 3.probe.rs:176-183), and the
value words copied are 's, padding included.The order of step 4 matters: a recycled word can hold any bit pattern, including a plausible header with stamp , so D4's stamp check proves nothing by itself, and no safety proof uses record-header stamps. V2 is the defence, and it covers D4 because D4 is itself one of HT1's loads.
Statement. Every store attempt stores a key and a value that some
caller passed together to insert(). So every record published by an
I2 pairs its key with such a value.
Outline. An insert stores its caller's pair. A promotion stores its caller's key and the value words of a hit in the previous epoch, which by HT2 are those of an attempt one epoch older with the same key. Induct on the attempt's epoch, which is well-founded. The promoted copy may be the reprobe's hit: that is still a validated hit of , sequenced after V1 and after a miss in , so HT2 and the induction apply unchanged. HT3 needs no readiness.
Proof. By induction on .
only_epoch, F10) and
stores the caller's key and the value words that the promoted hit for
in epoch (the first probe of , or the reprobe) copied (F10). By HT2, those words and the matched key are those
of an attempt with e_{w_0} = e - 1 < e_w. By the induction
hypothesis, 's pair was passed to some insert(); equals
's key, and the value words are 's, copied whole.Statement. Every shared store addresses a word inside the mapping:
record stores (W4, W5) lie in the attempt's reservation, hence in the
record area; index stores address slot & bucket_mask; slot and arena
words use indices below participant_capacity and 3.
Outline. A record store lands in the attempt's own reservation, which lies in the record area; every other store uses a masked or bounded index.
Proof. Record stores: F3 (the stores lie in the reservation, whose size
is exact because payload_size() does not wrap, and at most max_record_size)
and SP4. Index stores: put.rs:59. Slots: every
slot index is below participant_capacity (loops at cache/mod.rs:231,
269, 295, rotation.rs:132, and the C3 loop, promotion.rs:60; the
owner's slot is checked at rotation.rs:235), and arena_of takes epoch % 3 (cache/mod.rs:359-361). These indices address the mapping only under A7.
Statement. Every shared load addresses a word inside the mapping, regardless of the values earlier loads returned:
find_candidate::<false> (probe.rs:112-113) is used only when the
whole group fits by the explicit branch guard; the bucket count is a
power-of-two multiple of 8 (A5, config.rs:81);offset + 16 <= record_area_size, so D4 and the lengths word are in
bounds;u32 bound is exact) and 8 + align8(key_len) + value_len == len,
before any key or value load;align8(value_len) <= align8(output capacity).
The allocation bound assumes the caller supplies writable storage for
output_words whole words with output capacity <= 8 * output_words,
as required by ParticipantLock::lookup_raw's unsafe contract
(lock.rs:95-97).
Safe ParticipantLock::lookup supplies this contract through OutputBuffer's
private fields and unconditional constructor assertion
(output.rs:14-30).
The internal production_assert! is omitted in release builds and does
not establish this caller precondition.C0 loads a word of the arena's first line, in bounds like .
Wait-helper loads address the header rotation-owner word (including its
high half on AArch64) or a participant pin selected by the bounded scan.
The initialization magic CAS reads the header word at byte 0. Own-pin and own-state loads address the
slot retained from successful registration, a bounded scan slot (including
is_free and the claim assertion), or the owner-decoded slot when
reap_owner releases its temporary registration. The owner slot
is bounded by the O2/O3 writer's registration index, even for stale O1 values.
These addresses are in bounds
even though the loads are omitted from the concurrency event graph.
Outline. Nothing a reader loads is trusted until checked, and the one value it uses unchecked, the index entry's offset, is in range for every entry ever written, whatever its incarnation.
Proof.
probe.rs:165-169, put.rs:129-134) computes
in u32; since , it is exact.
As is a multiple of 8, it also bounds
align8(len).probe.rs:173, put.rs:142) is done in 64-bit
arithmetic and makes every key and value word lie in
[offset + 16, offset + 8 + align8(len)).pos < value_len <= output_capacity
(probe.rs:186-196); output holds align8(output_capacity) bytes
(caller contract, lookup.rs:37-39, 55).Statement. A P4 never replaces a claim whose thread incarnation is still running.
Outline. A steal needs a dead verdict, and A9 rules one out for a running thread.
Proof. P4 follows a verdict Ok(false) about the claim's pid (and
start time, SL1 step 3). By A9, a running thread is never reported dead.
Not needed for any guarantee: SL1 holds for every steal.
Statement. An O3 never ends a tenure whose holder is still running.
Outline. A takeover needs evidence that the owner's registration ended, and a running owner ends its registration only after releasing ownership.
Proof. OW1's outcomes Stolen, Free, Other and Lost all show
the owner's registration ended (by a dead verdict, A9, or by a write
after its ender). A registration ends while its thread runs only by its
own P3, after its O4 (rotate() installs a ReleaseRotation guard immediately
following successful acquisition). The guard performs O4 on both ordinary return
and unwinding, before control can reach a catch handler or the owning
ParticipantLock's destructor and P3. This includes panics from
GetPid callbacks, which have no no-panic contract. Abort paths do not
continue to P3. Thus a completed P3 has an earlier O4, and the O3
fails. Not needed for safety: OW1 holds for every takeover.
Statement. In table_put of an attempt for , if I0 (acquire)
returns an entry with 's stamp, that entry was published by an
attempt of , and every D4 and D5 load of entry_has_key() reads
's store, so the check decides on 's key.
Outline. The writer's own pin plays the role of the reader's V2: no later incarnation can write while it is pinned.
Proof. As HT2 steps 1-4, with I0's acquire in place of I1F, with
's pin (QU2(b)) in place of V2, and with A6 (entry_life) for the
stamp. G1-G3 do not need it: probe() re-checks every hit's key. G4
does, through RB4: without it a key could get a second slot in one
incarnation, and the -last write in the hit's slot would not be the
last publication of the key. It needs I0's acquire.
Statement. Within incarnation :
Outline. Within an incarnation an index slot, once taken, keeps its key, and a writer moves past a slot only when it holds another key. (d): a CAS replaces only an entry without its stamp; it read its immediate -predecessor, and any publication of at or before that predecessor would carry the stamp.
Proof. (a)-(c): a CAS replaces only an entry without its stamp (F8), a store
only an entry for its key (F9, RB3), and no other incarnation's write
falls inside 's block (QU2(a), A6). So once a slot holds an entry of
, it keeps that key (induction on , A0). A writer advances past a
slot only after finding there an entry of for another key (F9; a
failed CAS re-reads the same slot, put.rs:93-97); its I0 acquired that
entry, so the entry happens before the writer's I2. With equal keys
having equal hashes (A8) and each slot visited once (dist_inj), the key
cannot also sit further on.
(d):
(d) is the CAS branch of the argument for (a), made explicit. G1-G3 do not need RB4. G4 needs (a), (b) and (d) for FR2 and FR3, and (c) for FR3's case and for FR4.
Statement. Every return of insert() and lookup() leaves the
caller's slot unpinned, errors, dropped promotions and FullRejected
included; rotate() and release_claim() are entered unpinned.
Outline. Ordinary reservation/store exits run
W7 before returning to the caller or rotating.
A cleanup guard on each public call also clears an outstanding pin on unwind,
before a caller can catch the panic and reuse the registration. Dropping the
registration runs the same cleanup before release_claim().
Proof. By reading (F1): reserve() unpins (W7, store.rs:352)
before every non-reservation exit after W2 (store.rs:353-359), and
returns before W2 when the promotion's epoch moved (store.rs:269-271);
store() unpins (store.rs:175) right after publication, before any
rotate() or return. lookup() pins only through store(). A leaked
pin would only stall reuse until R3 times out or reaps the slot.
production_asserts check the entry conditions (rotation.rs:76,
participant.rs:238, lock.rs:150-155).
UnpinOnExit in lock.rs is installed before
assert_usable() in insert() and lookup_raw();
lookup() delegates to the latter. Its destructor loads the pin
with relaxed ordering and calls unpin() when the value is nonzero.
The same cleanup precedes release_claim() in
ParticipantLock::drop. COH-WR prevents that load from missing an
outstanding own W2. R3A preserves the pinned epoch, and its nonzero guard
prevents an announcement over zero. Thus a zero cleanup load needs no write;
a nonzero load executes W7, which clears the waiting flag and wakes a waiter.
The guard runs before unwind reaches a caller's catch boundary, including in
a no-std embedding with unwinding. No classification of every possible helper
panic is needed for this cleanup argument. Aborting execution does not enter
release_claim(). PinCleanup.lean proves the arbitrary
interruption/reuse trace and both physical cleanup-load branches.
Recency. The remaining invariants carry G4. They add no axiom
to A0: only REL-ACQ, release sequences of RMWs, FENCE-ACQ, SC-4.4 and the
coherence rules. RD1 is QU1's core argument with "pinned epoch " in
place of "passes"; RD2 is a release-sequence argument over fetch_max;
FR1-FR4 combine them with HT2 and RB4. No proof uses the sampler,
fairness, or readiness ever being reached.
Statement. Let be a certificate Cert: an R3C of a tenure with R0 , or a C4 of a recheck whose C1 read . Then every action from W2 to W7 of every store attempt for happens before .
A store attempt is, by definition, a pass whose B0 saw its incarnation unsealed (section 3.2). When an R3 load observes a pin of , the rotation scan marks promotion unready and takes neither the wait nor the reap edge from that observation. An older-pin observation can enter either path while scanning the arena to reuse.
Observation/reap race. Reaping acts on the slot's current dead registration,
not on the pin value previously loaded by R3: reap_pinner() calls
steal(None, ...). For example, a tenure for epoch 4 reads pin 2 and exhausts its wait.
Before P0, the same registrant can unpin (W7), start another pass (W1 reads 4, W2 pins 4),
and die before W3/B0. P4 and P5 steal its unchanged registration; P6 then clears pin 4.
The acquire reload reads zero and the scan can certify 4. This is why the control-flow
statement concerns the value observed by R3, rather than every pin a later reap may clear.
The consistent two-slot Lean witness
ReapChangedPinWitness.current_pin_cleared_after_older_observation includes the
actual certificate and proves that the new pin is the last pin write before P6.
The new pass never reached B0, and the death-verdict/P6 handoff still orders the
registrant's actions before the reload, as used in the certificate proof.
Outline. The certifier fences after the seal of (EP3(ii) or (iv)). It then loads every slot's pin with acquire and finds none pinned to . By PN3, each load read the attempt's W2 or a later write. A value with a different pinned epoch has a later non-announce head: an unpin, a new pin or a reaper's clear. PN2 orders the attempt before that head, PN1 orders the head before the load, and the load is sequenced before . This is QU1's argument with "pinned epoch " in place of "passes".
Proof.
rotation.rs:143-147), and R3C runs only if none matched: take any of them. In the recheck
case take the C3 load of slot . That load is acquire and returned a
pinned epoch other than .The recheck's own ordering is redundant on the reachable path. A recheck runs only after a validated hit of in the same pass. Its V1 read , which only the R5 of a tenure with R0 writes, and V1 is sequenced before the hit's I1F, so that R5 synchronises with I1F (FENCE-ACQ). ran its R1F-fenced scan before its R5 (F5), and by PN3(b) each of its R3 loads of slot read W2 or later. C3 happens after that load, so it reads at or after what the load read (COH-RR). So C3 sees what 's scan saw or later, without C1's acquire or C2. GenMC agrees: relaxing C1, removing C2, and both together, all pass. The proof above keeps the local argument, which does not depend on the hit that precedes the recheck.
Statement. Let be an acquire load (C0) of the readiness word of
arena that returns , with . Then some certificate for
(a fetch_max with argument ) synchronises with . With RD1,
every action of every store attempt for happens before .
Outline. The word is written only by (value 0) and by
certificates, which are release fetch_max RMWs (F13). The -first
write with value must be a fetch_max with argument . Every later
write up to the one read is an RMW, so all of them are in its release
sequence. A delayed certificate with a smaller argument neither lowers the
value nor breaks the sequence. The value is the full 64-bit epoch,
compared exactly, and is never reset (R4 does not write it), so a
certificate for on the same word, or for an epoch that matches in
its low 32 bits, is never mistaken for one for .
Proof.
The init write need not be -first: its value 0 suffices. A plain
release store in place of fetch_max would break step 2: a delayed store
with a smaller argument would lower the value and end the release
sequence.
Statement. Let a lookup with V1 promote the hit of a probe of (F10). Then every action from W2 to W7 of every store attempt for happens before the I1 that read 's entry.
Outline. F10 allows only two promoted hits. is the hit of a
probe::<true> whose C0 returned : RD2 and then RD1 apply, and
C0 is sequenced before that probe's I1 loads. Or is the reprobe's
hit, sequenced after a successful recheck: its C4 is a certificate for
(RD1), sequenced before the reprobe's I1 loads.
Proof.
promotion_ready flag of a
probe::<true> hit (set only when C0 read ), or after
reprobe_for_promotion() returned a hit, which it does only after the
recheck returned true, so after C4.probe.rs:74, before line 99), which is
the I1 that read 's entry or is sequenced before it.promotion.rs:66,
then lookup.rs:198), hence before the one that read 's entry.Statement. Let a promotion attempt , for and key , promote the hit in . Then the entry that 's I1 read is a publication of by an attempt for , and stores 's value (HT2, HT3). Every publication of by an attempt for , insert or promotion, lies in 's slot, and or . So the promoted record is that of the -last publication of in incarnation : the last insert store into the key's slot, or the slot's CAS if no insert store followed it (RB4(d)). Nothing is assumed about timing, about sampling, or about readiness ever being reached.
Outline. All of 's publications in are in one slot (RB4(b)). FR1 makes each of them happen before the I1 that read , so by COH-WR that I1 read the last one. HT2 then identifies the record and the value.
Proof.
Retry
or Miss (HT1, F14) and nothing is promoted. The promotion itself
may be dropped (only_epoch, a full arena). RB4(c) (no gap) is not
needed here.Statement. Let insert(k, v) return Ok.
Let be a lookup of whose final pass's V1 happens after
's return; in particular, a call that happens after that return qualifies.
may miss. If it returns a hit, every root of the attempt whose record
it returns is , or an insert of that does not
precede : a later or concurrent insert, never one that
returned before was called. This holds regardless of promotions or
rotations, even after 's generation has been evicted. No retention is
assumed. The claim is about provenance: two inserts may store equal values.
By F15, an attempt of published before the successful
return. Write for its epoch, for its publication, and for
its slot. In Lean, successful_insert_no_resurrection obtains this
publication from InsertOps.ok_pub and applies no_older_value
in Recency.lean.
Outline. Let be what V1 read in 's final pass (), and the
epoch of the returned hit ( or ). If , happens before the
hit's I1, so the hit read or a later publication of in , and those
are stores of inserts that do not precede (RB4(d)). If , by
induction on every publication of in has such a root: inserts at
do not happen before 's W1, and promotions copy the last publication of
(FR2). If , so , the probe of before it missed; that
needs one of its loads to read a write of incarnation or later, and
the returned hit's D6 then forces Retry.
Proof.
Miss. By
F14, with , a load of that probe, the I1 of a slot at
distance or a record load of a candidate there, read something other
than its expected value: for a chain slot before , an entry stamped
(for another key); for , the entry stamped for ; for a candidate's
record loads, the record of the attempt that published the entry its I1
read. Each expected write happens before : chain entries by RB4(c), as
each happens before , which happens before the probe (step 5); 's by
step 5; record stores because I2 synchronises with I1F (FENCE-ACQ, as in
HT2 step 2). So read that write or one -after it (COH-WR). The
-later writes of incarnation keep, in an index slot, the stamp and
the key (RB4(a)), hence the tag (A8), so they are expected values too;
record words are written once per incarnation (SP5). Writes of earlier
incarnations are -before (QU2(a)), and the epochs and do not
write arena (F12). So read a store of an attempt for an epoch
. is sequenced before the D6 of the returned hit's probe (the
probe of , or the reprobe; F14), so by HT1 that probe's V2 reads at
least , and it returns Retry (A6), not a hit: a
contradiction.Limits. "Precedes" is happens-before, not real time. Inserts concurrent
with , including later-generation ones, may win. Lookups that do not
happen after may return an older value, but their promotions cannot
make a lookup covered by this theorem return a superseded insert's record.
An insert that fails with an error other than Corrupt is invisible
(FR5), but a lookup after it need not return the previous value.
Statement. Take FR3's setting, and suppose that 's final pass read
at V1, and that no load of that pass reads a store of an
attempt for an epoch (retention). Then does not return
Ok(None): it returns a hit (to which FR3 applies),
InsufficientCapacity or Corrupt.
Sufficient conditions:
Without retention this theorem gives no non-miss guarantee. The
probe() code has Miss exits before V2, so V2 age
validation cannot rule out those exits. For
, the stated V1 hypothesis also fails; the theorem makes
no assertion about a copy in or .
Outline. The probe of reads, along 's chain, entries of for other keys that happen before (RB4(c)), then or a later entry for at , and the records of those entries. It is the probe of the current arena if ; if and the probe of missed, it is the probe of the previous arena, or the reprobe. Retention excludes later incarnations, so F14's condition for a miss cannot arise.
Proof.
InsufficientCapacity, there is nothing to show. Otherwise it missed,
and the probes of are the probe of the previous arena and possibly the
reprobe.Miss needs a load at distance that read a store of an
attempt for an epoch (F14), which retention excludes. So that
probe does not return Miss.Retry is not an outcome of the final pass, by definition. The
remaining outcomes are a hit, InsufficientCapacity and Corrupt.Statement. Let the insert return an error other than
Corrupt. No store attempt that descends from makes an
I2. So no index entry is a publication by such an attempt, and no lookup
returns a hit whose record such an attempt wrote: neither 's value
directly nor a promoted copy of it, in any generation. Corrupt
returns are outside this guarantee.
Outline. An insert that returns an error other than
Corrupt made no publication of its own. A promotion descends from only through a
hit, and a hit's entry is a publication (HT2) by an attempt one epoch
earlier; by induction on the epoch, nothing that descends from ever
publishes.
Proof.
Corrupt, so returned on exit (1) or (2). On those exits,
no attempt of made an I2. Only an attempt
that reserve() returned reaches table_put, and each such attempt of
returned FullRejected, which makes no I2 (F15): only FullRejected
repeats store()'s loop, exit (2) follows one, and on exit (1) the
last reserve() returned no reservation.probe() returns Ok.
By HT2, the hit's entry is an I2 of an attempt , and every record load
of the hit read a store of : the returned record is 's. By step 2,
does not descend from , and, each attempt having exactly one root
(section 3.2), its root is another insert. This covers every generation
and every chain of promotions.Scope notes (not additional guarantees). A failed insert can still have side effects: the rotations it performs or triggers, which evict the oldest generation, so an older value of the key may stop being returned; slot reaping; and record-area space consumed by bytes reserved, or written but never indexed. So it is not guaranteed that a lookup after a failed insert returns the previous value; it is guaranteed only that it does not return the failed insert's value, nor a copy of it.
Bounded test evidence. GenMC's
promotion_keeps_newer_stale_epoch_insert contains the assertion
inserted.is_ok() || last == Outcome::V1, and checks that the run
ends at epoch 3 after setup publishes V1
in epoch 2. These are checks in the bounded harness, not conclusions of the
Lean failed-insert theorem; that theorem proves provenance exclusion and
does not promise a hit or preservation of the previous value.
Each atomic access, its ordering in the code, the minimal ordering the proofs need, and why.
| Event | Code | Minimal | Needed by, and why |
|---|---|---|---|
| W1 | acquire | acquire | EP4 (R4 before use: SP1, QU2). |
| W2 | release | release | PN1: a scan may read the registrant's next W2, which must carry the previous attempt's writes. Essential. |
| W3 | SeqCst fence |
SeqCst |
PN3 (store buffering with R1F, for the reuse and readiness scans, and with C2), HT1 (release fence for D6). Both are writer roles, needed in every reservation pass, a rotator's included. The scan role, the SeqCst fence ordering a rotation's pin scan after the seal, is R1F's; GenMC skipping only the rotators' W3, with R1F kept, passes, which shows that the scan does not rely on it and nothing about the writer role (section 6, necessity row PN3). |
| B0 | relaxed | relaxed | PN3 uses only its value, through . |
| H0, H1, H2 | relaxed | relaxed | SP3: the chunk words are sequential (SL1). |
| B1 | relaxed | relaxed, but an RMW | SP1, SP2, SP4 use RMW adjacency, not ordering. |
| W4, W5 | relaxed | relaxed | Published by I2 (HT2). |
| I0 | acquire | acquire for G4; relaxed for G1-G3 | RB3, hence RB4 and G4 (FR2, FR3). Keep it. |
| I2 | release | release | HT2 (with I1F), RB3 (with I0). |
| W6 | relaxed | unconstrained | Occupancy is a heuristic; QU1 orders it before the reusing R4. |
| W7 | release swap | release store | PN1, QU1. The swap is for the ROTATION_WAITING wake only (liveness). |
| I1 | relaxed | relaxed | A non-matching entry is never dereferenced. |
| I1F | acquire fence | acquire fence | HT2: imports the candidate's publication. |
| D4, D5 | relaxed | relaxed | HT2. |
| D6 | acquire fence | acquire fence | HT1: FENCE-FENCE with the writer's W3. |
| V1, V2 | relaxed | relaxed | HT1, HT2 use COH-RR only. |
| C0 | acquire, sequenced before the probe's first I1 | acquire, and that placement | RD2, FR1. GenMC: relaxed fails, and so does moving it after the probe's reads (promotion_keeps_newer_stale_epoch_insert). |
| C1 | acquire | relaxed on the reachable path | EP3(iv) in RD1's local argument; redundant given the hit's I1F (the remark after RD1). GenMC: relaxed passes. Kept for local reasoning. |
| C2 | SeqCst fence |
none on the reachable path | PN3(c) in RD1's local argument; redundant likewise. GenMC: removing it passes, and so does removing it with C1 relaxed. Kept. |
| C3 | acquire | acquire | RD1 step 5 (PN1: the head happens before the load). GenMC: relaxed fails promotion_keeps_newer_stale_epoch_insert. |
| O0 | relaxed | none needed | A short-cut; R0 decides. |
| O1 | acquire | acquire | OW1 step 1. GenMC: O1 relaxed gives two owners (competing_dead_rotation_takeovers). |
| O2 | AcqRel / failure relaxed |
acquire when it reads an O4; release | OW2 (acquire), OW1 step 1 (release, read by a later O1). |
| O3 | Release / failure relaxed |
release | OW1 step 1 for the next taker. No acquire needed: the taker's edges come from O1, reap_owner() and A3 (section 15). |
| O4 | release | release | OW2. |
| R0 | relaxed | relaxed | OW2 already orders the earlier R5 before it. |
| R1 | relaxed CAS | relaxed, but an RMW keeping the bump | EP2 (stickiness), SP2 (bump). An acquire would import only what this incarnation's writers did before B1, through their W3; their pins () never block the scan, so nothing needs it. |
| R1F | SeqCst fence |
SeqCst |
PN3(a), (b): the scan role, store buffering with each writer's W3, for the reuse scan (QU1) and the readiness scan (RD1). GenMC: removing it fails promotion_keeps_newer_stale_epoch_insert; removing it and skipping the rotators' W3 also fails chunked_insert_blocks_reuse. |
| R3 | acquire | acquire | PN1, QU1, RD1. Ordered by R1F (EP3(ii), (iii); PN3). |
| R3A | relaxed CAS | relaxed, but an RMW keeping the pinned epoch | PN1: it must stay in the release sequence of the pin store it replaces. |
| R3C, C4 | release fetch_max |
release RMW | RD2: an RMW, so that a delayed smaller certificate does not lower the value and every certificate continues the earlier release sequences; release, for REL-ACQ with C0. GenMC: relaxing both (in mark_promotion_ready()) fails promotion_keeps_newer_stale_epoch_insert; relaxing each alone was not measured. |
| R4 | relaxed | relaxed | EP4 (through R5's release). |
| R5 | release | release | EP4 (W1 reading R5), hence SP1 and QU2; EP3(iv) (C1 reading R5), hence RD1's recheck case; and the path through V1 and I1F in the remark after RD1. EP1 does not need it: OW2(b) orders R5 before the next R0. |
| id allocation | relaxed fetch_add |
relaxed RMW | Uniqueness comes from the RMW (A4). |
is_free |
relaxed | none needed | A short-cut; P1 decides. |
| P0 | acquire | acquire | SL1 step 3 (start time), OW1 step 2. |
| P1 | acquire | acquire | SL1 (P3 P1). |
| T0, T1, T2 | relaxed | relaxed | SL1: P4's success confirms whose start time T0 read. |
| P2, P5 | release | release | SL1 step 3 ( P0). |
| P3 | release (own load relaxed) | release | SL1, OW1 ( P3). |
| P4 | release / failure acquire | release; failure acquire | SL1 step 3 (later claims), OW1 ( a reaper's P4; Lost). No acquire on success: P0 already acquired the very write P4 reads (A4). GenMC: failure relaxed breaks live_rotation_owner_identity_handoff. |
| P6 | release store | release | PN1, QU1: stands in for the dead W7. A store, not an exchange: the wake that follows does not depend on the flag. An acquire would import nothing the verdict did not. |
init global_epoch := 2 |
release | (contract) | A1 is a contract of initialize/from_raw; the release helps attachers that acquire the epoch. |
init promotion_ready_epoch := 0 |
relaxed | (contract) | A1. RD2 needs only its value 0, not that it is -first. |
init magic |
SeqCst CAS, release store |
(no crate reader) | Possibly stronger than needed; an external reader might acquire it. |
exact_occupancy (R4, W6, B0) |
relaxed | unconstrained | Heuristic target; not an arena word for HT2. |
The suite in verification/ model-checks the
production code under RC11 (cargo nextest run -p shm_gen_cache_verification); its README lists the expected execution
counts. Programs by invariant:
| Invariant | Programs |
|---|---|
| SL1 | competing_registrars_preserve_winner, rotation_reaping_races_registration, reaper_dies_mid_reap, reaping_dead_registered_reaper_wakes_blocked_rotation, slot_reaping_imports_dead_writer_records |
| OW1, OW2 | competing_dead_rotation_takeovers, dead_rotation_owner_reaping_races_takeover, rotation_owner_reaper_dies_mid_reap, competing_reapers_of_dead_owner_slot, live_rotation_owner_identity_handoff, owner_slot_reaping_preserves_epoch_publication, competing_rotations |
| EP1-EP4 | competing_rotations, dead_owner_after_arena_reset_recovers, dead_owner_epoch_publication_preserves_reservation, owner_slot_reaping_preserves_epoch_publication |
| PN1-PN3, QU1, QU2 | chunked_insert_blocks_reuse, reserved_insert_blocks_reuse, reaping_preserves_live_reservation, slot_reaping_imports_dead_writer_records, rotation_waits_for_unpin |
| SP1-SP5 | chunk_claim_clipped_at_capacity, slot_reaping_resets_reservation_chunk, reserved_insert_survives_occupancy, full_table_insert_retries |
| HT1, HT2, BD2 | concurrent_read_write, lookup_during_reuse, internal_probe_copy_during_reuse, recycled_arena_visibility, public_lookup_capacity_during_reuse, same_key_collision |
| HT3 | promotion_races_insertion, promotion_keeps_newer_insert |
| G4: RD1, RD2, FR1-FR3 | promotion_keeps_newer_stale_epoch_insert: after the joins, inserted.is_err() || last == V2. Its witness STALE_HIT_NOT_PROMOTED (the concurrent lookup read the old value, the insert succeeded, and the final lookup read the new one) shows a stale hit served without shadowing; by itself it does not observe that the stale hit was not promoted. The witnesses PROMOTED_VIA_CERTIFICATE (the lookup promoted because its C0 returned the source epoch, the rotation's scan having seen no pin of it) and PROMOTED_VIA_RECHECK (its C0 did not certify, and the recheck's C4 and the reprobe's hit did) show that the program reaches both promotion paths of F10; the verification build reports the path through GetPid::promotion_hook, since no public result tells them apart. |
| G5: FR5 | promotion_keeps_newer_stale_epoch_insert: after the joins, inserted.is_ok() || last == V1, with V1 retained (epoch 2, still probed at epoch 3). It checks only that instance, in which no rotation evicts V1's generation. |
| Liveness (outside the model) | reaping_wakes_blocked_rotation, reaping_dead_reaper_wakes_blocked_rotation, reaping_dead_reaper_redelivers_lost_wake, owner_wait_misses_reacquired_release, rotation_waits_for_unpin, published_insert_skips_busy_rotation |
The witness PROMOTION_DROPPED_ARENA_FULL of
promotion_keeps_newer_insert shows a lookup that read the older value and
whose promotion found arena 3 full. GenMC does not cover the sampler-declined
path (every program uses 8 buckets, so should_recheck always returns
true); the sampler is covered by the unit tests in
cache/promotion.rs only.
FR4 is checked in one instance only, the final lookup of
promotion_keeps_newer_stale_epoch_insert.
The proofs found no safety violation under A0-A9 and F1-F15.
A3 carries all crash recovery. Every step about a dead
participant (SL1 step 3, OW1, QU1's dead writer) reduces to "the dead
thread's writes happen before the verdict". On Linux that is
/proc/<tid>/stat followed by membarrier(MEMBARRIER_CMD_GLOBAL),
which no memory model covers; the GenMC programs replace it with
GetPid fixtures. With NoopGetPid nothing is ever reaped.
Recency: promotions cannot hide a completed insert. Without promotion readiness, this trace would let a promotion shadow a newer insert:
insert(k, v2) pins , and the rotation happens.Readiness blocks it. The rotation's scan sees the insert's pin
(R3), so the lookup's C0 finds no certificate and the early hit of is returned
but not promoted. A certificate obtained later, by a recheck, leads only to
a fresh probe, which reads (FR1, FR2).
promotion_keeps_newer_stale_epoch_insert checks that a lookup after
the successful insert reads . What remains: G4 orders by
happens-before, not by real time; a lookup concurrent with an insert may
return an older value; inserts of one key that are
not ordered by happens-before are not ordered at all, including one that
lands in a later generation; a failed insert is invisible (G5), but its
rotations may evict the key's older value, so a lookup after it need not
return the previous value; and a
lookup that retries to or later, where is the epoch in which a successful
insert of the key published, may miss (FR4).
Misses are not validated. A probe can return Miss from a stale
index word, or from a recycled header at D4, without reaching V2.
That is sound for a cache; HT2 speaks of hits only. It is why FR4 needs
retention.
A4: id spending and counter wrap. Registration ids come from a u32
counter that wraps (zero skipped). The rotation reapers (reap_pinner(),
reap_owner()) draw an id only after a dead verdict (rotation.rs:210,
239). A registration spends one id per
register_participant() call, drawn before its scan (cache/mod.rs:268)
and used for at most one successful claim or steal (cache/mod.rs:298). So
ids are spent per registration call and per dead slot a rotation reaps. SL1
step 3 still needs a claim's word never to recur in its slot while held,
which is why a taker never reuses its own id for the slot it steals.
A2 outside the C API. The fork guard (fork_generation) lives in
ffi.rs; the C API also forbids use from a signal handler
(ffi.rs:139-141). A Rust caller that forks while holding a
ParticipantLock, or a signal handler that inserts on the interrupted
thread's registration, has two flows of control acting for one
registration, which voids SL1 (a handler's W2'/W7' can give a torn
hit).
A7 and from_raw. The parameters are not checked against the
header. An attacher with a smaller participant_capacity would skip
slots in R3, breaking QU1, and in R3's readiness test and C3, certifying
readiness falsely and letting a stale copy be promoted (RD1).
Liveness holes. Wake-ups, the wait budgets and the timeouts are not treated. Known holes:
participant.rs:268-274).UNPINNED would block reuse for good:
steal() reports Free and reaps nothing. The code keeps it from
arising (P3 asserts an unpinned slot, P6 precedes a stealer's P3),
but no invariant states it.Protocol fidelity is checked by reading. The proofs derive the invariants from F1-F15, not from the Rust source; that the source has F1-F15 is checked by reading, and GenMC checks the source only on small configurations.
Promotion hit rate. A rotation certifies only if no slot is
pinned to during its scan, which inserts under load often prevent. An
unready previous-generation hit is then rechecked with probability
min(1, 8 / bucket_count), so roughly bucket_count / 8 such hits may go
unpromoted per generation before a recheck succeeds, and a dead or slow
pinner of suppresses promotion from for as long as is the previous
generation. That degrades the LRU approximation, never safety. Measured
with sgc_bench (benches/README.md, 5 interleaved rounds, a 32-vCPU
Zen 5 Linux VM), against the same code without readiness: throughput is
unchanged within noise in every scenario, through both the Rust and the C
API (geomean 1.009 and 1.006), so C0's acquire load does not show; the hit
rate is unchanged except in mixed/1Mi/s0.8 with 4 and 8 threads, the
scenario with the most inserts (58% of operations), where it drops by 0.25
to 0.4 percentage points (41.8% to 41.6% and 41.4%). Single-threaded runs
are unaffected: no other pin is held when the rotation scans.