# validateBackup — Implementation Plan

Spec: [ProblemDefinition.md](ProblemDefinition.md). That document is authoritative; this one only sequences the work and prices it.

## Estimating basis (read before trusting the numbers)

Every stage estimate below is a **full task shape** price. It includes, for that stage: the code, its unit tests, the `CHANGELOG.md` entry, the `$VERSION` bump, the prose documentation touched, the auto-memory update, and the `perl -c` lint pass. Those are not free overhead — on a small stage they are the *majority* of the cost, which is exactly why estimates that price only the core diff have historically run 2–3× low.

Model rates used (Anthropic first-party, as of 2026-06-24):

| Model | ID | Input $/MTok | Output $/MTok |
|---|---|---|---|
| Opus 5 | `claude-opus-5` | $5.00 | $25.00 |
| Sonnet 5 | `claude-sonnet-5` | $2.00 | $10.00 |

**Reading the MTok columns.** Input tokens are **cumulative across turns, not the size of the context window** — every turn re-sends the whole conversation, so a stage running 20 turns against a 90K-token context bills roughly 1.8M input tokens even though it never exceeds a 1M window. This is why input dominates the bill despite output being priced 5× higher, and why splitting work across sessions saves real money: context does not carry cost between them.

**Output tokens are the controllable half.** They are billed at 5× input *and* then re-sent as input on every subsequent turn of that session, so each word written is paid for repeatedly. Prose volume — long findings write-ups, expansive docblocks, verbose CHANGELOG entries — is therefore the single largest self-inflicted cost driver, not a style question.

**Hours** are assistant working time (session wall-clock), for scheduling rather than billing. They do not track dollars proportionally: Opus costs 2.5× Sonnet per token but does not take 2.5× as long, and a documentation stage generates many tokens quickly while a debugging stage burns wall-clock on tool round trips. **Developer time is excluded** — running the tests, the manual `TESTING.md` pass, `svn` operations, and measuring active-side RSS and digest throughput are all yours and cost no tokens.

## Stages

| # | Stage | Model | MTok in | MTok out | Est. $ | Hours |
|---|---|---|---|---|---|---|
| 1 | `ZFS_Utils.pm` additions | Sonnet 5 | 1.4 | 0.060 | $3.40 | 1.0 |
| 2 | Script scaffolding + config | Sonnet 5 | 1.9 | 0.090 | $4.70 | 1.75 |
| 3 | Resolution + scrub/autoscrub | **Opus 5** | 1.7 | 0.075 | $10.38 | 2.0 |
| 4 | Listing + merge-join core | **Opus 5** | 2.8 | 0.125 | $17.13 | 3.0 |
| 5 | Sampling + batched digests | Sonnet 5 | 1.8 | 0.080 | $4.40 | 1.5 |
| 6 | Reporting + exit codes | Sonnet 5 | 1.8 | 0.085 | $4.45 | 1.5 |
| 7 | Unit tests | Sonnet 5 | 2.9 | 0.140 | $7.20 | 2.5 |
| 8 | Documentation | Sonnet 5 | 2.6 | 0.170 | $6.90 | 2.5 |
| | **Subtotal** | | **16.9** | **0.825** | **$58.55** | **15.75** |
| | Contingency (see below) | | | | $18–29 | 5–8 |
| | **Total** | | | | **$77–88** | **21–24** |

Worked example, so the arithmetic is checkable — Stage 4: `2.8 × $5.00 = $14.00` input, `0.125 × $25.00 = $3.13` output, total `$17.13`.

### Stage 1 — `ZFS_Utils.pm` additions · Sonnet 5 · $3.40 · 1.0 h

Add `shellQuote`, `sshCommand`, `listSnapshots`, `getDatasetProperties` to **`@EXPORT_OK`**, leaving `@EXPORT` untouched — `sneakernet` defines its own `shellQuote`, and adding it to the default export would trigger a redefinition warning in production code. Update `ZFS_Utils.md` and the root `CHANGELOG.md`, bump `$VERSION`.

Sonnet is adequate because the spec names the four functions and the export mechanism explicitly. The one risk is backwards compatibility with `sneakernet` and `replicate`, and that is mechanical to check rather than subtle.

Note the shape of this estimate: four small functions might be 80 lines of code, yet the stage costs $3.40 because the library doc, the changelog, the version bump, and the tests are nearly all of the work. This is the "small task" pattern that historically got quoted at under a dollar.

Do **not** fold in the stale/orphaned `logMsg` header comment fix noted in `ZFS_Utils.md`'s Freshness block — it is unrelated to this feature and belongs in its own change.

### Stage 2 — Script scaffolding + config · Sonnet 5 · $4.70 · 1.75 h

Script header per `CodingStandards.md` §3, `$programDefinition`, `loadOrCreateConfig`, `checkConfigKeys`, `Getopt::Long`, the `sessionType` profile overlay, `cleanupConfig` precedence (CLI → profile → file → default), `initializeLogging`, the PID lock file, pre-flight checks (tempdir writable and free, required binaries, ssh reachability), and `cleanup`. Structural template is `replicate/replicate`, not `sneakernet`.

Also `validateBackup.datastructure` — roughly 35 commented keys, following the §11 hash-literal rules. That file is most of this stage's output tokens: it is prose-dense, and each key needs an explanatory comment because that is how config options get documented in this project.

`sessionType` is new machinery with no precedent in the project; the closest pattern is `replicate`'s globals-with-per-item-override in `cleanupConfig`.

### Stage 3 — Resolution + scrub/autoscrub · Opus 5 · $10.38 · 2.0 h

Dataset expansion and prefix mapping, per-dataset shared-snapshot resolution ordered by the `creation` property, `mountpoint`/`legacy`/`none`/`mounted=no`/zvol handling, `zpool status` scrub-age parsing, and `autoscrub` with its parallel start, poll loop, and `scrubWaitMax` cap.

**Opus is warranted on two grounds.** `autoscrub` is the one place this read-only tool mutates system state and then blocks for hours — a mistake there is operationally expensive, not merely wrong. And snapshot resolution failing quietly means validating the wrong point in time, which reports success against data nobody asked about.

### Stage 4 — Listing + merge-join core · Opus 5 · $17.13 · 3.0 h

Listing command construction with the stderr exit-status sentinel, `LC_ALL=C` sorting, `readListingRecord`, `mergeJoinListings`, `classifyPair`, `inMtimeWindow`, `recordFinding`.

**The most expensive stage, and the one that most needs Opus.** Every trap in the spec's "Traps the implementation must respect" section lives here, and they share a failure mode: they produce plausible, well-formed, confidently-reported *wrong answers* rather than crashing. `find -s` ordering silently fabricates missing/extra pairs; a locale or `:encoding` layer silently changes the comparison; a truncated listing is byte-for-byte indistinguishable from a complete one. A validation tool that lies is worse than one that fails, so this is the wrong place to economise.

### Stage 5 — Sampling + batched digests · Sonnet 5 · $4.40 · 1.5 h

`selectChecksumSample`, the NUL-delimited sample file, the batched digest command for each side with the path list piped into ssh's stdin, `readChecksumOutput`, `compareChecksums`.

**Decided: stays on Sonnet**, with close review of the output rather than a model upgrade (Opus here would have been $11.00 / 1.75 h).

Two traps in this stage are silent-failure class: `rand(0)` and `rand(1)` both meaning "digest everything", and positional rather than path-based digest correlation misattributing every digest after the first unreadable file. Since Stage 7 runs late, **there is no test feedback on this stage at the time it is written** — the review has to be by reading the code. Concretely, two things to eyeball before moving on:

- `selectChecksumSample` must `return unless $n;` *before* the `int(rand($n)) == 0` test, so `randFile: 0` disables digesting instead of selecting everything.
- `readChecksumOutput`/`compareChecksums` must key on the path `sha256 -r` prints, with no reliance on line order anywhere in the path from command to comparison.

### Stage 6 — Reporting + exit codes · Sonnet 5 · $4.45 · 1.5 h

Run and per-dataset summary blocks, the findings file, `summarizeFindingsByDirectory`, the mandatory caveat lines (partial listing, malformed records, active age filter), and the four exit codes. Delivery reuses `ZFS_Utils::sendReport` with the findings file passed as `$extraArtifacts`.

### Stage 7 — Unit tests · Sonnet 5 · $7.20 · 2.5 h

`testLibrary/test_*.pl`, following the project's extract-and-stub convention (the sub under test is copied verbatim into the test file with a comment saying so, since the script has unguarded top-level code and cannot be `require`d). No `Test::More`; `PASS:`/`FAIL:` lines and a summary, matching the ~30 existing files.

Priority order, highest value first: `mergeJoinListings` (ordering and collation, including the `Documentation/` vs `Documentation.md` case that breaks naive ordering), `selectChecksumSample` (the `randFile` 0/1 semantics), `readChecksumOutput` (correlation by path, with an unreadable file mid-list), `findSharedSnapshot` (empty intersection, all-excluded, undated names), `resolveSnapshotRoot` (legacy/none/unmounted/zvol), `inMtimeWindow`.

`zfs` and `ssh` are stubbed by writing a small `/bin/sh` shim into a `File::Temp::tempdir` and prepending it to `PATH`, as `testLibrary/test_destroyedSourceDataset.pl` already does. Note that no `ssh` shim exists in the project yet — this stage creates the first one.

### Stage 8 — Documentation · Sonnet 5 · $6.90 · 2.5 h

`README.md` (must meet §12: purpose, inventory, at least two concrete command examples, at least one cron example), `USAGE.md` (every parameter and config option), `Documentation/validateBackup.md` (reference doc), `Documentation/TESTING.md` (manual test plan — written before any automation, per the order stated in `replicate/TESTING_PLAN_PROMPT.md`), and `CHANGELOG.md`.

Note this stage has the **highest output-token count of any stage** (0.170 MTok) while costing less than Stage 4 — it is pure prose generation with little iteration. Sonnet deliberately: prose is the largest cost driver and Opus buys little here. No real hostnames anywhere (§14) — "the active host", "the backup host", `reports@example.com`.

## Contingency: $18–29, 5–8 h

Not padding. The spec lists **seven BSD behaviours that could not be verified from the development machine**, plus the active-side memory claim and digest throughput. If any assumption is wrong the rework lands in Stages 4 and 5, the two most sensitive:

- `stat -f '%N\t%z\t%m'` not emitting a bare epoch, or not preserving a literal tab, changes the record format and therefore the sort, the join, and their tests.
- `.zfs/snapshot/<name>` not being traversable under `snapdir=hidden` would force a different access path entirely.
- `sort` not honouring `TMPDIR`, or `sha256 -r`'s output format differing, are smaller but still touch tested code.

The low end assumes the assumptions hold and the first hardware run is clean. The high end assumes one format assumption is wrong and Stage 4's join plus its tests are reworked once — roughly 60% of Stage 4 repeated at Opus rates.

## Cost control

Three things materially change the total, in descending order:

1. **Keep written findings terse.** Long prose written unasked is the largest self-inflicted driver, for the compounding reason above — every output token is billed once at 5× input, then re-billed as input on each later turn. Brief notes with an offer to expand cost a fraction and lose nothing.
2. **Filter tool output at the source.** `grep`/`sed -n` rather than reading whole files. `ZFS_Utils.pm` is 122KB (~35K tokens); reading it whole costs on *every subsequent turn* of that session, not just once.
3. **One stage per session where practical.** Input is cumulative within a session and resets between them, and these stages have clean seams. Running stages 1–8 as one long session would cost multiples of the table above.

If a stage is clearly going to exceed its estimate, that gets raised **before** the work continues, not after — an overrun found afterwards cannot be re-scoped.

## Sequencing

1 → 2 → 3 → 4 → 5 → 6 are ordered by dependency. Stage 8 last, once the parameter set has stopped moving: writing `USAGE.md` before that means writing it twice.

**Decided: Stage 7 runs once, late** — after stages 1–6 — rather than trailing each stage. This matches the standing batch-testing preference (write all the code, run the suite once at the end) and preserves the one-stage-per-session saving, and it is what the table already prices; the totals do not change.

The tradeoff, stated once rather than argued: a test that surfaces a design problem in Stage 4's join costs more to act on late than it would have immediately, which is part of why the contingency is weighted toward Stage 4. Set against that, batching has worked on this project so far, and per-stage testing mainly helps a human author localise errors — which is not the binding constraint here.

## Version control checkpoint

`validateBackup/` stays untracked in SVN for the whole of stages 1–8. Per normal practice here, the project is added **after coding is complete and syntax-validated, but before hand testing by the developer** — so `svn add validateBackup` falls between Stage 8 and the manual `TESTING.md` pass, not before Stage 1.

That ordering has one consequence worth stating: **until that checkpoint there is no SVN history for anything under `validateBackup/`**, so `svn diff` and `svn revert` are unavailable as a safety net across stages, and `getWorkingCopyRevision` will report nothing useful for the new subproject. Stage 1 is the exception — it modifies `ZFS_Utils.pm`, `ZFS_Utils.md`, and the root `CHANGELOG.md`, which *are* tracked, so that stage's changes are diffable and revertable from the start.

The assistant does not run `svn add`, `svn commit`, or any other write-side SVN command; the exact command is listed for the developer at the checkpoint.