# sneakernet - Test Plan for v1.6.0 - v1.9.0 Features

Hands-on tests for the features added over the last two days. Assumes you have a
test ZFS machine already set up (pool, a label-mountable transport drive, and a
working `sneakernet.conf.yaml`). Role is chosen by `hostname -s` vs
`source.hostname` / `target.hostname`; you swap roles by changing the server name,
and your config already maps **different parent datasets** for source vs target
(e.g. `tank/src` -> `tank/dst`), so a single machine can play both sides.

Everything here is destructive to the test pool only - do not point it at
production data.

Features covered:

| Area | Config / function | Added |
|---|---|---|
| Transport-file verification | `transport.verifyStream`, `verifyTransportFile` | 1.6.0 |
| Standalone file checker | `utilities/checkSneakernetFile` | utility |
| Status carry-over for disabled datasets | `mergeStatusLists` | 1.7.0 |
| Timestamped status backups + retention | `statusFileBackups`, `pruneStatusBackups` | 1.7.0 |
| Full-overwrite of an existing target | `target.allowFullOverwrite`, `receiveTransportStream` | 1.8.0 |
| xz stream compression | `transport.compression`, `buildCompressionPipeline` | 1.9.0 |

Requires `zfs`, `openssl`, and (for Test 6) `xz` on `PATH`.

---

## Conventions

Set these to match your machine, then define helpers. Plug in **your** method for
switching the server name into `as_source` / `as_target`:

```sh
SN=/path/to/zfs_utils/sneakernet/sneakernet
UTIL=/path/to/zfs_utils/utilities/checkSneakernetFile
CONF=$(dirname "$SN")/sneakernet.conf.yaml
STATUS=/path/to/sneakernet_target.status        # = source.statusFile in your config
KEY=...                                          # = transport.encryptionKey
MNT=/mnt/snexfer                                 # = transport.mountPoint

as_source(){ : ; }   # <-- your way of making `hostname -s` match source.hostname
as_target(){ : ; }   # <-- your way of making `hostname -s` match target.hostname

src(){  as_source; "$SN" "$@"; }                 # run as SOURCE
dst(){  as_target; "$SN" "$@"; }                 # run as TARGET
trip(){ src "$@"; echo "--- now target ---"; dst "$@"; }   # one full source->target trip
```

Source datasets and a dated-snapshot helper (snapshot names must be date-parseable):

```sh
newsnap(){ zfs snapshot "$1@$(date +%Y-%m-%d_%H.%M.%S)"; sleep 1; }
reset_target(){ zfs destroy -r tank/dst 2>/dev/null; }   # adjust to your target parent
```

Set verbosity high so verification messages and composed command strings show
(`verbosity: 3` in the config, or pass `-v 3` per run as shown below).

**Run-order rule:** a source run refuses if `serial.txt` already exists on the
transport; a target run refuses if it's missing. Always pair `src` -> `dst`, or
clear `serial.txt` and the `datasets/` dir between source-only runs.

Smoke test before starting:

```sh
reset_target
newsnap tank/src/ds1; newsnap tank/src/ds2
trip
zfs list -rt all tank/dst        # expect ds1 and ds2 present with their snapshots
```

---

## Test 1 - Transport-file verification (`transport.verifyStream`)

**1a. header mode (default).** Ensure `transport.verifyStream: header`:

```sh
reset_target
newsnap tank/src/ds1
src -v 3 2>&1 | tee /tmp/t1a.log
grep -E "verifyTransportFile: (IV file .* verified|header check OK)" /tmp/t1a.log
dst
```

*Pass:* a "header check OK" line per written file; run completes and writes
`serial.txt`.

**1b. full mode (on an incremental).** Establish a baseline first, then switch to
full and send an incremental:

```sh
reset_target; trip
sed -i '' "s/verifyStream: header/verifyStream: full/" "$CONF"
newsnap tank/src/ds1                 # new snapshot => incremental send
src -v 3 2>&1 | tee /tmp/t1b.log
grep -E "verifyTransportFile: (header check OK|full-stream check OK)" /tmp/t1b.log
dst
sed -i '' "s/verifyStream: full/verifyStream: header/" "$CONF"
```

*Pass:* both "header check OK" and "full-stream check OK" appear.

> **Initial/full send under full mode:** the full check dry-runs `zfs receive -nF`
> into the **source** dataset, which already has snapshots. For a base-less full
> stream ZFS may report *"destination has snapshots"* even in a dry run. If a
> first/full send aborts under full mode, that's the same ZFS constraint exercised
> in Test 5 - use `header` for initial fulls. Note the exact behavior you see; the
> real ZFS box is the authority here.

---

## Test 2 - `utilities/checkSneakernetFile`

After a source run, point the checker at a real encrypted file (IV auto-read from
the `.IV` sidecar):

```sh
reset_target
newsnap tank/src/ds1
src                                              # leaves files on $MNT/datasets
# (transport is unmounted at end of run; re-mount to inspect)
mount /dev/gpt/snexfer "$MNT" 2>/dev/null
F=$(ls "$MNT/datasets/" | grep -v '\.IV$' | head -1); echo "$F"
perl "$UTIL" --file "$MNT/datasets/$F" --key "$KEY" --target tank/src/ds1 --debug
echo "exit=$?"
```

*Pass:* `HEADER CHECK: OK` and `FULL-STREAM CHECK: OK`, exit 0. (`--target` must
have matching lineage - the source dataset works.)

Failure paths:

```sh
# wrong key -> exit 3
perl "$UTIL" --file "$MNT/datasets/$F" --key $(printf '0%.0s' {1..64}); echo "exit=$?"

# corrupt ciphertext -> exit 4 (or 3 if the header block was hit)
cp "$MNT/datasets/$F" /tmp/c.bin; cp "$MNT/datasets/$F.IV" /tmp/c.bin.IV
dd if=/dev/urandom of=/tmp/c.bin bs=1 count=512 seek=4096 conv=notrunc 2>/dev/null
perl "$UTIL" --file /tmp/c.bin --key "$KEY" --target tank/src/ds1; echo "exit=$?"

# missing IV sidecar -> exit 1
cp "$MNT/datasets/$F" /tmp/noiv.bin
perl "$UTIL" --file /tmp/noiv.bin --key "$KEY"; echo "exit=$?"

umount "$MNT"; dst                               # consume the good trip
```

*Pass:* exits 3, 4 (or 3), and 1 respectively.

---

## Test 3 - Status carry-over for disabled datasets (`mergeStatusLists`)

A dataset disabled for a run should keep its last-sent snapshot, so re-enabling it
yields an **incremental**, not a full, send.

```sh
reset_target; trip
cat "$STATUS"                                    # lines for ds1 AND ds2
```

Disable `ds2` in the config and run:

```sh
perl -0777 -i -pe 's/^  ds2:.*?maxDelta: [\d.]+\n//ms' "$CONF"   # or comment the ds2 block by hand
newsnap tank/src/ds1
src -v 3 2>&1 | tee /tmp/t3.log
grep -i "Carried over" /tmp/t3.log
cat "$STATUS"                                    # ds2 line STILL present
dst
```

*Pass:* "Carried over 1 status entry…" logged; the `tank/dst/ds2@…` line survives
even though ds2 wasn't processed.

Re-enable `ds2` (restore the block), snapshot it, and run:

```sh
newsnap tank/src/ds2
src -v 4 2>&1 | tee /tmp/t3b.log
grep -iE "ds2.*incremental|incremental send" /tmp/t3b.log    # NOT a full send
dst
```

*Pass:* ds2 is sent incrementally because its prior status survived.

---

## Test 4 - Timestamped status backups + retention (`statusFileBackups`)

Each source run renames the prior status file to `<status>.YYYY-MM-DD_HH.MM.SS`.

```sh
sed -i '' "s/statusFileBackups: .*/statusFileBackups: 2/" "$CONF"
for i in 1 2 3 4; do newsnap tank/src/ds1; trip; sleep 1; done
ls -1 "$STATUS".* | wc -l           # expect 2 (the two newest)
ls -1 "$STATUS".*
```

*Pass:* exactly 2 timestamped backups remain, the most recent ones.

Edge values:

```sh
sed -i '' "s/statusFileBackups: 2/statusFileBackups: 0/" "$CONF"
newsnap tank/src/ds1; trip
ls -1 "$STATUS".* 2>/dev/null | wc -l            # expect 0

sed -i '' "s/statusFileBackups: 0/statusFileBackups: -1/" "$CONF"
for i in 1 2 3; do newsnap tank/src/ds1; trip; sleep 1; done
ls -1 "$STATUS".* | wc -l                        # grows every run (no pruning)

sed -i '' "s/statusFileBackups: -1/statusFileBackups: 5/" "$CONF"   # restore
```

---

## Test 5 - Full-overwrite of an existing target (`target.allowFullOverwrite`)

The original failure case: a re-enabled dataset produces a full send, but the
target dataset still exists with snapshots, so `zfs receive -F` refuses.

Force a full send by clearing the source status while the target keeps ds1:

```sh
reset_target; trip                               # target has tank/dst/ds1 with snaps
rm -f "$STATUS" "$STATUS".*                       # next send is FULL
newsnap tank/src/ds1
```

**5a. allowFullOverwrite = 0 (default) -> clean refusal, target untouched:**

```sh
grep -i allowFullOverwrite "$CONF"               # confirm it is 0
B=$(zfs list -H -t snap -o name tank/dst/ds1 | wc -l)
trip -v 3 2>&1 | tee /tmp/t5a.log
grep -i "cannot overwrite existing dataset\|allowFullOverwrite" /tmp/t5a.log
[ "$(zfs list -H -t snap -o name tank/dst/ds1 | wc -l)" -eq "$B" ] && echo "PASS: untouched"
```

*Pass:* actionable error logged, no destroy, snapshot count unchanged.

**5b. allowFullOverwrite = 1 -> destroy + retry, overwrite succeeds:**

```sh
sed -i '' "s/allowFullOverwrite: 0/allowFullOverwrite: 1/" "$CONF"
rm -f "$STATUS" "$STATUS".*; newsnap tank/src/ds1
trip -v 3 2>&1 | tee /tmp/t5b.log
grep -i "destroying existing\|Retrying receive" /tmp/t5b.log
zfs list -rt snap tank/dst/ds1                   # only the freshly-sent snapshot(s)
```

*Pass:* "destroying existing… to receive a full stream" then "Retrying receive…";
receive succeeds with only the new snapshots.

**5c. Safety - an unrelated failure must NOT destroy.** Feed a garbage (validly
encrypted random) "stream" so the receive fails with *invalid stream* rather than
*has snapshots*, target-only, and confirm the target is left alone:

```sh
reset_target; trip                               # target has tank/dst/ds1
mount /dev/gpt/snexfer "$MNT" 2>/dev/null
IVHEX=$(openssl rand 16 | xxd -p | tr -d '\n')
head -c 200000 /dev/urandom | openssl enc -aes-256-cbc -K "$KEY" -iv "$IVHEX" > "$MNT/datasets/tank.dst.ds1"
printf '%s' "$IVHEX" > "$MNT/datasets/tank.dst.ds1.IV"
date +%s > "$MNT/serial.txt"; umount "$MNT"
B=$(zfs list -H -t snap -o name tank/dst/ds1 | wc -l)
dst -v 3 2>&1 | tee /tmp/t5c.log
grep -qi "destroying existing" /tmp/t5c.log && echo "FAIL: destroyed!" || echo "PASS: no destroy"
[ "$(zfs list -H -t snap -o name tank/dst/ds1 | wc -l)" -eq "$B" ] && echo "PASS: target intact"
sed -i '' "s/allowFullOverwrite: 1/allowFullOverwrite: 0/" "$CONF"   # restore safe default
```

*Pass:* no "destroying existing" line, an import error logged, target intact.
(Adjust the hand-crafted filename to your `pathSubstitution`: `tank.dst.ds1` uses
the default `.`.)

---

## Test 6 - xz stream compression (`transport.compression`)

Enable in the config (the target auto-detects `.xz` regardless, but set it so both
roles read the same file):

```sh
sed -i '' "s/      method: off/      method: xz/" "$CONF"
```

**6a. Files compressed and round-trip correctly:**

```sh
reset_target
yes "compressible line abcdefgh" | head -50000 > /tank/src/ds1/big.txt
newsnap tank/src/ds1
src -v 3 2>&1 | tee /tmp/t6.log
grep -E "xz -zc -6 -T0" /tmp/t6.log              # xz appears before openssl in the pipeline
mount /dev/gpt/snexfer "$MNT" 2>/dev/null; ls -l "$MNT/datasets/"; umount "$MNT"   # *.xz + *.xz.IV
dst -v 3
diff /tank/src/ds1/big.txt /tank/dst/ds1/big.txt && echo "PASS: contents match"
```

*Pass:* `.xz` files written, `xz -zc` shown before `openssl`, target decompresses
and contents match.

**6b. Compression shrinks output (rough check):**

```sh
sed -i '' "s/      method: xz/      method: off/" "$CONF"
reset_target; src; U=$(du -sk "$MNT/datasets" 2>/dev/null || (mount /dev/gpt/snexfer "$MNT"; du -sk "$MNT/datasets"; umount "$MNT")); dst
sed -i '' "s/      method: off/      method: xz/" "$CONF"
reset_target; src; mount /dev/gpt/snexfer "$MNT" 2>/dev/null; C=$(du -sk "$MNT/datasets"|cut -f1); umount "$MNT"; dst
echo "compressed=${C}K vs uncompressed=${U}"     # expect compressed smaller
```

**6c. threads / nice honored** (visible at `-v 3`):

```sh
sed -i '' "s/      threads: 0/      threads: half/; s/      nice: 0/      nice: 10/" "$CONF"
newsnap tank/src/ds1
src -v 3 2>&1 | grep -E "nice -n 10 xz -zc -6 -T[0-9]+"
sed -i '' "s/      threads: half/      threads: 0/; s/      nice: 10/      nice: 0/" "$CONF"
```

*Pass:* pipeline shows `nice -n 10 xz -zc -6 -T<half-your-cores>`.

**6d. Verification under compression:** with `method: xz` and `verifyStream: header`,
the header check must still find the magic (it decompresses after decrypting):

```sh
reset_target; trip
newsnap tank/src/ds1
src -v 3 2>&1 | grep "verifyTransportFile: header check OK"
dst
```

*Pass:* "header check OK" still appears for `.xz` files.

**6e. (Optional) Mixed drive:** after a compressed source run, hand-drop one
uncompressed file (+ `.IV`) for a second dataset into `datasets/`, rewrite
`serial.txt`, and run `dst`. The target should decompress only the `.xz` file and
pass the plain one straight through (suffix-driven detection).

Restore `method: off` if that's your normal default when finished.

---

## Pass-criteria quick reference

| Test | Expect |
|---|---|
| 1a | `verifyTransportFile: header check OK` |
| 1b | `... header check OK` and `... full-stream check OK` |
| 2 good / bad | `HEADER/FULL ... OK` exit 0; failures exit 3 / 4 / 1 |
| 3 disable / re-enable | `Carried over N status entr…`, ds2 line persists; then **incremental** ds2 |
| 4 | exactly `statusFileBackups` timestamped backups kept |
| 5a / 5b / 5c | refuse+untouched / `destroying existing`+`Retrying receive` / **no** destroy + intact |
| 6a / 6c / 6d | `.xz` written + contents match / `nice -n 10 xz … -T<n>` / header OK under xz |

## Notes

- `-v 3` / `-v 4` surface verification messages and the composed command strings
  (so you can confirm `xz`, `-T`, `nice`, and incremental-vs-full).
- `maxDelta` loose in test config avoids false size-guard aborts; tighten for prod.
- Not exercised here (unchanged by 1.6.0-1.9.0): GELI, report drive, cleanup-script
  scheduling.
```