Subversion Repositories zfs_utils

# ZFS_Utils Module Documentation

Perl module providing utilities for ZFS management, GELI encryption, and sneakernet-based replication on FreeBSD systems.

**Version:** 0.2  

**Copyright:** 2024–2025 Daily Data Inc.  

**License:** Simplified BSD License (FreeBSD License)

---

## Exported Functions & Variables

Functions and variables listed in `@EXPORT_OK` and available via `use ZFS_Utils qw(...)`

### Functions

#### `runCmd(@args)`

Execute a shell command and return output.

**Parameters:**

- `@args` - Command and arguments (joined with spaces)

**Returns:**

- In scalar context: full command output as a string

- In list context: output split into lines  

- Empty string on failure

**Behavior:**

- Logs the command execution via `logMsg()`

- If `$merge_stderr` is true, stderr is merged with stdout

- Returns exit status and logs on command failure

- Calls `die` on fatal signal

**Example:**

```perl

my @files = runCmd('ls', '/tmp');

my $output = runCmd('grep pattern /var/log/file');

```

---

#### `shredFile($filename)`

Securely overwrite and delete a file using `gshred`.

**Parameters:**

- `$filename` - Path to file to securely delete

**Returns:** nothing

**Notes:**

- Uses `/usr/local/bin/gshred -u -f -s 32` (3-pass overwrite)

- Silently does nothing if file does not exist

- **Ineffective on COW filesystems** (e.g., ZFS)

- Best used with UFS or ramdisk storage

---

#### `logMsg($message, $filename?, $timeStampFormat?)`

Log a message to file and/or console with timestamp.

**Parameters:**

- `$message` - Message to log (required)

- `$filename` - Path to log file (optional; defaults to `$logFileName`)

- `$timeStampFormat` - strftime format string (optional; defaults to `'%Y-%m-%d %H:%M:%S'`)

**Returns:** nothing

**Behavior:**

- Appends timestamped message to log file if `$filename` is set and non-empty

- Prints to console if `$displayLogsOnConsole` is true

- Format: `TIMESTAMP\tMESSAGE`

**Example:**

```perl

use ZFS_Utils qw(logMsg);

logMsg("Operation started");

logMsg("Debug info", '/var/log/custom.log', '%H:%M:%S');

```

---

#### `mountDriveByLabel($label, $mountPath?, $timeout?, $checkEvery?, $filesystem?, $devPath?)`

Wait for a labeled device to appear and mount it. Works with gpt and msdos labels

**Parameters:**

- `$label` - GPT label name (required; alphanumeric, hyphen, underscore only)

- `$mountPath` - Mount destination (optional; defaults to `/mnt/$label`)

- `$timeout` - Seconds to wait (optional; defaults to 600)

- `$checkEvery` - Polling interval in seconds (optional; defaults to 15)

- `$filesystem` - Filesystem type for mount (optional; defaults to `'ufs'`)

- `$devPath` - Path to GPT devices (optional; defaults to `/dev/gpt/`)

**Returns:**

- Mount path on success

- Empty string on timeout or error

**Behavior:**

- Validates label format (alphanumeric, hyphen, underscore)

- Checks if already mounted; returns path immediately if so

- Polls for device appearance with `$checkEvery` second intervals

- Creates mount point if needed (via `make_path`)

- Logs all operations and errors

- Prints "Waiting for drive labeled..." during polling

**Example:**

```perl

use ZFS_Utils qw(mountDriveByLabel);

my $mp = mountDriveByLabel('replica', '/mnt/backup', 300, 10);

die "Failed to mount" unless $mp;

```

---

#### `loadConfig($filename, $default?)`

Load a YAML configuration file into a hashref.

**Parameters:**

- `$filename` - Path to YAML config file (required)

- `$default` - Default hashref to use if file missing (optional)

**Returns:** hashref loaded from YAML file, or `$default` if provided and file is missing

**Behavior:**

- Dies if no filename provided

- If file missing and `$default` provided, writes `$default` to file in YAML format

- Tries `YAML::XS` first, falls back to `YAML::Tiny`

- Logs which YAML module was used

- Dies if file exists but does not contain a hashref

- Returns empty hashref if both file is missing and no default is provided

**Dependencies:** YAML::XS or YAML::Tiny (at least one required)

**Example:**

```perl

use ZFS_Utils qw(loadConfig);

my $cfg = loadConfig(

    '/etc/app.yaml',

    { debug => 0, port => 8080 }

);

```

---

#### `mountGeli($geliConfig)`

Prepare and decrypt GELI-encrypted disks, then mount the ZFS pool.

**Parameters:**

- `$geliConfig` - hashref with GELI configuration

**Expected keys in `$geliConfig`:**

- `localKey` - Path to local hex key (or hex string)

- `keydiskname` - GPT label of disk holding remote binary keyfile

- `keyfile` - Filename on key disk

- `target` - Path to write combined GELI key

- `diskList` - Arrayref of disk paths to decrypt (passed to `decryptAndMountGeli`)

- `poolname` - Name of ZFS pool to mount (passed to `decryptAndMountGeli`)

**Returns:**

- Pool name on success

- Empty string on error

**Workflow:**

1. Validates local key file exists

2. Mounts key disk via `mountDriveByLabel`

3. Creates combined GELI key via `makeGeliKey`

4. Decrypts disks and mounts pool via `decryptAndMountGeli`

---

#### `makeGeliKey($remote_keyfile, $localKeyHexOrPath, $target)`

Create a GELI key by XOR-ing a remote binary keyfile with a local hex key.

**Parameters:**

- `$remote_keyfile` - Path to binary keyfile (32 bytes; required)

- `$localKeyHexOrPath` - Hex string (64 hex chars) or path to file containing hex (required)

- `$target` - Path where to write resulting 32-byte binary key (required)

**Returns:** 1 on success; dies on error

**Behavior:**

- Reads exactly 32 bytes from `$remote_keyfile` in binary mode

- Accepts `$localKeyHexOrPath` as direct hex string or file path

- Cleans hex: removes `0x` prefix and whitespace

- Validates local key is exactly 64 hex characters (256-bit)

- XORs remote and local buffers byte-by-byte

- Creates target directory if needed

- Writes result with 0600 permissions

- Dies with descriptive error on any failure

**Example:**

```perl

use ZFS_Utils qw(makeGeliKey);

makeGeliKey(

    '/mnt/key_disk/geli.key',

    'a1b2c3d4...(64 hex chars)',

    '/root/combined.key'

);

```

---

### Exported Variables

#### `$logFileName`

Path to the log file for `logMsg()` output.

**Default:** `/tmp/zfs_utils.log`  

**Type:** Scalar string  

**Usage:** Can be overridden by caller before calling any function

**Example:**

```perl

use ZFS_Utils qw(logMsg $logFileName);

$logFileName = '/var/log/myapp.log';

logMsg("This goes to /var/log/myapp.log");

```

---

#### `$displayLogsOnConsole`

Flag to enable/disable console output in `logMsg()`.

**Default:** 1 (enabled)  

**Type:** Integer (0 = disabled, non-zero = enabled)  

**Usage:** Can be toggled at runtime

**Example:**

```perl

use ZFS_Utils qw(logMsg $displayLogsOnConsole);

$displayLogsOnConsole = 0;  # suppress console output

logMsg("Silent operation");

```

---

## Non-Exported Functions

Functions not in `@EXPORT_OK`; use full package path or import explicitly.

### `decryptAndMountGeli($geliConfig)`

Decrypt each GELI disk and import/mount the ZFS pool.

**Parameters:**

- `$geliConfig` - hashref with GELI configuration

**Expected keys:**

- `poolname` - ZFS pool name (required; dies if missing)

- `diskList` - Arrayref of disk paths; auto-discovered if missing

- `target` - Path to GELI keyfile

**Returns:**

- Pool name on success

- Empty string on error (logs instead of dying)

**Behavior:**

- Dies if no pool name specified

- Auto-discovers available disks via `findGeliDisks()` if no `diskList` provided

- Decrypts each disk via `geli attach -k <keyfile> <disk>`

- Logs failures but continues with remaining disks

- Imports pool via `zpool import`

- Mounts all filesystems via `zfs mount -a`

- Logs all operations and errors

**Internal Use:** Called by `mountGeli()`

---

### `findGeliDisks()`

Find all disks available for GELI/ZFS use.

**Parameters:** none

**Returns:** List of disk names (e.g., `('da0', 'da1')`)

**Behavior:**

- Lists all disks from `geom disk list`

- Filters out disks with existing partitions

- Filters out disks already in zpools

- Returns only unpartitioned, unused disks

- Logs the search operation

**Internal Use:** Called by `decryptAndMountGeli()` if no disk list provided

---

### `makeReplicateCommands($sourceSnapsRef, $statusRef?, $newStatusRef?)`

Generate ZFS send commands for replication based on snapshot lists.

**Parameters:**

- `$sourceSnapsRef` - Arrayref of snapshot lines (required)

- `$statusRef` - Arrayref of previously replicated snapshots (optional)

- `$newStatusRef` - Arrayref to populate with newly replicated snapshots (optional)

**Expected format of snapshot lines:**

- First token (space-separated) is the full snapshot name: `pool/filesystem@snapshot [extra tokens...]`

**Returns:** Hashref of `{ filesystem => 'zfs send command' }`

**Behavior:**

- Parses snapshot names from input lines

- Identifies root filesystem (first snapshot's fs)

- Looks up last replicated snapshot per filesystem from `$statusRef`

- Attempts recursive send if all child snapshots share same name

- Falls back to per-filesystem incremental or full sends

- Populates `$newStatusRef` with new snapshot names for status tracking

**Replaces missing `from` snapshots with full sends**

**Example:**

```perl

my @snaps = (

    'tank/home@snap1 some extra info',

    'tank/home/user@snap1 another field'

);

my @old_status = ('tank/home@snap0');

my @new_status = ();

my $cmds = makeReplicateCommands(\@snaps, \@old_status, \@new_status);

foreach my $fs (keys %$cmds) {

    print "For $fs: " . $cmds->{$fs} . "\n";

}

# @new_status now contains the new snapshot names

```

---

## Module Variables (Internal)

- `$VERSION` - Module version (0.2)

- `$merge_stderr` - Global flag for `runCmd()` stderr handling (default: 0, set to 1 in runCmd)

---

## Usage Examples

### Basic Logging

```perl

use ZFS_Utils qw(logMsg $logFileName $displayLogsOnConsole);

$logFileName = '/var/log/backup.log';

$displayLogsOnConsole = 1;

logMsg("Backup started");

logMsg("Backup completed");

```

### Loading Configuration

```perl

use ZFS_Utils qw(loadConfig);

my $config = loadConfig('/etc/backup.yaml', {

    pool => 'tank',

    retention_days => 30,

});

```

### Mounting an Encrypted Pool

```perl

use ZFS_Utils qw(mountGeli);

my $result = mountGeli({

    localKey => '/root/.key/local.hex',

    keydiskname => 'key_disk',

    keyfile => 'geli.key',

    target => '/tmp/combined.key',

    diskList => ['/dev/gpt/encrypted1', '/dev/gpt/encrypted2'],

    poolname => 'backup',

});

die "Failed to mount" unless $result;

print "Pool $result mounted\n";

```

### Creating a GELI Key

```perl

use ZFS_Utils qw(makeGeliKey);

makeGeliKey(

    '/mnt/usb/remote.bin',

    'deadbeefcafebabe' . ('0' x 48),  # 64 hex chars

    '/root/.key/geli.key'

);

```

### Running Commands with Output

```perl

use ZFS_Utils qw(runCmd);

my @lines = runCmd('zfs', 'list', '-H');

foreach my $line (@lines) {

    print "Pool: $line\n";

}

my $output = runCmd('df', '-h');

print $output;

```

---

## Dependencies

- **Core:** Perl 5.10+, strict, warnings, Exporter, Data::Dumper, POSIX, File::Path

- **External (optional):** YAML::XS or YAML::Tiny (at least one required for `loadConfig()`)

- **System (FreeBSD):** gshred, geli, zfs, zpool, geom, gpart, mount

---

## Notes

- All functions use `logMsg()` for diagnostics; configure logging before use

- Functions prefer returning empty strings or undef over dying (except `makeGeliKey`)

- `mountGeli()` and `decryptAndMountGeli()` require FreeBSD with GELI and ZFS

- Binary key operations use raw `:raw` mode for safe byte handling

- XOR operations assume 256-bit (32-byte) keys

---

## License

Simplified BSD License (FreeBSD License) – see module header for full text.