Blame | Last modification | View Log | Download | RSS feed
# ZFS_Utils Module Documentation
Perl module providing utilities for ZFS management, GELI encryption, and sneakernet-based replication on FreeBSD systems.
**Version:** 0.2
**Copyright:** 2024–2025 Daily Data Inc.
**License:** Simplified BSD License (FreeBSD License)
---
## Exported Functions & Variables
Functions and variables listed in `@EXPORT_OK` and available via `use ZFS_Utils qw(...)`
### Functions
#### `runCmd(@args)`
Execute a shell command and return output.
**Parameters:**
- `@args` - Command and arguments (joined with spaces)
**Returns:**
- In scalar context: full command output as a string
- In list context: output split into lines
- Empty string on failure
**Behavior:**
- Logs the command execution via `logMsg()`
- If `$merge_stderr` is true, stderr is merged with stdout
- Returns exit status and logs on command failure
- Calls `die` on fatal signal
**Example:**
```perl
my @files = runCmd('ls', '/tmp');
my $output = runCmd('grep pattern /var/log/file');
```
---
#### `shredFile($filename)`
Securely overwrite and delete a file using `gshred`.
**Parameters:**
- `$filename` - Path to file to securely delete
**Returns:** nothing
**Notes:**
- Uses `/usr/local/bin/gshred -u -f -s 32` (3-pass overwrite)
- Silently does nothing if file does not exist
- **Ineffective on COW filesystems** (e.g., ZFS)
- Best used with UFS or ramdisk storage
---
#### `logMsg($message, $filename?, $timeStampFormat?)`
Log a message to file and/or console with timestamp.
**Parameters:**
- `$message` - Message to log (required)
- `$filename` - Path to log file (optional; defaults to `$logFileName`)
- `$timeStampFormat` - strftime format string (optional; defaults to `'%Y-%m-%d %H:%M:%S'`)
**Returns:** nothing
**Behavior:**
- Appends timestamped message to log file if `$filename` is set and non-empty
- Prints to console if `$displayLogsOnConsole` is true
- Format: `TIMESTAMP\tMESSAGE`
**Example:**
```perl
use ZFS_Utils qw(logMsg);
logMsg("Operation started");
logMsg("Debug info", '/var/log/custom.log', '%H:%M:%S');
```
---
#### `mountDriveByLabel($label, $mountPath?, $timeout?, $checkEvery?, $filesystem?, $devPath?)`
Wait for a labeled device to appear and mount it. Works with gpt and msdos labels
**Parameters:**
- `$label` - GPT label name (required; alphanumeric, hyphen, underscore only)
- `$mountPath` - Mount destination (optional; defaults to `/mnt/$label`)
- `$timeout` - Seconds to wait (optional; defaults to 600)
- `$checkEvery` - Polling interval in seconds (optional; defaults to 15)
- `$filesystem` - Filesystem type for mount (optional; defaults to `'ufs'`)
- `$devPath` - Path to GPT devices (optional; defaults to `/dev/gpt/`)
**Returns:**
- Mount path on success
- Empty string on timeout or error
**Behavior:**
- Validates label format (alphanumeric, hyphen, underscore)
- Checks if already mounted; returns path immediately if so
- Polls for device appearance with `$checkEvery` second intervals
- Creates mount point if needed (via `make_path`)
- Logs all operations and errors
- Prints "Waiting for drive labeled..." during polling
**Example:**
```perl
use ZFS_Utils qw(mountDriveByLabel);
my $mp = mountDriveByLabel('replica', '/mnt/backup', 300, 10);
die "Failed to mount" unless $mp;
```
---
#### `loadConfig($filename, $default?)`
Load a YAML configuration file into a hashref.
**Parameters:**
- `$filename` - Path to YAML config file (required)
- `$default` - Default hashref to use if file missing (optional)
**Returns:** hashref loaded from YAML file, or `$default` if provided and file is missing
**Behavior:**
- Dies if no filename provided
- If file missing and `$default` provided, writes `$default` to file in YAML format
- Tries `YAML::XS` first, falls back to `YAML::Tiny`
- Logs which YAML module was used
- Dies if file exists but does not contain a hashref
- Returns empty hashref if both file is missing and no default is provided
**Dependencies:** YAML::XS or YAML::Tiny (at least one required)
**Example:**
```perl
use ZFS_Utils qw(loadConfig);
my $cfg = loadConfig(
'/etc/app.yaml',
{ debug => 0, port => 8080 }
);
```
---
#### `mountGeli($geliConfig)`
Prepare and decrypt GELI-encrypted disks, then mount the ZFS pool.
**Parameters:**
- `$geliConfig` - hashref with GELI configuration
**Expected keys in `$geliConfig`:**
- `localKey` - Path to local hex key (or hex string)
- `keydiskname` - GPT label of disk holding remote binary keyfile
- `keyfile` - Filename on key disk
- `target` - Path to write combined GELI key
- `diskList` - Arrayref of disk paths to decrypt (passed to `decryptAndMountGeli`)
- `poolname` - Name of ZFS pool to mount (passed to `decryptAndMountGeli`)
**Returns:**
- Pool name on success
- Empty string on error
**Workflow:**
1. Validates local key file exists
2. Mounts key disk via `mountDriveByLabel`
3. Creates combined GELI key via `makeGeliKey`
4. Decrypts disks and mounts pool via `decryptAndMountGeli`
---
#### `makeGeliKey($remote_keyfile, $localKeyHexOrPath, $target)`
Create a GELI key by XOR-ing a remote binary keyfile with a local hex key.
**Parameters:**
- `$remote_keyfile` - Path to binary keyfile (32 bytes; required)
- `$localKeyHexOrPath` - Hex string (64 hex chars) or path to file containing hex (required)
- `$target` - Path where to write resulting 32-byte binary key (required)
**Returns:** 1 on success; dies on error
**Behavior:**
- Reads exactly 32 bytes from `$remote_keyfile` in binary mode
- Accepts `$localKeyHexOrPath` as direct hex string or file path
- Cleans hex: removes `0x` prefix and whitespace
- Validates local key is exactly 64 hex characters (256-bit)
- XORs remote and local buffers byte-by-byte
- Creates target directory if needed
- Writes result with 0600 permissions
- Dies with descriptive error on any failure
**Example:**
```perl
use ZFS_Utils qw(makeGeliKey);
makeGeliKey(
'/mnt/key_disk/geli.key',
'a1b2c3d4...(64 hex chars)',
'/root/combined.key'
);
```
---
### Exported Variables
#### `$logFileName`
Path to the log file for `logMsg()` output.
**Default:** `/tmp/zfs_utils.log`
**Type:** Scalar string
**Usage:** Can be overridden by caller before calling any function
**Example:**
```perl
use ZFS_Utils qw(logMsg $logFileName);
$logFileName = '/var/log/myapp.log';
logMsg("This goes to /var/log/myapp.log");
```
---
#### `$displayLogsOnConsole`
Flag to enable/disable console output in `logMsg()`.
**Default:** 1 (enabled)
**Type:** Integer (0 = disabled, non-zero = enabled)
**Usage:** Can be toggled at runtime
**Example:**
```perl
use ZFS_Utils qw(logMsg $displayLogsOnConsole);
$displayLogsOnConsole = 0; # suppress console output
logMsg("Silent operation");
```
---
## Non-Exported Functions
Functions not in `@EXPORT_OK`; use full package path or import explicitly.
### `decryptAndMountGeli($geliConfig)`
Decrypt each GELI disk and import/mount the ZFS pool.
**Parameters:**
- `$geliConfig` - hashref with GELI configuration
**Expected keys:**
- `poolname` - ZFS pool name (required; dies if missing)
- `diskList` - Arrayref of disk paths; auto-discovered if missing
- `target` - Path to GELI keyfile
**Returns:**
- Pool name on success
- Empty string on error (logs instead of dying)
**Behavior:**
- Dies if no pool name specified
- Auto-discovers available disks via `findGeliDisks()` if no `diskList` provided
- Decrypts each disk via `geli attach -k <keyfile> <disk>`
- Logs failures but continues with remaining disks
- Imports pool via `zpool import`
- Mounts all filesystems via `zfs mount -a`
- Logs all operations and errors
**Internal Use:** Called by `mountGeli()`
---
### `findGeliDisks()`
Find all disks available for GELI/ZFS use.
**Parameters:** none
**Returns:** List of disk names (e.g., `('da0', 'da1')`)
**Behavior:**
- Lists all disks from `geom disk list`
- Filters out disks with existing partitions
- Filters out disks already in zpools
- Returns only unpartitioned, unused disks
- Logs the search operation
**Internal Use:** Called by `decryptAndMountGeli()` if no disk list provided
---
### `makeReplicateCommands($sourceSnapsRef, $statusRef?, $newStatusRef?)`
Generate ZFS send commands for replication based on snapshot lists.
**Parameters:**
- `$sourceSnapsRef` - Arrayref of snapshot lines (required)
- `$statusRef` - Arrayref of previously replicated snapshots (optional)
- `$newStatusRef` - Arrayref to populate with newly replicated snapshots (optional)
**Expected format of snapshot lines:**
- First token (space-separated) is the full snapshot name: `pool/filesystem@snapshot [extra tokens...]`
**Returns:** Hashref of `{ filesystem => 'zfs send command' }`
**Behavior:**
- Parses snapshot names from input lines
- Identifies root filesystem (first snapshot's fs)
- Looks up last replicated snapshot per filesystem from `$statusRef`
- Attempts recursive send if all child snapshots share same name
- Falls back to per-filesystem incremental or full sends
- Populates `$newStatusRef` with new snapshot names for status tracking
**Replaces missing `from` snapshots with full sends**
**Example:**
```perl
my @snaps = (
'tank/home@snap1 some extra info',
'tank/home/user@snap1 another field'
);
my @old_status = ('tank/home@snap0');
my @new_status = ();
my $cmds = makeReplicateCommands(\@snaps, \@old_status, \@new_status);
foreach my $fs (keys %$cmds) {
print "For $fs: " . $cmds->{$fs} . "\n";
}
# @new_status now contains the new snapshot names
```
---
## Module Variables (Internal)
- `$VERSION` - Module version (0.2)
- `$merge_stderr` - Global flag for `runCmd()` stderr handling (default: 0, set to 1 in runCmd)
---
## Usage Examples
### Basic Logging
```perl
use ZFS_Utils qw(logMsg $logFileName $displayLogsOnConsole);
$logFileName = '/var/log/backup.log';
$displayLogsOnConsole = 1;
logMsg("Backup started");
logMsg("Backup completed");
```
### Loading Configuration
```perl
use ZFS_Utils qw(loadConfig);
my $config = loadConfig('/etc/backup.yaml', {
pool => 'tank',
retention_days => 30,
});
```
### Mounting an Encrypted Pool
```perl
use ZFS_Utils qw(mountGeli);
my $result = mountGeli({
localKey => '/root/.key/local.hex',
keydiskname => 'key_disk',
keyfile => 'geli.key',
target => '/tmp/combined.key',
diskList => ['/dev/gpt/encrypted1', '/dev/gpt/encrypted2'],
poolname => 'backup',
});
die "Failed to mount" unless $result;
print "Pool $result mounted\n";
```
### Creating a GELI Key
```perl
use ZFS_Utils qw(makeGeliKey);
makeGeliKey(
'/mnt/usb/remote.bin',
'deadbeefcafebabe' . ('0' x 48), # 64 hex chars
'/root/.key/geli.key'
);
```
### Running Commands with Output
```perl
use ZFS_Utils qw(runCmd);
my @lines = runCmd('zfs', 'list', '-H');
foreach my $line (@lines) {
print "Pool: $line\n";
}
my $output = runCmd('df', '-h');
print $output;
```
---
## Dependencies
- **Core:** Perl 5.10+, strict, warnings, Exporter, Data::Dumper, POSIX, File::Path
- **External (optional):** YAML::XS or YAML::Tiny (at least one required for `loadConfig()`)
- **System (FreeBSD):** gshred, geli, zfs, zpool, geom, gpart, mount
---
## Notes
- All functions use `logMsg()` for diagnostics; configure logging before use
- Functions prefer returning empty strings or undef over dying (except `makeGeliKey`)
- `mountGeli()` and `decryptAndMountGeli()` require FreeBSD with GELI and ZFS
- Binary key operations use raw `:raw` mode for safe byte handling
- XOR operations assume 256-bit (32-byte) keys
---
## License
Simplified BSD License (FreeBSD License) – see module header for full text.