# Report format

The report is one YAML document (Unix) or JSON document (Windows). It is a map of sections; each sensor adds facts to its sections, and the client adds `report`, `system` (from the config), `sysinfo` and its own version. How sensors produce the facts is in [README.md](README.md) (sensor line format) and `unix/sensors/Documentation/README.md` (exit codes). Section numbers below are kept stable because code comments cite them.

Terms: the *existing CAMP importer* ingests YAML, ignores unknown top-level sections, and does not tolerate unknown keys or type changes inside the sections it reads. *v3.7.1* is the previous client generation, kept in `reference/v3.7.1`. Fixtures are real command output under `t/fixtures/<role>/`.

## 1.1 Format

- YAML via `YAML::Tiny`. First line exactly `#sysinfo: v3.8.0 YAML` (no space after `#`), then the document starting `---`.
- `outputType: json` in config stays supported (JSON body, no header line). Default YAML.
- Test mode (`-t`/`--test`, or config `TESTING` true): write `/tmp/sysinfo.testing.yaml`, call no transport.

## 1.2 Old sections (compatibility)

The existing CAMP importer reads these sections. Every key path an older client (v3.7.1) produced on a host is still produced, with the same name, nesting and type (scalar, map or list), and the same value except as listed under "Notes on values". Nothing new is added inside these sections, except `xen.virtual.<guest>.memory|numcpu`.

| Section | Keys | Producer ([README.md](README.md) sensor list) |
|---|---|---|
| `report` | `client`, `date` (`YYYY-MM-DD HH:MM:SS`, local), `version` (`v3.8.0`) | client |
| `system` | `hostname`, `serial` (= effective `serialNumber`, `''` if none), `UUID` (= effective `UUID`); every config key except `moduleDirs`, `scriptDirs`, `transports`, `logging`, `sensorTimeout`, `update` (fleet: `clientName`, `serialNumber`, `location`, `os_type`, `tags` list, sometimes `TESTING`, `postRunScript`); nothing added by the CLI or loader (`testing`, `outputType`, `dataType`) | client |
| `system` | `last_boot` (epoch), `uptime` (seconds) | `linux_system`, `freebsd_system` |
| `system` | `Power Draw`, `Power Current N`, `Power Voltage N` (strings with unit) | `all_ipmi` |
| `attributes` | flat `'<dev> <Key>'`, Key ∈ `Capacity` (bytes, no commas), `Make` (smartctl Model Family), `Model`, `Serial`, `Rotation` (text), `Sector Size`, `type` (raw `-d` string from `smartctl --scan`, trailing space kept), `uptime` (leading digits of attr 9 raw), `writes`. Monthly ([README.md](README.md) (periodic sensors)). `VBOX HARDDISK` excluded as v3.7.1 | `all_smart` |
| `diskinfo.<name>` | filesystems from `df` (`fstype`, `size`, `used` KiB, `mount`) | `all_filesystems` |
| `diskinfo.<name>` | md (`level`, `components`, `size`), LVM (`fstype` = `lvm2 lv|vg|pv`, `size`, `used`, `vg`, `components`), ZFS (`fstype` = `zfs`, `size`, `used`, `level`, `components`: the v3.7.1 flattened string, kept), FreeBSD raw disks (`size` KiB) | `linux_mdadm`, `linux_lvm`, `all_zfs`, `freebsd_storage` |
| `network.<if>` | `address`, `netmask`, `ip6address`, `ip6networkbits`, `mac`, `mtu` (netmask formats differ per OS, kept) | `linux_network`, `freebsd_network` |
| `network.ipmi` | as v3.7.1 `all_ipmi_network` | `all_ipmi` |
| `operatingsystem` | Linux `distribution`, `description`, `release`, `codename`, `os_name`, `kernel`, `os_version`; FreeBSD/OPNsense `os_name`, `kernel` (v3.7.1 got these from `linux_os_values`, which had no OS gate and ran `uname -s`/`-r` on FreeBSD too) | `linux_system`, `freebsd_system` |
| `pci.<slot>.<field>` | Linux as v3.7.1 `linux_pci_lspci`; FreeBSD see F3 | `linux_pci`, `freebsd_pci` |
| `software.<pkg>` | `version`, `description` (dpkg: starts with the architecture, kept; pkg) | `linux_dpkg_software`, `freebsd_pkg_software` |
| `software.sysinfo-client.version` | exactly one entry, dash name, the client version (`v3.8.1`) | client |
| `xen.virtual.<guest>.<field>` | Xen (`xl`) and libvirt guests, category `xen` | `linux_virtual` |

Notes on values (the value differs from what v3.7.1 produced; name and type never change):

- **F1** `attributes.'<dev> writes'` = `storage.disks.<dev>.smart.bytes_written` (§1.4); absent when that is absent. Every v3.7.1 value is a memory address × sector size.
- **F2** `attributes.'<dev> Sector Size'` also filled from smartctl `Sector Sizes:` (the logical size), so it may be newly present.
- **F3** FreeBSD `pci` becomes `pci.<selector>.<field>` (v3.7.1 produced the scalar `pci: rev` on FreeBSD 13+). Fields: `driver` (text before `@`); from the header line `class`, `rev` → `revision`, `hdr` → `header`, and whichever of `card`, `chip`, `vendor`, `device`, `subvendor`, `subdevice` are present; then the indented `key = value` lines (`vendor`, `device`, `class`, `subclass`) override, values processed as v3.7.1 (`cleanUp`, quotes kept). Never a line with an empty selector.
- `system.*` values may change where `server.info` overrides the config (`server.info` in [README.md](README.md)).
- Time-varying: `report.date`, `report.version`, `software.sysinfo-client.version`, `system.uptime`, `system.last_boot`, `system.Power *`, `diskinfo.*.used`, `diskinfo.*.size` of mounted filesystems, `attributes.* uptime`, `xen.virtual.*.memory|numcpu` (may be newly present).

## 1.3 `identity` (new section, sensors `linux_identity` and `freebsd_identity`, daily)

| Key | Linux | FreeBSD / OPNsense |
|---|---|---|
| `system_uuid_raw` | `dmidecode -s system-uuid`, else `/sys/class/dmi/id/product_uuid` | `kenv -q smbios.system.uuid`, else `sysctl -n kern.hostuuid` |
| `system_uuid` | normalised `system_uuid_raw` (rule U) | same |
| `system_uuid_status` | `ok`, `placeholder`, `missing` (rule P) | same |
| `system_serial`, `system_serial_status` | `dmidecode -s system-serial-number`, else `/sys/class/dmi/id/product_serial` | `kenv -q smbios.system.serial` |
| `board_serial` | `dmidecode -s baseboard-serial-number` | `kenv -q smbios.planar.serial` |
| `system_manufacturer`, `system_product` | `dmidecode -s system-manufacturer`, `-s system-product-name` | `kenv -q smbios.system.maker`, `smbios.system.product` |
| `machine_id` | `/etc/machine-id`, else `/var/lib/dbus/machine-id` | `/etc/hostid` |
| `os_distribution`, `os_version`, `os_nickname` | not emitted | OPNsense only: `opnsense-version -O` (JSON) `product_name`, `product_version`, `product_nickname`; absent when the command is missing. Not in `operatingsystem`, where `os_name` stays `FreeBSD` (§1.2) |
| `virtualization` | `virt-what` if present, else inferred from manufacturer/product (QEMU, KVM, Xen, VMware, VirtualBox/innotek, Microsoft Virtual Machine) and `/proc/xen`; `none` if physical | `sysctl -n kern.vm_guest` |
| `configured_uuid`, `configured_serial` | config file `UUID`/`serialNumber` (env, below) | same |
| `server_info_uuid`, `server_info_serial` | from `server.info` when present (env) | same |
| `uuid`, `uuid_source` | effective UUID (32 hex) and `server_info`, `dmi` or `config` | same |
| `serial`, `serial_source` | effective serial and `server_info`, `dmi`, `config` or `uuid` | same |

- **Rule U (normalise)**: lower-case, remove every non-hex character; valid only if exactly 32 hex digits remain. CAMP stores this form (`char(32)`).
- **Rule P (placeholder)**: UUID is `missing` if empty or `Not Settable`/`Not Present`; `placeholder` if not valid under U, or normalised all `0`, all `f`, `03000200040005000006000700080009`, `00020003000400050006000700080009`. Supermicro `00000000-0000-0000-0000-<MAC>` is `ok` (unique). Serial is `missing` if empty; `placeholder` if (case-insensitive, trimmed) it matches `^(1234567890|0123456789|0+|NotSpecified|Not Specified|Not Settable|None|N/A|Default string|To Be Filled By O\.E\.M\.|System Serial Number|Chassis Serial Number|0123456789ABCDEF|Unknown|\.+)$`. The lists live in `library.pm` (`isPlaceholderUUID`, `isPlaceholderSerial`, `normaliseUUID`) and replace `%serialNumberTranslation`/`%UUIDTranslation` in `sysinfoconf.pm`.
- **Effective UUID**: first valid, non-placeholder of: `server.info` UUID, live DMI UUID, config UUID. Absent if none.
- **Effective serial**: first non-placeholder of: `server.info` serial, live DMI serial, config serial; else, if `virtualization` ≠ `none` and `uuid` exists, `uuid` (source `uuid`); else absent. Never an invented value (the v3.7.1/trunk md5 fallback is removed).
- The client passes `SYSINFO_CONFIGURED_UUID`, `SYSINFO_CONFIGURED_SERIAL` (config file values before `server.info`), `SYSINFO_SERVERINFO_UUID`, `SYSINFO_SERVERINFO_SERIAL` (unset when absent) in every sensor's environment.
- Old `system.UUID`/`system.serial` are not normalised or replaced by these.
- CAMP matching (for the server side): `identity.uuid`, then `identity.serial`, then client + hostname, warning on any non-UUID match. For reports without `identity` CAMP applies rules U and P to `system.UUID`/`system.serial` itself.

## 1.4 `storage` (new section)

All keys tab paths under `storage`; sizes integer bytes; booleans `0`/`1`; absent = unknown; units in key names. `<dev>` = kernel name without `/dev/` (`sda`, `da4`, `nvme0n1`); a physical disk behind a hardware RAID controller (`smartctl --scan` `-d megaraid,N`, `sat+megaraid,N`, `cciss,N`, `3ware,N`, `areca,N`) is `<base>_<type>_<N>` (`sda_megaraid_4`) with `behind_controller` = type. Disk discovery and key derivation is one shared function in `library.pm` so both storage sensors agree.

**Disk discovery** (Linux `lsblk -J -b -O`, fallback `lsblk -J -b -o NAME,KNAME,TYPE,SIZE,MODEL,SERIAL,WWN,VENDOR,REV,TRAN,ROTA,LOG-SEC,PHY-SEC,PKNAME`, type `disk`; FreeBSD `sysctl -n kern.disks` + `geom disk list`). Exclude loop, zram, nbd, rbd, optical (`sr*`, `cd*`), SES (`ses*`), `dm-*`, `md*`, iSCSI/CTL LUNs (`tran` = `iscsi`, vendor/model `CTLDISK`).

**Daily, sensors `linux_storage` and `freebsd_storage`** — `storage.disks.<dev>`:

| Key | Source |
|---|---|
| `device` | `/dev/<dev>` |
| `serial`, `wwn`, `vendor`, `model` | lsblk `serial`/`wwn`/`vendor`/`model`; geom `ident`/`lunid`/`descr` |
| `transport` | `sata`, `sas`, `nvme`, `usb`, `virtio`, `xen`, `scsi`, `unknown` (lsblk `tran`; FreeBSD driver name) |
| `size_bytes`, `logical_sector_bytes`, `physical_sector_bytes` | lsblk `size`/`log-sec`/`phy-sec`; geom `Mediasize`/`Sectorsize`/`Stripesize` |
| `rotation_rpm` | geom `rotationrate`; lsblk `rota` = 0 → `0`; else absent |
| `is_virtual`, `logical_volume`, `behind_controller` | virtio/xen/QEMU/VBOX/VMware/Hyper-V; vendor DELL/LSI/AVAGO/BROADCOM/HP with PERC, MR, MegaRAID or LOGICAL VOLUME in model; as above |
| `slot` | FreeBSD `sesutil map`; Linux `/sys/class/enclosure/*/*/device/block/<dev>`. Never guessed |
| `health\tsource` | `json` or `none` (no smartctl ≥ 7.0, or no SMART) |
| `health\tpassed` | `smartctl -j -H -A [-d type]`: `smart_status.passed` → 1/0 |
| `health\treallocated_sectors`, `pending_sectors`, `offline_uncorrectable` | ATA attrs 5, 197, 198 raw |
| `health\tmedia_errors`, `critical_warning` | NVMe health log |
| `health\tgrown_defects` | SCSI `scsi_grown_defect_list` |

Also daily in `linux_mdadm`, `linux_lvm` and `all_zfs` (all members resolved to a `<dev>` as `disk`, plus `partition` when the member is a partition; unresolvable → `disk_unresolved` = 1; resolution via lsblk `pkname`, `readlink -f`, FreeBSD `gpart show -p`/`geom part list`, `glabel status`):

- `partitions.<part>.disk|size_bytes`.
- `md.<md>.level|state|degraded|size_bytes|raid_devices`; `md.<md>.members.<m>.disk|partition|role|slot`, `role` ∈ `active`, `spare`, `failed`, `missing` (from `/proc/mdstat`, `mdadm --detail --export`, `/sys/block/<md>/md/array_state|degraded`).
- `zfs.<pool>.health|size_bytes|allocated_bytes|free_bytes` (`zpool list -pH -o name,size,alloc,free,health`); `zfs.<pool>.vdevs.<vdev>.class|type|state`, `class` ∈ `data`, `log`, `cache`, `spare`, `special`, `dedup` (section headers in `zpool status -P <pool>`, parsed by indentation), `type` ∈ `mirror`, `raidz1|2|3`, `draid…`, `disk`; `…members.<m>.disk|partition|state|read_errors|write_errors|checksum_errors`.
- `lvm.pvs.<pv>.vg|size_bytes|free_bytes|disk|partition|md`; `lvm.vgs.<vg>.size_bytes|free_bytes`; `lvm.lvs.<vg>/<lv>.vg|size_bytes|attr` (`pvs|vgs|lvs --units b --nosuffix --noheadings --separator '|'`).
- Old `diskinfo` md/LVM/ZFS/FreeBSD raw-disk keys, byte-identical to v3.7.1 logic (§1.2).

Expected structures (fixture roles, see Tests in [README.md](README.md)): `freebsd-zfs` pool `storage` has `raidz2-0` (class `data`, da4–da13) and `mirror-1` (class `dedup`, da2, da3); `zroot` `mirror-0` members `da0p2`/`da1p2` with `disk` `da0`/`da1`. `linux-md-lvm` `md1` raid6, eight members `sdc1…sdj1` → `sdc…sdj`, role `active`; `lvm.pvs./dev/md1.md` = `md1`, `vg` = `vg-md1`.

**Monthly, sensor `all_smart`** (gated, [README.md](README.md) (periodic sensors); `smartctl -j -x <dev> [-d type]`, smartmontools ≥ 7.0; without `-j` only the old `attributes` keys, by the v3.7.1 text logic, and `smart.source` = `none`) — `storage.disks.<dev>.smart`:

| Key | Source |
|---|---|
| `source` | `json` or `none` |
| `model_family`, `firmware`, `serial`, `wwn` | `model_family`, `firmware_version`, `serial_number`, `wwn` (`naa-oui-id` hex) |
| `power_on_hours` | `power_on_time.hours` (disk age) |
| `temperature_c` | `temperature.current` |
| `bytes_written`, `bytes_written_source` | rule W |
| `writes_raw`, `writes_raw_unit` | the counter rule W used and its unit word |
| `wear_percent_used`, `wear_source` | rule E |

- **Rule W** (first match): (1) device statistics `Logical Sectors Written` × `logical_block_size` → `devstat`; (2) attr 241 named `Total_LBAs_Written` (or 246 on Micron) × `logical_block_size` → `attr241_lba`; (3) attr named `*_GiB` × 2^30 → `attr_gib`; (4) attr named `*_32MiB` × 2^25 → `attr_32mib`; (5) NVMe `data_units_written` × 512000 → `nvme_data_units`; (6) SCSI `scsi_error_counter_log.write.gigabytes_processed` × 10^9 → `scsi_gb_processed`. Otherwise absent; never a guess.
- **Rule E** (first match, 0 = new, 100 = rated life used): NVMe `percentage_used` → `nvme`; device statistics `Percentage Used Endurance Indicator` → `devstat`; `scsi_percentage_used_endurance_indicator` → `scsi`; ATA attr named `Media_Wearout_Indicator`, `Wear_Leveling_Count`, `SSD_Life_Left` or `Percent_Lifetime_Remain`: 100 − normalised value → `attr_<name>`. Absent for HDDs.
- JSON field names are verified against fixtures; a mismatch is reported, not guessed around.
- Monthly keys are absent on other days by design; CAMP keeps the last value. `sysinfo.sensors.all_smart.status` = `not_due` says so.

## 1.5 `sysinfo` (new section, written by the client)

`client_version`, `data_version`, `config_file` (path used), `server_info_file` (path used, absent if none), `sensors.<filename>.status` (`ok`, `wrong_os`, `not_applicable`, `not_due`, `error`, `timeout`, `skipped_permission`; exit code to status mapping in `sensors/Documentation/README.md`), `.seconds` (wall time, 1 decimal), `.exit_code`, `.message` (first STDERR line or first rejected line, ≤ 200 chars; only when non-empty), `.rejected_lines` (only if > 0).

## 1.7 `havirt_node`, `havirt_domain` (new sections, sensor `linux_havirt`, daily)

`havirt` is the owner's tool for libvirt clusters (installed on shared NFS, linked as `/usr/local/bin/havirt` on each node). Every node sees the whole cluster, so each node reports **only itself and the guests running on it**; CAMP gets each guest from the hypervisor running it, as with `xen.virtual`. The sensor runs exactly two commands, `havirt node list --format tsv` and `havirt domain list --extended --format tsv`. Both read havirt's status file, which `havirt node scan` refreshes from cron; the sensor never runs `scan` or any other havirt command (scan uses SSH to every node and takes a lock).

- **This node**: havirt names nodes by the output of `hostname`. The sensor runs `hostname` and matches it against the `Node` column exactly; if that fails, it compares the first dot-separated label of each side, case-insensitively.
- **TSV parsing**: columns are read by header name, not position, so new havirt columns pass through. In the extended domain list, a line with no leading whitespace is a summary row, `  NICs:` and `  Disks:` start a detail block, and each 4-space-indented line is a tab-separated detail of the preceding summary row: NIC = `dev`, `bridge`, `mac`; disk = `target`, `type`, `source`. Unknown block names are skipped.
- `havirt_node.name` = this node's `Node` value; `havirt_node.<column>` = every other column of its `node list` row (`clock`, `cpu_cores`, `cpu_count`, `cpu_model`, `cpu_socket`, `maintenance`, `memory` KiB, `numa_cells`, `threads_per_core`, …).
- `havirt_domain.<domain>.<column>` = every column of the summary row except `Node` and `Domain` (`maintenance`, `memory` KiB, `nic_count`, `vcpu`, `vnc` = the assigned VNC port, …). Only rows whose `Node` is this node.
- `havirt_domain.<domain>.nic.<mac>` = bridge. The MAC is lower-cased. The `dev` name is not reported: host-side tap names (`vnetN`) change on every guest start.
- `havirt_domain.<domain>.disk.<target>.type` (`block`, `file`, …) and `.source` (path, or havirt's `unknown`), verbatim.
- Values verbatim. **An empty value is omitted, never printed**: the sensor line split drops a trailing empty field, which would turn the key into the value (havirt prints `maintenance` empty when off).
- Exit codes per `sensors/Documentation/README.md`: 2 (not applicable) when `havirt` is not found in `PATH` or at `/usr/local/bin/havirt`, or this host has no row in `node list` (`not a havirt node` on STDERR); 4 (error) when either havirt command exits non-zero or prints no header (e.g. the NFS share is not mounted), first STDERR line = the reason. A node with no running guests emits only `havirt_node` and exits 0.
- Linux only. Runs beside `linux_virtual`, which is unchanged (`xen.virtual.<guest>` from `virsh list`).
- For CAMP (not the sensor): a guest's NIC MAC matches `network.<if>.mac` in that guest's own report, which links guest to device when names differ. Windows guests, which run no sysinfo, are known only from this section.
