# Data Serialization Tools

Revision History: dsEdit v1.4 and dsMerge v1.1 (2026-10-25) - shared code moved to `DataSerialization.pm`. DataSerialization v1.0. dsMerge v1.0 (2026-10-25) - new. dsEdit v1.3 (2026-10-25) - `-f/--format`, `-F/--outformat` and `--delimiter`. v1.2.1 (2026-10-25) - INI review fixes (see below). v1.2 (2026-10-25) - Added INI read/edit/write support, conversion to/from YAML/JSON, and INI acceptance tests.

This directory contains small terminal tools for working with structured data files.

## dsEdit

`dsEdit` is a Perl editor for YAML, JSON, and INI configuration/data files. It loads a file into memory, lets the user navigate nested hashes and arrays, modifies values, and saves changes safely.

### Usage

```bash
./dsEdit /path/to/file.yaml
./dsEdit /path/to/file.json
./dsEdit /path/to/file.ini
./dsEdit -a 'key1.key2:value' -d 'old.key' /path/to/file.yaml
./dsEdit -a 'tags[]:prod' -a 'tags[]:web' /path/to/file.yaml
./dsEdit -d 'tags[]:web' /path/to/file.yaml
./dsEdit --outfile /path/to/output.json /path/to/file.yaml
./dsEdit --outfile /path/to/output.ini /path/to/file.yaml
```

### Non-interactive options

- `-a`, `--addentry 'key1.key2:value'`: Add or update an entry at a dotted path. Leading and trailing whitespace is trimmed from the value before writing. If the target exists and is an array, the value is appended. If the target is scalar or missing, it is replaced/created.
- `-a`, `--addentry 'key1.key2[]:value'`: Append `value` to a list key. On YAML/JSON files this uses array values. On INI files this uses comma-separated scalar values (`a,b,c`) and creates the key if missing.
- `-d`, `--delentry 'key1.key2'`: Delete the target key/index and descendants.
- `-d`, `--delentry 'key1.key2[]:value'`: Delete the first list element exactly matching `value` from `key1.key2`. On INI files this removes the value from a comma-separated scalar and deletes the key when the list becomes empty.
- `-o`, `--outfile /path/to/outfile.[format]`: Write output to the provided path and determine the output format from extension (`.yaml`, `.yml`, `.json`, `.ini`). This also supports same-format copies.
- `-f`, `--format yaml|json|ini`: Treat the input file as this format instead of detecting it from the extension (`yml` is accepted for `yaml`). An in-place save keeps this format.
- `-F`, `--outformat yaml|json|ini`: Write the `--outfile` in this format instead of detecting it from its extension. Requires `--outfile`.
- `--delimiter ':'|'='`: INI only. Read and write key/value lines with this delimiter only, and write `key: value` or `key = value` to match. Without it, either delimiter is read and `key = value` is written. A key containing the delimiter in use cannot be written. This makes colon-style files such as `server.info` editable: `dsEdit -f ini --delimiter ':' -a 'Location:Plant 2' /etc/server.info`. Comments in the file are not kept, and keys are rewritten in sorted order.
- `-h`, `--help`: Show command help.

Multiple `-a` and `-d` options can be used in the same command.

### Notes

- The format drivers, the INI reader and writer, format detection and the atomic write are in `DataSerialization.pm`, shared with `dsMerge`. It must stay in the same directory as `dsEdit` and `dsMerge`, which find it relative to the script (symlinks are followed). Copy all three files together; there are no other helper modules.
- It supports YAML and JSON when the matching Perl modules are installed, and INI with a built-in parser/writer.
- Save operations use a temporary file and a backup file before replacing the original target.
- `[]` is only interpreted as list syntax on the final path segment for non-interactive `-a`/`-d` operations. Literal key names ending in `[]` are not supported.
- INI parsing treats lines before the first section as global keys, supports repeated section headers, accepts both `key=value` and `key: value`, ignores blank/comment lines (`#` and `;` at line start), and keeps all values as strings.
- INI writing sorts global keys, sections, and section keys for deterministic output. Global scalar/array keys are written first, then sections.
- INI writing supports only one nesting level (global scalar/array keys and section scalar/array keys). Writing fails with an error path if deeper nesting exists, if an array element contains a comma, or if a key/section name contains `.`, `=`, or a newline.
- INI reading rejects an empty section name (`[]`), a section with the same name as an earlier global key, and any text after the closing `]` of a header (inline comments are not supported). Spaces around comma-list elements are ignored by the `[]` append/delete forms. JSON/YAML booleans are written to INI as their string form (JSON `true` becomes `1`, `false` becomes `0`).
- INI arrays are written as comma-joined scalar values (`key = a,b,c`). Reading does not split comma-joined values back into arrays, so `a,b,c` is read as a single string.

## dsMerge

`dsMerge` is a Perl tool that merges any mix of YAML, JSON and INI files into one. It is separate from `dsEdit` on purpose; both use `DataSerialization.pm` for the formats, so a fix there applies to both (the format rules in the dsEdit notes above apply, including INI being all strings, one level deep and comments being dropped).

### Usage

```bash
dsMerge site.yaml host.json defaults.ini                     # result on standard output, in the primary's format
dsMerge -o merged.yaml site.yaml host.json defaults.ini
dsMerge -F json -o - site.yaml host.json
cat extra.json | dsMerge -f json site.yaml -
dsMerge -f ini --delimiter ':' -o merged.info a.info b.info
```

### Rules

- The first file is the primary. The other files are merged into it in the order given, one after another, and the result is written once.
- A key is taken from an other file only if the primary lacks it or its value is empty: null, `''`, an empty mapping or an empty list. `0`, `'0'` and boolean false are values, not empty.
- Mappings merge recursively. Anything else already non-empty in the primary is kept, including lists (there is no union of lists) and a scalar where the other file has a mapping, or the reverse.
- The top level of every file must be a mapping. Nothing is written if any input cannot be read or parsed, or if the result cannot be written in the output format (for example nesting deeper than one level for INI).

### Options

- `-o`, `--outfile PATH`: write the result to PATH, with the format taken from its extension (an existing file is kept as a `.bak`). `-o -`, or no `-o`, writes to standard output in the primary's format.
- `-f`, `--format yaml|json|ini`: the format of an input whose extension is not recognised, and of standard input. It does not override a recognised extension, because the inputs are meant to be mixed.
- `-F`, `--outformat yaml|json|ini`: the output format, overriding the `--outfile` extension.
- `--delimiter ':'|'='`: INI key/value delimiter to read and write, as for dsEdit.
- `-`: as a file name, read that input from standard input (once only; it needs `-f`).
- `-h`, `--help`: show help. Messages go to standard error, so standard output only ever holds the result.

### Tests

`perl DataSerialization.t`, `perl ini.t` (also runs `test_dsEdit_list_ops.sh`) and `perl dsMerge.t` (core modules only, run from this directory). `dsMerge.t` covers the merge rules, formats, standard input and output, and the error cases.