Subversion Repositories zfs_utils

# Sample script sync

sync is just a quick and dirty script I created to utilize replicate beyond a cron job. There is no reason to use it unless you want to go beyond putting a line (with parameters) in your crontab.

However, it is useful for me, and it does show how to script the replicate script.

## Configuration

The configuration file is named sync.yaml, and it is in the same directory as the script itself. There is a sample you can use.

- **email** - how to send reports
  
  - **from** - e-mail account which is put in the From: header
  
  - **notify** - e-mail account put in the To: header
  
  - **subject** - Subject line of e-mail

- **target** and **source** - parameters for connecting to the target and source
  
  - **server** - One of them must be remote, but not both. You can put the keyword server, followed by a DNS name or an IP address. Again, **server** only exist in either target or source, but not both.
  
  - **dataset** - Either a list of what you want to replicate from, or a single entry for where you want to replicate to.
    
    - **target** - single dataset to replicate to. The actual dataset will be target/child, where child is calculated from the source entries
    
    - **source** - A list of sources. The key (thing on the left) is a child dataset. The value (thing on the right) is the full parent. So, if you want to replicate from pool/mydata/personal, you wouild put personal on the left, and pool/mydata on the right.

- General - true is anything other than a blank or a 0
  
  - **recurse** - if set to true, will replicate the datasets and all children of them
  
  - **filter** - used as a regular expression to filter the snapshots on the source. datasets not matching this filter are not in the initial sync, but if you sync a newer snapshot, it most likely will bring that one over also.
  
  - **bandwidth** - if this has a value, will limit the bandwidth in *bytes per second* of the transfer. If you want it in bits per second (normal bandwidth unit), divide the bits per second to get the bytes per second. This is performed by placing the Unix command *pv* between the send and receive and using the -L flag. If pv is not installed, this will be ignored.
  
  - **verbose** - integer between 0 and 2
    
    - 0 - no report is generated
    
    - 1 - basic report is generated with name of command, bytes transferred and number of seconds it took to do the replication
    
    - 2 - full report including all commands used, and all values returned by the verbose flag to either send or receive, whichever is local
  
  - **dryrun** - passes dryrun flag to replicate. replicate reports what command(s) would be executed, but does not run them.
  
  - **testing** - if true, does not execute replicate, but shows the command that would be used.
  
  - **shutdown** - Upon completion of replication, shut down the local server. **Warning**: if you get this wrong, you can end up shutting down a production server (ie, source)
  
  - **stats**: if set, will save a date/time stamp, the child dataset name (from source), the number of seconds the job took and the number of bytes which were transferred to a file with this name. If the filename contains the '/' character, it is assumed to be a fully qualified path/filename. Otherwise it is assumed to be a file in the same directory as sync

## Requirements

- YAML::Tiny installed.
  
  - pkg install p5-YAML-Tiny-1.74 or
  
  - cpan -i YAML::Tiny

- It also requires *pv* if you want bandwith limitations.
  
  - pkg install pv

- sendmail (or some variant) configured for report e-mails

- sync also assumes the replicate script and configuration file are in the same directory as sync itself

## Program Flow

1. Configuration file loaded, die if not found or corrupt

2. servername loaded from `hostname` command

3. quick e-mail generated telling user that replication has been started

4. Verifies whichever machine is remote is actually up

5. Checks for maintenance flag and exits immediately if it exists. Used for servers which have the shutdown flag set; bypasses that.

6. Parse command line flags. These will override configuration file

7. for each source defined
   
   1. build replicate command line
   
   2. save replicate command into status variable
   
   3. If not testing
      
      1. execute replicate command line
      
      2. save output from replicate command to status
      
      3. updates statistics file, if set (and not testing)

8. if testing
   
   1. dump status to screen

9. if not testing
   
   1. send status via e-mail
   
   2. shut down machine if flag set