# ha-sync

## Overview

`ha-sync` performs bidirectional file synchronisation between the **Dell node** (`192.168.2.100`) and the **HP node** (`192.168.2.193`) over NFS-mounted volumes. Each sync run is recorded to MySQL, and runs are scheduled as Kubernetes CronJobs in the `infrastructure` namespace.

A distributed Kubernetes Lease (`ha-sync-<jobname>`) prevents concurrent runs of the same job, and an optional dry-run mode lets you preview what _would_ happen without touching any files.

**CronJobs are now managed by `ha-sync-ctl` and the DB** — static YAML manifests have been archived. Do not apply `cron-*.yaml` manually.

---

## Tools

| Binary | Purpose |
|--------|---------|
| `ha-sync` | Sync runner — invoked by K8s CronJobs |
| `ha-sync-ctl` | Management CLI — create/manage jobs, inspect runs and operations |

---

## ha-sync (runner)

The runner is normally invoked by a Kubernetes CronJob. It is not intended for direct use.

### CLI Flags

| Flag | Required | Default | Description |
|------|----------|---------|-------------|
| `--src` | ✅ | — | Source directory path inside the pod |
| `--dest` | ✅ | — | Destination directory path inside the pod |
| `--pair` | ✅ | — | Logical sync-pair name (e.g. `media`) |
| `--direction` | | `fwd` | Direction label (e.g. `dell-to-hp`) |
| `--src-host` | | `""` | Source host identifier (e.g. `dell`) — recorded in DB |
| `--dest-host` | | `""` | Destination host identifier (e.g. `hp`) — recorded in DB |
| `--db-dsn` | | `$HA_SYNC_DB_DSN` | MySQL DSN |
| `--lock-ttl` | | `3600` | Lease TTL in seconds |
| `--log-dir` | | `/var/log/ha-sync` | Operation log directory |
| `--log-retain-days` | | `10` | Purge logs older than N days |
| `--mtime-threshold` | | `2s` | mtime equality tolerance |
| `--delete-missing` | | `false` | Delete files at dest absent from src |
| `--workers` | | `4` | Concurrent file-copy workers |
| `--dry-run` | | `false` | Preview only — no files touched |
| `--verbose` | | `false` | Verbose log output |

### Environment Variables

| Variable | Description |
|----------|-------------|
| `HA_SYNC_DB_DSN` | MySQL DSN when `--db-dsn` is not set |

---

## ha-sync-ctl (management CLI)

The management CLI controls jobs, inspects runs, and drills into per-file operations.

### Environment Variables

| Variable | Description |
|----------|-------------|
| `HA_SYNC_DB_DSN` | MySQL DSN (required for all DB commands) |
| `HA_SYNC_NAMESPACE` | K8s namespace (default: `infrastructure`) |
| `KUBECONFIG` | Path to kubeconfig (falls back to in-cluster) |

### Global Flags

| Flag | Description |
|------|-------------|
| `--output=json` | Output results as JSON instead of table |

---

### Job Commands

#### `ha-sync-ctl jobs list`

List all sync jobs with their last run status and file counts.

```
NAME              PAIR       DIRECTION    ENABLED  SCHEDULE       LAST RUN        STATUS
media-d2h         media      dell→hp      yes      0 2 * * *      2h ago          ok
media-h2d         media      hp→dell      yes      0 3 * * *      2h ago          ok
games-d2h         games      dell→hp      no       0 4 * * *      5d ago          disabled
```

#### `ha-sync-ctl jobs create`

Create a new sync job and apply its CronJob to K8s.

```bash
ha-sync-ctl jobs create \
  --name=media-dell-to-hp \
  --pair=media \
  --direction=dell-to-hp \
  --src=/mnt/dell/media \
  --dest=/mnt/hp/media \
  --src-host=dell \
  --dest-host=hp \
  --schedule="0 2 * * *" \
  --workers=4 \
  --lock-ttl=86400
```

| Flag | Default | Description |
|------|---------|-------------|
| `--name` | required | Unique job name |
| `--pair` | required | One of: `media`, `photos`, `owncloud`, `games`, `infra`, `ai` |
| `--direction` | required | `dell-to-hp` or `hp-to-dell` |
| `--src` | required | Source path |
| `--dest` | required | Destination path |
| `--src-host` | `""` | Source host label |
| `--dest-host` | `""` | Destination host label |
| `--schedule` | `0 2 * * *` | Cron schedule |
| `--lock-ttl` | `3600` | Lock TTL in seconds |
| `--workers` | `4` | Parallel workers |
| `--dry-run` | `false` | Start in dry-run mode |
| `--delete-missing` | `false` | Delete files at dest not in src |
| `--excludes` | `""` | Comma-separated glob patterns to exclude |

#### `ha-sync-ctl jobs edit <name-or-id>`

Update one or more fields of an existing job. Only flags you pass are updated.

```bash
ha-sync-ctl jobs edit media-dell-to-hp --schedule="0 3 * * *" --workers=8
```

#### `ha-sync-ctl jobs delete <name-or-id>`

Delete a job from the DB and remove its CronJob from K8s.

```bash
ha-sync-ctl jobs delete media-dell-to-hp
```

#### `ha-sync-ctl jobs enable <name-or-id>`

Enable a job (unsuspends the K8s CronJob).

```bash
ha-sync-ctl jobs enable games-dell-to-hp
```

#### `ha-sync-ctl jobs disable <name-or-id>`

Disable a job (suspends the K8s CronJob — no new runs scheduled).

```bash
ha-sync-ctl jobs disable games-dell-to-hp
```

#### `ha-sync-ctl jobs trigger <name-or-id>`

Trigger an immediate run by creating a K8s Job from the CronJob.

```bash
ha-sync-ctl jobs trigger media-dell-to-hp
```

#### `ha-sync-ctl jobs lock-status <name-or-id>`

Show whether the job is currently running (locked via K8s Lease).

```bash
ha-sync-ctl jobs lock-status media-dell-to-hp
# locked=true  holder=ha-sync-media-d2h-abc  expires_at=2026-04-09T03:15:00Z
```

#### `ha-sync-ctl jobs apply-all`

Apply (create or update) K8s CronJobs for all enabled jobs in the DB. Safe to run multiple times (idempotent).

```bash
ha-sync-ctl jobs apply-all
# Applied 10 CronJobs
```

#### `ha-sync-ctl jobs import-k8s`

Import existing K8s CronJobs (labelled `app=ha-sync`) into the DB. Skips jobs already present (idempotent).

```bash
ha-sync-ctl jobs import-k8s
# Imported: 10  Already existed: 0
```

---

### Run Commands

#### `ha-sync-ctl runs list [--job=<name>] [--limit=N]`

List recent sync runs.

```
RUN ID    JOB           STARTED              DURATION   COPIED  DELETED  SKIPPED  ERRORS
abc123    media-d2h     2026-04-09 02:00     4m32s      142     0        891      0
def456    photos-d2h    2026-04-09 01:00     1m15s      0       0        3421     0
```

#### `ha-sync-ctl runs show <run-id>`

Show detail for a single run including per-action counts.

---

### Operation Commands

#### `ha-sync-ctl ops list <run-id> [--limit=N] [--action=copy|delete|skip|error]`

List per-file operations for a run.

```
ACTION   PATH                               FROM→TO    SIZE     OWNER
copy     /media/movies/film.mkv             dell→hp    4.2 GB   dan
copy     /media/music/album/track.flac      dell→hp    32 MB    dan
skip     /media/photos/img001.jpg           dell→hp    3.1 MB   dan
delete   /media/old/deleted.mp4             dell→hp    800 MB   dan
```

---

## Migration: Static CronJobs → DB-managed

The old `cron-*.yaml` manifests are archived in `deployment/ha-sync/archive/`. Run this **once** after deploying the new image:

```bash
# 1. Import existing K8s CronJobs into DB
ha-sync-ctl jobs import-k8s

# 2. Delete old static CronJobs from the cluster
kubectl delete cronjobs -n infrastructure \
  ha-sync-ai-dell-to-hp ha-sync-ai-hp-to-dell \
  ha-sync-games-dell-to-hp ha-sync-games-hp-to-dell \
  ha-sync-infra-dell-to-hp ha-sync-infra-hp-to-dell \
  ha-sync-media-dell-to-hp ha-sync-media-hp-to-dell \
  ha-sync-owncloud-dell-to-hp ha-sync-owncloud-hp-to-dell \
  ha-sync-photos-dell-to-hp ha-sync-photos-hp-to-dell

# 3. Apply DB-managed CronJobs
ha-sync-ctl jobs apply-all
```

---

## Delete Safety Warning

> ⚠️ `--delete-missing` defaults to **false**.

Enabling `--delete-missing` on both directions simultaneously risks data loss if one node is temporarily unavailable or partially mounted.

**Recommended practice:**

1. Complete an initial full sync without `--delete-missing` and verify parity via the UI dashboard.
2. Enable `--delete-missing` **only on the primary direction** (e.g. `dell-to-hp`) once parity is confirmed.
3. Leave the reverse direction without `--delete-missing` unless you understand the implications.

---

## UI Dashboard

**<https://ha-sync.vandachevici.ro>**

3-view SPA showing:
- **Jobs grid** — all jobs at a glance: status, last run, file counts, lock indicator, enable/disable toggle, Run Now button
- **Job detail** — configuration + full run history; click a run to drill in
- **Operations drawer** — per-file breakdown: action, path, from→to host, size, owner; filterable by action type

Auto-refreshes every **30 seconds**.