dan/homelab
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
homelab/scripts/cli/ha-sync.md
Dan V deb6c38d7b chore: commit homelab setup — deployment, services, orchestration, skill 
- Add .gitignore: exclude compiled binaries, build artifacts, and Helm
  values files containing real secrets (authentik, prometheus)
- Add all Kubernetes deployment manifests (deployment/)
- Add services source code: ha-sync, device-inventory, games-console,
  paperclip, parts-inventory
- Add Ansible orchestration: playbooks, roles, inventory, cloud-init
- Add hardware specs, execution plans, scripts, HOMELAB.md
- Add skills/homelab/SKILL.md + skills/install.sh to preserve Copilot skill
- Remove previously-tracked inventory-cli binary from git index

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-09 08:10:32 +02:00

270 lines
8.3 KiB
Markdown
Raw Blame History

# ha-sync
## Overview
`ha-sync` performs bidirectional file synchronisation between the **Dell node** (`192.168.2.100`) and the **HP node** (`192.168.2.193`) over NFS-mounted volumes. Each sync run is recorded to MySQL, and runs are scheduled as Kubernetes CronJobs in the `infrastructure` namespace.
A distributed Kubernetes Lease (`ha-sync-<jobname>`) prevents concurrent runs of the same job, and an optional dry-run mode lets you preview what _would_ happen without touching any files.
**CronJobs are now managed by `ha-sync-ctl` and the DB** — static YAML manifests have been archived. Do not apply `cron-*.yaml` manually.
---
## Tools
| Binary | Purpose |
|--------|---------|
| `ha-sync` | Sync runner — invoked by K8s CronJobs |
| `ha-sync-ctl` | Management CLI — create/manage jobs, inspect runs and operations |
---
## ha-sync (runner)
The runner is normally invoked by a Kubernetes CronJob. It is not intended for direct use.
### CLI Flags
| Flag | Required | Default | Description |
|------|----------|---------|-------------|
| `--src` | ✅ | — | Source directory path inside the pod |
| `--dest` | ✅ | — | Destination directory path inside the pod |
| `--pair` | ✅ | — | Logical sync-pair name (e.g. `media`) |
| `--direction` | | `fwd` | Direction label (e.g. `dell-to-hp`) |
| `--src-host` | | `""` | Source host identifier (e.g. `dell`) — recorded in DB |
| `--dest-host` | | `""` | Destination host identifier (e.g. `hp`) — recorded in DB |
| `--db-dsn` | | `$HA_SYNC_DB_DSN` | MySQL DSN |
| `--lock-ttl` | | `3600` | Lease TTL in seconds |
| `--log-dir` | | `/var/log/ha-sync` | Operation log directory |
| `--log-retain-days` | | `10` | Purge logs older than N days |
| `--mtime-threshold` | | `2s` | mtime equality tolerance |
| `--delete-missing` | | `false` | Delete files at dest absent from src |
| `--workers` | | `4` | Concurrent file-copy workers |
| `--dry-run` | | `false` | Preview only — no files touched |
| `--verbose` | | `false` | Verbose log output |
### Environment Variables
| Variable | Description |
|----------|-------------|
| `HA_SYNC_DB_DSN` | MySQL DSN when `--db-dsn` is not set |
---
## ha-sync-ctl (management CLI)
The management CLI controls jobs, inspects runs, and drills into per-file operations.
### Environment Variables
| Variable | Description |
|----------|-------------|
| `HA_SYNC_DB_DSN` | MySQL DSN (required for all DB commands) |
| `HA_SYNC_NAMESPACE` | K8s namespace (default: `infrastructure`) |
| `KUBECONFIG` | Path to kubeconfig (falls back to in-cluster) |
### Global Flags
| Flag | Description |
|------|-------------|
| `--output=json` | Output results as JSON instead of table |
---
### Job Commands
#### `ha-sync-ctl jobs list`
List all sync jobs with their last run status and file counts.
```
NAME PAIR DIRECTION ENABLED SCHEDULE LAST RUN STATUS
media-d2h media dell→hp yes 0 2 * * * 2h ago ok
media-h2d media hp→dell yes 0 3 * * * 2h ago ok
games-d2h games dell→hp no 0 4 * * * 5d ago disabled
```
#### `ha-sync-ctl jobs create`
Create a new sync job and apply its CronJob to K8s.
```bash
ha-sync-ctl jobs create \
--name=media-dell-to-hp \
--pair=media \
--direction=dell-to-hp \
--src=/mnt/dell/media \
--dest=/mnt/hp/media \
--src-host=dell \
--dest-host=hp \
--schedule="0 2 * * *" \
--workers=4 \
--lock-ttl=86400
```
| Flag | Default | Description |
|------|---------|-------------|
| `--name` | required | Unique job name |
| `--pair` | required | One of: `media`, `photos`, `owncloud`, `games`, `infra`, `ai` |
| `--direction` | required | `dell-to-hp` or `hp-to-dell` |
| `--src` | required | Source path |
| `--dest` | required | Destination path |
| `--src-host` | `""` | Source host label |
| `--dest-host` | `""` | Destination host label |
| `--schedule` | `0 2 * * *` | Cron schedule |
| `--lock-ttl` | `3600` | Lock TTL in seconds |
| `--workers` | `4` | Parallel workers |
| `--dry-run` | `false` | Start in dry-run mode |
| `--delete-missing` | `false` | Delete files at dest not in src |
| `--excludes` | `""` | Comma-separated glob patterns to exclude |
#### `ha-sync-ctl jobs edit <name-or-id>`
Update one or more fields of an existing job. Only flags you pass are updated.
```bash
ha-sync-ctl jobs edit media-dell-to-hp --schedule="0 3 * * *" --workers=8
```
#### `ha-sync-ctl jobs delete <name-or-id>`
Delete a job from the DB and remove its CronJob from K8s.
```bash
ha-sync-ctl jobs delete media-dell-to-hp
```
#### `ha-sync-ctl jobs enable <name-or-id>`
Enable a job (unsuspends the K8s CronJob).
```bash
ha-sync-ctl jobs enable games-dell-to-hp
```
#### `ha-sync-ctl jobs disable <name-or-id>`
Disable a job (suspends the K8s CronJob — no new runs scheduled).
```bash
ha-sync-ctl jobs disable games-dell-to-hp
```
#### `ha-sync-ctl jobs trigger <name-or-id>`
Trigger an immediate run by creating a K8s Job from the CronJob.
```bash
ha-sync-ctl jobs trigger media-dell-to-hp
```
#### `ha-sync-ctl jobs lock-status <name-or-id>`
Show whether the job is currently running (locked via K8s Lease).
```bash
ha-sync-ctl jobs lock-status media-dell-to-hp
# locked=true holder=ha-sync-media-d2h-abc expires_at=2026-04-09T03:15:00Z
```
#### `ha-sync-ctl jobs apply-all`
Apply (create or update) K8s CronJobs for all enabled jobs in the DB. Safe to run multiple times (idempotent).
```bash
ha-sync-ctl jobs apply-all
# Applied 10 CronJobs
```
#### `ha-sync-ctl jobs import-k8s`
Import existing K8s CronJobs (labelled `app=ha-sync`) into the DB. Skips jobs already present (idempotent).
```bash
ha-sync-ctl jobs import-k8s
# Imported: 10 Already existed: 0
```
---
### Run Commands
#### `ha-sync-ctl runs list [--job=<name>] [--limit=N]`
List recent sync runs.
```
RUN ID JOB STARTED DURATION COPIED DELETED SKIPPED ERRORS
abc123 media-d2h 2026-04-09 02:00 4m32s 142 0 891 0
def456 photos-d2h 2026-04-09 01:00 1m15s 0 0 3421 0
```
#### `ha-sync-ctl runs show <run-id>`
Show detail for a single run including per-action counts.
---
### Operation Commands
#### `ha-sync-ctl ops list <run-id> [--limit=N] [--action=copy|delete|skip|error]`
List per-file operations for a run.
```
ACTION PATH FROM→TO SIZE OWNER
copy /media/movies/film.mkv dell→hp 4.2 GB dan
copy /media/music/album/track.flac dell→hp 32 MB dan
skip /media/photos/img001.jpg dell→hp 3.1 MB dan
delete /media/old/deleted.mp4 dell→hp 800 MB dan
```
---
## Migration: Static CronJobs → DB-managed
The old `cron-*.yaml` manifests are archived in `deployment/ha-sync/archive/`. Run this **once** after deploying the new image:
```bash
# 1. Import existing K8s CronJobs into DB
ha-sync-ctl jobs import-k8s
# 2. Delete old static CronJobs from the cluster
kubectl delete cronjobs -n infrastructure \
ha-sync-ai-dell-to-hp ha-sync-ai-hp-to-dell \
ha-sync-games-dell-to-hp ha-sync-games-hp-to-dell \
ha-sync-infra-dell-to-hp ha-sync-infra-hp-to-dell \
ha-sync-media-dell-to-hp ha-sync-media-hp-to-dell \
ha-sync-owncloud-dell-to-hp ha-sync-owncloud-hp-to-dell \
ha-sync-photos-dell-to-hp ha-sync-photos-hp-to-dell
# 3. Apply DB-managed CronJobs
ha-sync-ctl jobs apply-all
```
---
## Delete Safety Warning
> ⚠️ `--delete-missing` defaults to **false**.
Enabling `--delete-missing` on both directions simultaneously risks data loss if one node is temporarily unavailable or partially mounted.
**Recommended practice:**
1. Complete an initial full sync without `--delete-missing` and verify parity via the UI dashboard.
2. Enable `--delete-missing` **only on the primary direction** (e.g. `dell-to-hp`) once parity is confirmed.
3. Leave the reverse direction without `--delete-missing` unless you understand the implications.
---
## UI Dashboard
**<https://ha-sync.vandachevici.ro>**
3-view SPA showing:
- **Jobs grid** — all jobs at a glance: status, last run, file counts, lock indicator, enable/disable toggle, Run Now button
- **Job detail** — configuration + full run history; click a run to drill in
- **Operations drawer** — per-file breakdown: action, path, from→to host, size, owner; filterable by action type
Auto-refreshes every **30 seconds**.
Reference in a new issue View git blame Copy permalink