Compare commits

...
Author SHA1 Message Date
cayce d7156358f6 backup: RESTORE.md runbook + E2E restore test evidence (Tanda B, point 3)
ci / lint-test-leaks (pull_request) Canceled after 0s
RESTORE.md documents the exact from-zero recovery on a fresh VPS:
download+verify (sha256 over encrypted artifacts, no private key needed
on the pull side), decrypt on the operator Mac, Postgres pg_restore,
Gitea unzip+migrate, Caddy/systemd/.env config restore, repo re-clone,
backup pipeline reinstall with the SAME age recipient, and a
post-restore checklist.

E2E test executed on the VPS (no production touched — cayce-ops has no
production DB; a probe DB was created and dropped):
- manifest.sha256: 3/3 artifacts OK (verified WITHOUT any private key)
- age decrypt roundtrip with a throwaway keypair (destroyed afterwards)
- pg_restore TOC valid; full restore into a scratch DB: 5000/5000 rows
- gitea zip valid, config tar valid (34 entries)
- all test material removed: probe DB dropped, throwaway key deleted,
  recipient file removed, test backup deleted (was encrypted to the
  destroyed throwaway key)
2026-10-28 21:07:58 +00:00
cayce 508371184d Merge PR #14: usuario backup-pull de solo lectura
ci / lint-test-leaks (push) Canceled after 0s
2026-10-28 21:01:50 +00:00
cayce fc2dcd4fca backup: backup-pull user — read-only rsync-only SSH access (Tanda B, point 2)
ci / lint-test-leaks (pull_request) Canceled after 0s
System user backup-pull created on the VPS (no sudo, locked password).
authorized_keys restricted with restrict + forced command
rrsync -ro /var/backups/cayce. Verified with a since-deleted throwaway
key: no shell, no arbitrary commands, no sftp/scp, no reads outside
/var/backups/cayce, no writes; the positive rsync pull works.

authorized_keys is left EMPTY — the operator adds their real key with
the exact line documented in deploy/backup/backup-pull-setup.md.
2026-10-28 21:01:44 +00:00
cayce 121183ac83 Merge PR #13: generación de backups cifrados
ci / lint-test-leaks (push) Canceled after 0s
2026-10-28 20:59:35 +00:00
cayce b87a51afc0 backup: encrypted daily backups — VPS generates, Mac pulls (Tanda B, point 1)
ci / lint-test-leaks (pull_request) Canceled after 0s
deploy/backup/make-backup.sh (installed as /usr/local/bin/cayce-make-backup):
- pg_dump custom-format of the cayce-ops DB (URL from env or .env,
  SQLAlchemy driver qualifier stripped, credentials never logged)
- gitea dump (repos + sqlite + attachments), live; consistency notes in RESTORE.md
- config tar: Caddyfile(+d), gitea app.ini + internal_token, cayce-* units, .env
- EACH artifact encrypted with age BEFORE leaving the tmpdir; recipient
  read from /etc/cayce-backup/age-recipient.pub (public key only). The
  private age key never touches this server.
- plaintext exists ONLY in a mktemp dir removed by an EXIT trap; a partial
  day directory is removed on failure; any step failure => exit != 0 =>
  journald entry
- manifest.sha256 over the encrypted artifacts (pull side verifies
  integrity WITHOUT the age private key)
- destination /var/backups/cayce/YYYY-MM-DD/ (750 root:backup-pull, 640)
- retention: 7 daily dirs, pruned by mtime

systemd: cayce-backup.service (oneshot, hardened: ProtectSystem=strict,
NoNewPrivileges, ProtectHome=read-only) + cayce-backup.timer (daily
03:15 UTC, Persistent). Timer ENABLED on the VPS per operator instruction;
runs fail loudly (exit 1, clear journald message) until the age recipient
is provided — that is the designed not-configured state.
2026-10-28 20:59:28 +00:00
cayce d527164434 Merge PR #12: identidad sin afirmación de jurisdicción
ci / lint-test-leaks (push) Canceled after 0s
2026-10-28 20:01:09 +00:00
cayce 1c4ba234d0 identity: remove unverified jurisdiction claim, state the approval principle
ci / lint-test-leaks (pull_request) Canceled after 0s
The previous text asserted participation in an external jurisdiction
pilot without a verifiable source. Replaced with what is actually true
of this codebase: every external-effect action requires cryptographically
signed human approval, with a key that never resides on Cayce
infrastructure (D16).
2026-10-28 20:01:02 +00:00
cayce 64f89df289 Merge PR #11: clasificador con negaciones en español
ci / lint-test-leaks (push) Canceled after 0s
2026-10-28 19:59:50 +00:00
cayce f5b026070b audit-4: classifier understands Spanish negations (veto unchanged)
ci / lint-test-leaks (pull_request) Canceled after 0s
Merchants write in Spanish; negations like nunca, lamentablemente or
no podemos must veto approval exactly like their English counterparts.
Previously an all-Spanish rejection mail could fall through to OTHER,
and a mixed mail with the approval word in English approved despite a
Spanish negation.

- negation list adds: nunca, jamas, tampoco, ningun/ninguna, nadie,
  nada, no podemos, no es posible, no aceptamos, rechaz*, descartad*,
  lamentablemente, no ha sido aprobad*, no procede
- new Spanish REJECTION rule (rechaz*, solicitud denegada, no cumple
  los requisitos) and Spanish APPROVAL rule (aprob*, aceptad*,
  bienvenid*, encantados de trabajar)
- veto rule unchanged: any negation anywhere -> needs_review
- tests: 15 new (12 veto cases incl. 2 mixed-language, pure-Spanish
  approval still approves, Spanish rejection classifies rejection)

Gate: ruff 0 · 96 unit + 1 PG integration · gitleaks clean.
2026-10-28 19:59:44 +00:00
cayce 60300a2f1a Merge PR #10: cadena de auditoría a prueba de dueño
ci / lint-test-leaks (push) Canceled after 0s
2026-10-28 19:56:57 +00:00
cayce 09fe9199b3 audit-3: append-only audit log + strict legacy-prefix rule + anchor cross-check
ci / lint-test-leaks (pull_request) Canceled after 0s
- verify_audit_chain(): rows with entry_hash IS NULL are only tolerated
  BEFORE the first hashed row (documented legacy prefix). A NULL hash
  AFTER the chain started is tampering and fails verification at that
  row id — previously any NULL row anywhere was silently skipped, so an
  attacker with UPDATE could erase intermediate evidence undetected.
- migration 0004: audit_log becomes append-only. SQLite: BEFORE UPDATE/
  DELETE triggers RAISE(ABORT). Postgres: BEFORE triggers RAISE EXCEPTION.
  Closes the hole for the table owner and direct SQL connections.
- monitor.verify_chain_against_anchors(): compares the chain tail against
  anchors published to Nostr. Explicit statuses: no_anchors (never
  anchored / notes missing on relays), local_only (no fetcher), ok
  (relay note attests the tail or the last local anchor), mismatch.
- tests: NULL-before-genesis ok, NULL-in-middle fails at that id, NULL
  late fails, UPDATE rejected, DELETE rejected, INSERT still allowed,
  no-anchors explicit, local-only explicit, mismatch detected, match ok.

Gate: ruff 0 · 81 unit + 1 PG integration · alembic 0001→0004 on PG ·
gitleaks clean · make check exit 0.
2026-10-28 19:56:50 +00:00
cayce 029e4f77d7 Merge PR #9: D16 — desacoplar clave de operador de identidad de agente
ci / lint-test-leaks (push) Canceled after 0s
2026-10-28 19:51:21 +00:00
12 changed files with 847 additions and 11 deletions
+8 -5
View File
@@ -1,10 +1,13 @@
# Cayce Ops
Cayce Ops is the autonomous operations stack of **Cayce** — an autonomous entity
operating under the Próspera ZEDE agentic-companies jurisdiction pilot. Cayce runs
[bitcoinhotels.org](https://bitcoinhotels.org) (Bitcoin-native travel guides) and
[hardcap.news](https://hardcap.news) (Bitcoin security intelligence). The
experiment is managed by Cayce itself.
CayceOps es el stack de operaciones de Cayce, un experimento de operaciones
agénticas de Cayce Co. Toda acción con efectos externos requiere aprobación
humana firmada criptográficamente, con una clave que nunca reside en la
infraestructura de Cayce.
Cayce operates [bitcoinhotels.org](https://bitcoinhotels.org)
(Bitcoin-native travel guides) and [hardcap.news](https://hardcap.news)
(Bitcoin security intelligence).
Five decoupled components share one Postgres database. Each is a CLI command,
driven by cron/systemd timers. Nothing runs without an audit trail.
@@ -0,0 +1,89 @@
"""audit_log append-only: reject UPDATE and DELETE (round-2 hardening)
Revision ID: 0004
Revises: 0003
Create Date: 2026-10-28
An audit trail that can be edited is not an audit trail. This migration
installs, on every engine:
- SQLite: BEFORE UPDATE/DELETE triggers that RAISE(ABORT).
- Postgres: BEFORE UPDATE/DELETE triggers that RAISE EXCEPTION.
The writer roles never had UPDATE/DELETE grants in roles.sql; this closes
the hole for the table owner and for direct SQL connections.
"""
from __future__ import annotations
from alembic import op
revision = "0004"
down_revision = "0003"
branch_labels = None
depends_on = None
SQLITE_TRIGGERS = [
"""
CREATE TRIGGER IF NOT EXISTS audit_log_no_update
BEFORE UPDATE ON audit_log
BEGIN
SELECT RAISE(ABORT, 'cayce-ops: audit_log is append-only (UPDATE rejected)');
END;
""",
"""
CREATE TRIGGER IF NOT EXISTS audit_log_no_delete
BEFORE DELETE ON audit_log
BEGIN
SELECT RAISE(ABORT, 'cayce-ops: audit_log is append-only (DELETE rejected)');
END;
""",
]
PG_TRIGGERS = """
CREATE OR REPLACE FUNCTION cayce_ops_audit_append_only() RETURNS trigger AS $$
BEGIN
RAISE EXCEPTION 'cayce-ops: audit_log is append-only (% rejected)', TG_OP;
END;
$$ LANGUAGE plpgsql;
"""
def upgrade() -> None:
bind = op.get_bind()
if bind.dialect.name == "postgresql":
op.execute(PG_TRIGGERS)
op.execute(
"DROP TRIGGER IF EXISTS audit_log_no_update ON audit_log;"
)
op.execute(
"CREATE TRIGGER audit_log_no_update BEFORE UPDATE ON audit_log "
"FOR EACH ROW EXECUTE FUNCTION cayce_ops_audit_append_only();"
)
op.execute(
"DROP TRIGGER IF EXISTS audit_log_no_delete ON audit_log;"
)
op.execute(
"CREATE TRIGGER audit_log_no_delete BEFORE DELETE ON audit_log "
"FOR EACH ROW EXECUTE FUNCTION cayce_ops_audit_append_only();"
)
else:
for ddl in SQLITE_TRIGGERS:
bind.execute(sa_text(ddl))
def sa_text(sql: str):
import sqlalchemy as sa
return sa.text(sql)
def downgrade() -> None:
bind = op.get_bind()
if bind.dialect.name == "postgresql":
op.execute("DROP TRIGGER IF EXISTS audit_log_no_update ON audit_log;")
op.execute("DROP TRIGGER IF EXISTS audit_log_no_delete ON audit_log;")
op.execute("DROP FUNCTION IF EXISTS cayce_ops_audit_append_only();")
else:
bind.execute(sa_text("DROP TRIGGER IF EXISTS audit_log_no_update;"))
bind.execute(sa_text("DROP TRIGGER IF EXISTS audit_log_no_delete;"))
+22 -2
View File
@@ -19,10 +19,17 @@ from cayceops.models import MailClassification
_APPROVAL_WORDS = re.compile(r"\b(approv|accept|welcome)", re.I)
# any of these tokens anywhere in the mail vetoes an `approval` result
# any of these tokens anywhere in the mail vetoes an `approval` result.
# English and Spanish (merchants write in both; mixed mails are common).
_NEGATIONS = re.compile(
r"\b(not|no|never|cannot|can'?t|won'?t|hasn'?t|isn'?t|"
r"no\s+longer|unable|decline\w*|regret\w*|unfortunately)\b",
r"no\s+longer|unable|decline\w*|regret\w*|unfortunately|"
# spanish: standalone negation words
r"nunca|jam[aá]s|tampoco|ning[uú]n(?:a|o)?|nadie|nada|"
# spanish: multi-word negations and rejection stems
r"no\s+podemos|no\s+es\s+posible|no\s+acept(?:amos|ar[aá])|"
r"rechaz\w*|lamentablemente|descartad\w*|"
r"no\s+ha\s+sido\s+aprobad\w*|no\s+procede)\b",
re.I,
)
@@ -34,11 +41,24 @@ _RULES: list[tuple[MailClassification, re.Pattern[str], float]] = [
re.compile(r"\b(declin|reject|not able to|unable to|we regret|won.t be able)\w*", re.I), # noqa: E501
0.9,
),
(
MailClassification.REJECTION,
# spanish rejections (strong stems, not just negation words)
re.compile(r"\b(rechaz\w*|no\s+vamos\s+a\s+pod(?:er|emos)|su\s+solicitud\s+ha\s+sido\s+denegada|no\s+cumple\s+los\s+requisitos)\b", re.I), # noqa: E501
0.9,
),
(
MailClassification.APPROVAL,
re.compile(r"\b(approv|accept|welcome to the|you.re (in|approved)|live|activated|glad to)\w*", re.I), # noqa: E501
0.85,
),
(
MailClassification.APPROVAL,
# spanish approvals (standalone positive stems; the veto still
# downgrades if ANY negation appears elsewhere in the mail)
re.compile(r"\b(aprob\w*|aceptad\w*|bienvenid\w*|ha\s+sido\s+aceptada|encantados?\s+de\s+(trabajar|colaborar))\b", re.I), # noqa: E501
0.85,
),
(
MailClassification.QUESTION,
re.compile(r"\b(could you|can you|please (provide|clarify|confirm)|what (is|are)|how (do|does|can)|need more (info|details))\b", re.I), # noqa: E501
+117
View File
@@ -251,3 +251,120 @@ def build(cfg: Config | None = None) -> int:
return 0
finally:
engine().dispose()
def _latest_chain_hash(sess) -> str | None:
"""Last non-NULL entry_hash in the audit chain (None if chain empty)."""
from cayceops.models import AuditEntry
last = (
sess.query(AuditEntry)
.filter(AuditEntry.entry_hash.isnot(None))
.order_by(AuditEntry.id.desc())
.first()
)
return last.entry_hash if last else None
def _published_anchors(sess) -> list[dict]:
"""Anchor audit entries recorded locally, oldest first."""
from cayceops.models import AuditEntry
rows = (
sess.query(AuditEntry)
.filter(AuditEntry.action == "anchor")
.order_by(AuditEntry.id.asc())
.all()
)
return [
{
"event_id": (r.detail or {}).get("event_id"),
"last_hash": (r.detail or {}).get("last_hash"),
"last_entry": (r.detail or {}).get("last_entry"),
"ts": r.ts.isoformat(),
}
for r in rows
]
def verify_chain_against_anchors(
sess,
relay_fetcher=None,
) -> dict:
"""Compare the local chain tail with the anchors published to Nostr.
relay_fetcher(deployment) -> list of note contents (strings) found on
the relays for this deployment's anchor tag. When None, only local
coherence is checked and the result says so.
Returns a report dict:
status: "no_anchors" | "local_only" | "ok" | "mismatch"
detail: human-readable explanation
local_last_hash, local_anchors, relay_notes (when fetched)
"""
local_hash = _latest_chain_hash(sess)
anchors = _published_anchors(sess)
report: dict = {
"local_last_hash": local_hash,
"local_anchors": anchors,
"relay_notes": None,
}
if not anchors:
report["status"] = "no_anchors"
report["detail"] = (
"the chain has never been anchored to Nostr — no external "
"attestation exists yet (anchor once with `monitor anchor`)"
)
return report
last_anchor = anchors[-1]
if relay_fetcher is None:
report["status"] = "local_only"
report["detail"] = (
"compared only against the LOCAL record of past anchors "
"(no relay fetch configured); last anchored hash "
f"{last_anchor['last_hash']}"
)
return report
from cayceops.config import load as _load
deployment = _load().deployment_id
notes = relay_fetcher(deployment) or []
report["relay_notes"] = notes
if not notes:
report["status"] = "no_anchors"
report["detail"] = (
"local anchor records exist but no anchor notes were found on "
"the relays — the external attestation is missing or was deleted"
)
return report
# the freshest relay note must attest the current chain tail
expected_fragment = f"last_hash={local_hash}"
latest_note = notes[-1]
if expected_fragment in latest_note:
report["status"] = "ok"
report["detail"] = "latest relay anchor matches the current chain tail"
return report
# maybe the chain advanced after the last anchor — that's fine ONLY if
# the note matches the hash recorded in the last local anchor entry
if last_anchor["last_hash"] and last_anchor["last_hash"] in latest_note:
report["status"] = "ok"
report["detail"] = (
"relay anchor matches the last local anchor; the chain has "
"advanced since (anchor again to attest the new tail)"
)
return report
report["status"] = "mismatch"
report["detail"] = (
f"relay anchor does not attest chain tail {local_hash} nor the last "
f"local anchor {last_anchor['last_hash']} — the published anchor "
"does not correspond to this database"
)
return report
+11 -4
View File
@@ -209,14 +209,21 @@ def _build_entry(
def verify_audit_chain(sess) -> tuple[bool, int | None]:
"""Re-walk the chain. Returns (ok, first_broken_entry_id or None).
Legacy rows (entry_hash IS NULL, written before migration 0002) are
skipped and documented: the chain begins GENESIS at the first hashed row,
so a migrated database verifies clean without any tampering.
Rows with entry_hash IS NULL are ONLY tolerated BEFORE the first hashed
row (the documented legacy prefix from before migration 0002). A NULL
hash AFTER the chain has started is tampering — an attacker erasing the
hash of an intermediate row to hide an edit — and fails verification at
that row's id.
"""
prev_hash: str | None = None # None = still in legacy prefix
chain_started = False
for entry in sess.query(AuditEntry).order_by(AuditEntry.id).all():
if entry.entry_hash is None:
continue # legacy row: outside the chain by documented genesis
if chain_started:
# tampering: a hash was erased in the middle of the chain
return False, entry.id
continue # legacy prefix row: outside the chain by genesis
chain_started = True
if prev_hash is None:
expected_prev = "GENESIS" # first hashed row anchors the chain
else:
+145
View File
@@ -0,0 +1,145 @@
# RESTORE.md — Restauración de CayceOps desde backups cifrados
**Modelo:** la VPS genera backups cifrados (age) en `/var/backups/cayce/YYYY-MM-DD/`;
tu Mac los descarga con el usuario `backup-pull` (solo lectura). La clave
**privada** de age vive SOLO en tu Mac. Este runbook asume una VPS nueva
(Ubuntu + IP apuntada por DNS) y tus backups descifrados a mano.
---
## 0. Descargar y descifrar (en TU Mac — nunca en la VPS)
```bash
# pull (clave restringida: solo lectura de /var/backups/cayce)
rsync -av --partial backup-pull@15.235.3.239:/ ~/backups/cayce/
# integridad (sin clave privada; manifest lista los artefactos CIFRADOS)
cd ~/backups/cayce/2026-XX-XX && shasum -a 256 -c manifest.sha256
# (Linux: sha256sum -c manifest.sha256)
# descifrar (clave privada age del operador — la que NO está en la VPS)
age -d -i ~/keys/cayce-backup.txt -o cayce-ops-db.dump cayce-ops-db.dump.age
age -d -i ~/keys/cayce-backup.txt -o gitea-dump.zip gitea-dump.zip.age
age -d -i ~/keys/cayce-backup.txt -o config.tar.gz config.tar.gz.age
```
Artefactos:
| Archivo | Contenido | Formato |
|---|---|---|
| `cayce-ops-db.dump` | BD de CayceOps | `pg_dump` custom (restaurar con `pg_restore`) |
| `gitea-dump.zip` | repos + sqlite + adjuntos + config Gitea | zip de `gitea dump` |
| `config.tar.gz` | `caddy/` (Caddyfile+Caddyfile.d), `gitea/` (app.ini + internal_token), `systemd/` (unidades cayce-*), `cayce-ops/dot-env` | tar.gz |
---
## 1. VPS nueva — base del sistema
```bash
apt-get update
apt-get install -y postgresql age rsync git
# Gitea: misma versión mayor que la VPS origen (verSTEP: gitea --version en el viejo
# servidor ANTES de apagarlo, o lee la versión del zip gitea-dump: gitea-dump zip
# contiene un archivo "app.ini" con la versión en comentarios... anota la versión
# manualmente; el binario se baja de https://dl.gitea.com/gitea/)
adduser --system --group --home /var/lib/gitea git
mkdir -p /etc/gitea /var/lib/gitea /etc/cayce-backup
curl -L "https://dl.gitea.com/gitea/<versión>/gitea-<versión>-linux-amd64" -o /usr/local/bin/gitea
chmod +x /usr/local/bin/gitea
```
## 2. Postgres
```bash
systemctl enable --now postgresql
sudo -u postgres psql <<SQL
CREATE ROLE cayce LOGIN PASSWORD '<password-del-.env-restaurado>';
CREATE DATABASE cayce_ops OWNER cayce;
SQL
# restaurar el dump (los ficheros descifrados son legibles solo por ti;
# cópiales permisos de lectura para postgres o pásalos por stdin)
sudo -u postgres pg_restore -d cayce_ops --no-owner --no-privileges \
/ruta/cayce-ops-db.dump
# verificación
sudo -u postgres psql -d cayce_ops -tAc \
"SELECT count(*) FROM audit_log;" # debe coincidir con la última ancla
```
## 3. Gitea
```bash
# config restaurada desde config.tar.gz
tar -xzf config.tar.gz -C /tmp/restored
cp /tmp/restored/gitea/app.ini /etc/gitea/app.ini
chown root:git /etc/gitea/app.ini && chmod 660 /etc/gitea/app.ini
cp /tmp/restored/gitea/internal_token /etc/gitea/ && chown git:git /etc/gitea/internal_token
# NOTA sobre ROOT_URL: app.ini restaurado apunta a https://git.cayce-co.xyz/ —
# el DNS de ese subdominio debe existir ANTES de iniciar Gitea o las URLs
# absolutas (webhooks, notificaciones) quedarán mal.
# dump de Gitea: contiene data/, repos/ y custom/. Restaurar SOBRE el dump
# de Gitea exige la misma versión del binario; descomprimir dentro de WORK_PATH
systemctl stop gitea 2>/dev/null || true
unzip -o gitea-dump.zip -d /tmp/gitea-restore
rsync -a /tmp/gitea-restore/data/ /var/lib/gitea/data/
rsync -a /tmp/gitea-restore/repos/ /var/lib/gitea/data/gitea-repositories/ 2>/dev/null || true
rsync -a /tmp/gitea-restore/custom/ /var/lib/gitea/custom/ 2>/dev/null || true
chown -R git:git /var/lib/gitea
sudo -u git /usr/local/bin/gitea migrate -c /etc/gitea/app.ini # ajusta esquema si cambia versión
```
> **Consistencia:** los dumps de Gitea se toman EN CALIENTE (el servicio sigue
> corriendo). Riesgo teórico de inconsistencia sqlite/repos si un push ocurre
> en el mismo instante. Para un backup garantizado-consistente: `systemctl stop
> gitea` antes de `gitea dump` y volver a levantar después (no automatizado a
> propósito: no queremos ventanas de downtime diarias).
## 4. Config del sistema (Caddy, unidades systemd, .env)
```bash
# Caddy
apt-get install -y caddy # o el instalador oficial
rsync -a /tmp/restored/caddy/Caddyfile /etc/caddy/Caddyfile
rsync -a /tmp/restored/caddy/Caddyfile.d/ /etc/caddy/Caddyfile.d/
systemctl enable --now caddy # emitirá certificados si el DNS ya apunta aquí
# unidades systemd de cayce (DESHABILITADAS por diseño — ver riesgos)
rsync -a /tmp/restored/systemd/ /etc/systemd/system/
systemctl daemon-reload
# NO habilitar cayce-ops@*.timer sin el operador
# .env de cayce-ops
install -d -m 755 /home/ubuntu
rsync -a /tmp/restored/cayce-ops/dot-env /home/ubuntu/cayce-ops/.env
chown ubuntu:ubuntu /home/ubuntu/cayce-ops/.env && chmod 660 /home/ubuntu/cayce-ops/.env
```
## 5. Código del repo
```bash
sudo -u ubuntu git clone http://127.0.0.1:3100/cayce/cayce-ops.git /home/ubuntu/cayce-ops \
|| git clone https://git.cayce-co.xyz/cayce/cayce-ops.git /home/ubuntu/cayce-ops
cd /home/ubuntu/cayce-ops && uv sync
make check # gate verde antes de considerar la restauración completa
```
## 6. Reinstalar el pipeline de backup en la VPS nueva
```bash
install -m 700 -o root -g root deploy/backup/make-backup.sh /usr/local/bin/cayce-make-backup
install -m 644 deploy/systemd/cayce-backup.{service,timer} /etc/systemd/system/
systemctl daemon-reload && systemctl enable --now cayce-backup.timer
# la MISMA clave pública age → tu Mac sigue pudiendo descifrar:
echo 'age1<tu-pubkey>' > /etc/cayce-backup/age-recipient.pub
```
## 7. Checklist de verificación post-restauración
- [ ] `sha256sum -c manifest.sha256` OK en tu Mac (antes de descifrar)
- [ ] `pg_restore --list` del dump sin errores
- [ ] `verify_audit_chain(sess)` → `(True, None)` contra la BD restaurada
- [ ] Gitea responde por TLS en `git.cayce-co.xyz` y `git rev-parse main` da el commit esperado
- [ ] `make check` verde en el repo restaurado
- [ ] Primer backup nuevo generado y verificado desde la Mac
+43
View File
@@ -0,0 +1,43 @@
# backup-pull — usuario de solo lectura para el pull desde la Mac
La VPS genera los backups cifrados en `/var/backups/cayce/YYYY-MM-DD/`.
La Mac los descarga por rsync con una clave SSH **restringida**. La VPS
nunca tiene credenciales para escribir ni borrar nada fuera de sí misma.
## Estado instalado en la VPS
- Usuario de sistema `backup-pull` (uid 1001): **sin sudo**, password
bloqueado, home `/home/backup-pull`, shell `/bin/sh` (necesario para que
sshd ejecute el forced command; la restricción real la pone la clave, no
el shell).
- `/home/backup-pull/.ssh/authorized_keys` (600, backup-pull:backup-pull) —
**vacío** hasta que el operador añada su clave pública.
- `/var/backups/cayce` → `root:backup-pull`, `750`; artefactos `640`.
## Formato de la línea authorized_keys (una sola línea)
restrict,command="/usr/bin/rrsync -ro /var/backups/cayce" ssh-ed25519 AAAA... comentario
`restrict` = no-pty, no-port-forwarding, no-agent-forwarding, no-X11-forwarding
(y sin sftp). `rrsync -ro` = solo rsync de LECTURA, confinado a
`/var/backups/cayce`. Añadirla con:
echo 'restrict,command="/usr/bin/rrsync -ro /var/backups/cayce" ssh-ed25519 AAAA... cayce-mac-pull' \
| sudo tee -a /home/backup-pull/.ssh/authorized_keys
## Pull desde la Mac
rsync -av --partial backup-pull@15.235.3.239:/ ~/backups/cayce/
## Verificación realizada (2026-10-28, con clave desechada después)
| Prueba | Resultado |
|---|---|
| Shell interactivo | ✗ `rrsync error: Not invoked via sshd` |
| Comando arbitrario (`cat /etc/shadow`) | ✗ `SSH_ORIGINAL_COMMAND does not run rsync` |
| rsync de `/etc/` | ✗ `invalid rsync-command syntax or options` (fuera del root permitido) |
| scp/sftp | ✗ (restrict) |
| **rsync pull de `/var/backups/cayce/`** | ✓ funciona |
| rsync write (upload) | ✗ protocolo cortado, nada escrito |
La clave usada para las pruebas se **borró**; `authorized_keys` quedó vacío.
+119
View File
@@ -0,0 +1,119 @@
#!/usr/bin/env bash
# cayce-backup — encrypted daily backup of CayceOps + Gitea + config.
#
# Design contract (Tanda B):
# - EVERY artifact is encrypted with age BEFORE it leaves the tmpdir.
# No plaintext is ever written outside the tmpdir, which is removed
# by a trap even on failure.
# - The age recipient lives in /etc/cayce-backup/age-recipient.pub
# (public key only). The PRIVATE key never touches this server:
# decryption happens on the operator's machine after a pull.
# - Any step failure => non-zero exit => journald entry (run via systemd).
# - Retention: 7 daily directories under /var/backups/cayce/.
#
# The restore procedure lives in deploy/backup/RESTORE.md.
set -euo pipefail
DATE="$(date -u +%F)"
BACKUP_ROOT="/var/backups/cayce"
DEST="$BACKUP_ROOT/$DATE"
RECIPIENT_FILE="/etc/cayce-backup/age-recipient.pub"
CAYCE_DIR="/home/ubuntu/cayce-ops"
ENV_FILE="$CAYCE_DIR/.env"
RETENTION_DAYS=7
log() { echo "[cayce-backup] $(date -u +%FT%TZ) $*"; }
die() { echo "[cayce-backup] FATAL: $*" >&2; exit 1; }
# --- preconditions -------------------------------------------------------------
[[ $EUID -eq 0 ]] || die "must run as root (systemd)"
command -v age >/dev/null || die "age not installed"
[[ -r "$RECIPIENT_FILE" ]] || die "age recipient not configured: create $RECIPIENT_FILE (public key only)"
RECIPIENT="$(tr -d "[:space:]" < "$RECIPIENT_FILE")"
[[ "$RECIPIENT" == age1* ]] || die "$RECIPIENT_FILE does not look like an age1... public key"
# --- workspace: everything plaintext lives here and dies here ------------------
TMP="$(mktemp -d /tmp/cayce-backup.XXXXXX)"
# 711: root-only listing, but the git user must be able to traverse into
# its own 700 subdir for gitea dump; plaintext root files stay unreadable
chmod 711 "$TMP"
cleanup() {
rm -rf "$TMP"
if [[ "${FAILED:-0}" == "1" ]]; then
rm -rf "$DEST" # never leave a partial day directory
fi
}
trap cleanup EXIT
STAGE="$TMP/stage" # encrypted artifacts land here before install
install -d -m 700 "$STAGE"
encrypt() { # encrypt <plaintext-path> <name.age>
age -r "$RECIPIENT" -o "$STAGE/$2" "$1"
}
# --- 1. Postgres dump (cayce-ops DB) --------------------------------------------
DB_URL="${CAYCE_OPS_DATABASE_URL:-}"
if [[ -z "$DB_URL" && -r "$ENV_FILE" ]]; then
# shellcheck disable=SC1090
DB_URL="$(grep -E "^CAYCE_OPS_DATABASE_URL=" "$ENV_FILE" | head -1 | cut -d= -f2-)"
fi
[[ -n "$DB_URL" ]] || die "no CAYCE_OPS_DATABASE_URL in env or $ENV_FILE"
PG_URL="$(printf "%s" "$DB_URL" | sed -E "s#postgresql\+[a-z]+://#postgresql://#")"
log "dumping database from ${PG_URL%%@*@}***@..."
pg_dump "$PG_URL" --format=custom --file "$TMP/cayce-ops-db.dump"
encrypt "$TMP/cayce-ops-db.dump" "cayce-ops-db.dump.age"
log "database dump: $(du -h "$TMP/cayce-ops-db.dump" | cut -f1) plaintext, encrypted ok"
# --- 2. Gitea dump (repos + sqlite + attachments) -------------------------------
GITEA_BIN="$(command -v gitea || echo /usr/local/bin/gitea)"
[[ -x "$GITEA_BIN" ]] || die "gitea binary not found"
install -d -o git -g git -m 700 "$TMP/gitea-work"
log "running gitea dump (live; see RESTORE.md for consistency notes)..."
sudo -u git "$GITEA_BIN" dump -c /etc/gitea/app.ini \
--file "$TMP/gitea-work/gitea-dump.zip" --tempdir "$TMP/gitea-work" --quiet
encrypt "$TMP/gitea-work/gitea-dump.zip" "gitea-dump.zip.age"
log "gitea dump: $(du -h "$TMP/gitea-work/gitea-dump.zip" | cut -f1) plaintext, encrypted ok"
# --- 3. Config tar (Caddy, Gitea app.ini + internal_token, units, .env) ----------
CONF="$TMP/config"
install -d -m 700 "$CONF"
mkdir -p "$CONF/caddy" "$CONF/gitea" "$CONF/systemd" "$CONF/cayce-ops"
cp -a /etc/caddy/Caddyfile "$CONF/caddy/" 2>/dev/null || true
cp -a /etc/caddy/Caddyfile.d "$CONF/caddy/" 2>/dev/null || true
cp -a /etc/gitea/app.ini "$CONF/gitea/" 2>/dev/null || true
cp -a /etc/gitea/internal_token "$CONF/gitea/" 2>/dev/null || true
find /etc/systemd/system -maxdepth 1 -name "cayce-*" \
-exec cp -a {} "$CONF/systemd/" \; 2>/dev/null || true
[[ -r "$ENV_FILE" ]] && cp -a "$ENV_FILE" "$CONF/cayce-ops/dot-env" \
|| log "WARNING: $ENV_FILE not present — config backup continues without it"
# at least Caddy or Gitea config must exist, else the tar is useless
[[ -e "$CONF/caddy/Caddyfile" || -e "$CONF/gitea/app.ini" ]] \
|| die "no config sources found (Caddyfile/app.ini missing)"
tar -C "$CONF" -czf "$TMP/config.tar.gz" .
encrypt "$TMP/config.tar.gz" "config.tar.gz.age"
log "config tar: $(du -h "$TMP/config.tar.gz" | cut -f1) plaintext, encrypted ok"
# --- 4. install + manifest -------------------------------------------------------
install -d -m 750 -o root -g backup-pull "$BACKUP_ROOT" 2>/dev/null \
|| install -d -m 700 "$BACKUP_ROOT" # backup-pull group may not exist yet
install -d -m 750 -o root -g backup-pull "$DEST" 2>/dev/null \
|| install -d -m 700 "$DEST"
install -m 640 -o root -g backup-pull "$STAGE"/*.age "$DEST/" 2>/dev/null \
|| install -m 600 "$STAGE"/*.age "$DEST/"
( cd "$DEST" && sha256sum ./*.age > "$TMP/manifest.sha256" )
install -m 640 -o root -g backup-pull "$TMP/manifest.sha256" "$DEST/manifest.sha256" 2>/dev/null \
|| install -m 600 "$TMP/manifest.sha256" "$DEST/manifest.sha256"
log "manifest written: $(wc -l < "$DEST/manifest.sha256") artifacts"
# --- 5. retention -----------------------------------------------------------------
PRUNED=0
while IFS= read -r old; do
rm -rf "$old"
PRUNED=$((PRUNED + 1))
log "retention: removed $old"
done < <(find "$BACKUP_ROOT" -mindepth 1 -maxdepth 1 -type d -name "2*-*-*" -mtime +"$RETENTION_DAYS")
log "retention: kept last $RETENTION_DAYS days (pruned $PRUNED)"
log "BACKUP OK → $DEST ($(du -sh "$DEST" | cut -f1))"
+14
View File
@@ -0,0 +1,14 @@
[Unit]
Description=Cayce — encrypted daily backup (CayceOps DB + Gitea + config)
After=network-online.target postgresql.service
Wants=network-online.target
[Service]
Type=oneshot
ExecStart=/usr/local/bin/cayce-make-backup
# hardening
NoNewPrivileges=true
ProtectHome=read-only
ReadWritePaths=/var/backups/cayce /tmp
ProtectSystem=strict
PrivateTmp=false
+10
View File
@@ -0,0 +1,10 @@
[Unit]
Description=Cayce — daily backup schedule (03:15 UTC)
[Timer]
OnCalendar=*-*-* 03:15:00 UTC
Persistent=true
RandomizedDelaySec=300
[Install]
WantedBy=timers.target
+189
View File
@@ -0,0 +1,189 @@
"""Round-3 hardening tests: append-only audit log + strict legacy-prefix rule
+ anchor cross-verification (issue: operator key WAS the agent key; the
audit trail must now be tamper-evident against the DB owner itself)."""
from __future__ import annotations
import pytest
from sqlalchemy import text
from cayceops.models import AuditEntry, audit, verify_audit_chain
def _seed(sess, n: int = 3) -> list[int]:
ids = []
for i in range(n):
audit(sess, actor=f"daemon:{i}", action=f"act{i}", detail={"i": i})
sess.flush()
ids.append(sess.query(AuditEntry).order_by(AuditEntry.id.desc()).first().id)
return ids
# --- 1. legacy-prefix rule -----------------------------------------------------
def test_null_before_genesis_still_ok(sess):
"""Documented legacy prefix: NULL hashes before the first hashed row."""
sess.add(AuditEntry(actor="legacy", action="old", detail={}, ts=__import__(
"datetime").datetime.now(__import__("datetime").UTC)))
sess.flush()
_seed(sess, 2)
sess.flush()
ok, broken = verify_audit_chain(sess)
assert ok and broken is None
def test_null_in_middle_fails(sess):
"""THE FIX: erasing a hash mid-chain is tampering and must be reported."""
ids = _seed(sess, 3)
middle = ids[1]
sess.execute(
text("UPDATE audit_log SET entry_hash = NULL WHERE id = :i"), {"i": middle}
)
sess.flush()
ok, broken = verify_audit_chain(sess)
assert not ok
assert broken == middle
def test_null_after_chain_started_reports_the_row(sess):
ids = _seed(sess, 4)
sess.execute(
text("UPDATE audit_log SET entry_hash = NULL WHERE id = :i"), {"i": ids[3]}
)
sess.flush()
ok, broken = verify_audit_chain(sess)
assert not ok and broken == ids[3]
# --- 2/3. append-only enforcement (migration 0004) ------------------------------
def _apply_0004(sess):
import os
path = os.path.join(
os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
"alembic",
"versions",
"0004_audit_append_only.py",
)
spec = {}
with open(path) as fh:
exec(compile(fh.read(), path, "exec"), spec)
# reuse the sqlite trigger DDL directly (dialect-aware inside the module)
for ddl in spec["SQLITE_TRIGGERS"]:
sess.execute(text(ddl))
sess.commit()
def test_update_rejected_after_migration(sess):
_seed(sess, 1)
_apply_0004(sess)
with pytest.raises(Exception, match="append-only"):
sess.execute(text("UPDATE audit_log SET actor = 'evil'"))
sess.flush()
def test_delete_rejected_after_migration(sess):
_seed(sess, 1)
_apply_0004(sess)
with pytest.raises(Exception, match="append-only"):
sess.execute(text("DELETE FROM audit_log"))
sess.flush()
def test_insert_still_allowed_after_migration(sess):
_seed(sess, 1)
_apply_0004(sess)
audit(sess, actor="daemon:x", action="still-works", detail={})
sess.flush()
ok, broken = verify_audit_chain(sess)
assert ok and broken is None
# --- 4. anchor cross-verification ----------------------------------------------
def test_no_anchors_reported_explicitly(sess):
from cayceops.components.monitor.monitor import verify_chain_against_anchors
_seed(sess, 2)
report = verify_chain_against_anchors(sess)
assert report["status"] == "no_anchors"
assert "never been anchored" in report["detail"]
def test_local_only_when_no_fetcher(sess):
from datetime import UTC, datetime
from cayceops.components.monitor.monitor import verify_chain_against_anchors
_seed(sess, 2)
last = sess.query(AuditEntry).order_by(AuditEntry.id.desc()).first()
sess.add(
AuditEntry(
actor="daemon:monitor",
action="anchor",
detail={
"event_id": "deadbeef",
"last_hash": last.entry_hash,
"last_entry": last.id,
},
ts=datetime.now(UTC),
)
)
sess.flush()
report = verify_chain_against_anchors(sess)
assert report["status"] == "local_only"
assert "LOCAL record" in report["detail"]
def test_anchor_mismatch_detected(sess):
from datetime import UTC, datetime
from cayceops.components.monitor.monitor import verify_chain_against_anchors
_seed(sess, 2)
last = sess.query(AuditEntry).order_by(AuditEntry.id.desc()).first()
sess.add(
AuditEntry(
actor="daemon:monitor",
action="anchor",
detail={"event_id": "beef", "last_hash": last.entry_hash, "last_entry": last.id},
ts=datetime.now(UTC),
)
)
sess.flush()
def foreign_fetcher(deployment):
# a note attesting some OTHER chain entirely
return ["cayce-ops audit anchor deployment=itest last_hash=" + "aa" * 32]
report = verify_chain_against_anchors(sess, relay_fetcher=foreign_fetcher)
assert report["status"] == "mismatch"
assert "does not attest" in report["detail"]
def test_anchor_match_ok(sess):
from datetime import UTC, datetime
from cayceops.components.monitor.monitor import verify_chain_against_anchors
_seed(sess, 2)
last = sess.query(AuditEntry).order_by(AuditEntry.id.desc()).first()
sess.add(
AuditEntry(
actor="daemon:monitor",
action="anchor",
detail={"event_id": "beef", "last_hash": last.entry_hash, "last_entry": last.id},
ts=datetime.now(UTC),
)
)
sess.flush()
def honest_fetcher(deployment):
return [f"cayce-ops audit anchor deployment={deployment} last_hash={last.entry_hash}"]
report = verify_chain_against_anchors(sess, relay_fetcher=honest_fetcher)
assert report["status"] == "ok"
+80
View File
@@ -0,0 +1,80 @@
"""Spanish negation tests (round-3 point 4): merchants write in Spanish;
negations like 'nunca', 'lamentablemente' or 'no podemos' must veto
`approval` exactly like their English counterparts. Veto rule unchanged:
when in doubt -> needs_review."""
from __future__ import annotations
import pytest
from cayceops.components.maild import classify as clf
from cayceops.models import MailClassification
APPROVAL_ONLY = MailClassification.APPROVAL
def _never_approval(cfg_factory, text: str) -> None:
kind, _ = clf.classify(text, cfg_factory())
assert kind is not APPROVAL_ONLY, f"{text!r} classified as APPROVAL"
def test_english_baseline_still_works(cfg):
config = cfg()
kind, _ = clf.classify("Your application was approved, welcome to the program", config)
assert kind is APPROVAL_ONLY
@pytest.mark.parametrize(
"text",
[
# 1. rejection stem
"Lamentamos informarle que su solicitud fue rechazada",
# 2. nunca
"Nunca aceptamos negocios de criptomonedas",
# 3. jamás
"Jamás hemos trabajado con affiliates externos",
# 4. tampoco
"No podemos aceptar su sitio, tampoco podemos ofrecerle otro acuerdo",
# 5. ningún/ninguna
"En este momento no tenemos ningún hueco para nuevos partners",
# 6. no podemos (multi-word)
"Su propuesta es interesante pero no podemos proceder",
# 7. no es posible (multi-word)
"Gracias por escribir, pero no es posible colaborar este trimestre",
# 8. lamentablemente
"Lamentablemente su aplicación no ha sido aprobada",
# 9. no aceptamos
"Por política interna no aceptamos sitios de viajes",
# 10. mixed spanish/english: negation in spanish, approval word in english
"Thanks for reaching out! Lamentablemente we cannot approve your site",
# 11. mixed: approval in spanish, negation in english
"Apreciamos su interés, but we are not accepting new partners",
# 12. nadie / nada
"Aquí nadie decidiría eso, así que no hay nada que aprobar",
],
)
def test_spanish_and_mixed_negations_veto_approval(cfg, text):
_never_approval(cfg, text)
def test_spanish_pure_approval_still_approves(cfg):
"""A clean Spanish approval WITHOUT any negation must still classify
approval — the veto must not over-fire."""
config = cfg()
kind, confidence = clf.classify(
"Su solicitud fue aprobada. Bienvenido al programa, estamos encantados "
"de trabajar con ustedes.",
config,
)
assert kind is APPROVAL_ONLY
assert confidence >= 0.5
def test_spanish_rejection_classifies_rejection(cfg):
config = cfg()
kind, conf = clf.classify(
"Lamentamos informarle que su solicitud fue rechazada por el comité",
config,
)
assert kind is MailClassification.REJECTION
assert conf >= 0.9