Compare commits

..
Author SHA1 Message Date
cayce d7156358f6 backup: RESTORE.md runbook + E2E restore test evidence (Tanda B, point 3)
ci / lint-test-leaks (pull_request) Canceled after 0s
RESTORE.md documents the exact from-zero recovery on a fresh VPS:
download+verify (sha256 over encrypted artifacts, no private key needed
on the pull side), decrypt on the operator Mac, Postgres pg_restore,
Gitea unzip+migrate, Caddy/systemd/.env config restore, repo re-clone,
backup pipeline reinstall with the SAME age recipient, and a
post-restore checklist.

E2E test executed on the VPS (no production touched — cayce-ops has no
production DB; a probe DB was created and dropped):
- manifest.sha256: 3/3 artifacts OK (verified WITHOUT any private key)
- age decrypt roundtrip with a throwaway keypair (destroyed afterwards)
- pg_restore TOC valid; full restore into a scratch DB: 5000/5000 rows
- gitea zip valid, config tar valid (34 entries)
- all test material removed: probe DB dropped, throwaway key deleted,
  recipient file removed, test backup deleted (was encrypted to the
  destroyed throwaway key)
2026-10-28 21:07:58 +00:00
cayce 508371184d Merge PR #14: usuario backup-pull de solo lectura
ci / lint-test-leaks (push) Canceled after 0s
2026-10-28 21:01:50 +00:00
cayce fc2dcd4fca backup: backup-pull user — read-only rsync-only SSH access (Tanda B, point 2)
ci / lint-test-leaks (pull_request) Canceled after 0s
System user backup-pull created on the VPS (no sudo, locked password).
authorized_keys restricted with restrict + forced command
rrsync -ro /var/backups/cayce. Verified with a since-deleted throwaway
key: no shell, no arbitrary commands, no sftp/scp, no reads outside
/var/backups/cayce, no writes; the positive rsync pull works.

authorized_keys is left EMPTY — the operator adds their real key with
the exact line documented in deploy/backup/backup-pull-setup.md.
2026-10-28 21:01:44 +00:00
cayce 121183ac83 Merge PR #13: generación de backups cifrados
ci / lint-test-leaks (push) Canceled after 0s
2026-10-28 20:59:35 +00:00
cayce b87a51afc0 backup: encrypted daily backups — VPS generates, Mac pulls (Tanda B, point 1)
ci / lint-test-leaks (pull_request) Canceled after 0s
deploy/backup/make-backup.sh (installed as /usr/local/bin/cayce-make-backup):
- pg_dump custom-format of the cayce-ops DB (URL from env or .env,
  SQLAlchemy driver qualifier stripped, credentials never logged)
- gitea dump (repos + sqlite + attachments), live; consistency notes in RESTORE.md
- config tar: Caddyfile(+d), gitea app.ini + internal_token, cayce-* units, .env
- EACH artifact encrypted with age BEFORE leaving the tmpdir; recipient
  read from /etc/cayce-backup/age-recipient.pub (public key only). The
  private age key never touches this server.
- plaintext exists ONLY in a mktemp dir removed by an EXIT trap; a partial
  day directory is removed on failure; any step failure => exit != 0 =>
  journald entry
- manifest.sha256 over the encrypted artifacts (pull side verifies
  integrity WITHOUT the age private key)
- destination /var/backups/cayce/YYYY-MM-DD/ (750 root:backup-pull, 640)
- retention: 7 daily dirs, pruned by mtime

systemd: cayce-backup.service (oneshot, hardened: ProtectSystem=strict,
NoNewPrivileges, ProtectHome=read-only) + cayce-backup.timer (daily
03:15 UTC, Persistent). Timer ENABLED on the VPS per operator instruction;
runs fail loudly (exit 1, clear journald message) until the age recipient
is provided — that is the designed not-configured state.
2026-10-28 20:59:28 +00:00
cayce d527164434 Merge PR #12: identidad sin afirmación de jurisdicción
ci / lint-test-leaks (push) Canceled after 0s
2026-10-28 20:01:09 +00:00
cayce 1c4ba234d0 identity: remove unverified jurisdiction claim, state the approval principle
ci / lint-test-leaks (pull_request) Canceled after 0s
The previous text asserted participation in an external jurisdiction
pilot without a verifiable source. Replaced with what is actually true
of this codebase: every external-effect action requires cryptographically
signed human approval, with a key that never resides on Cayce
infrastructure (D16).
2026-10-28 20:01:02 +00:00
cayce 64f89df289 Merge PR #11: clasificador con negaciones en español
ci / lint-test-leaks (push) Canceled after 0s
2026-10-28 19:59:50 +00:00
cayce f5b026070b audit-4: classifier understands Spanish negations (veto unchanged)
ci / lint-test-leaks (pull_request) Canceled after 0s
Merchants write in Spanish; negations like nunca, lamentablemente or
no podemos must veto approval exactly like their English counterparts.
Previously an all-Spanish rejection mail could fall through to OTHER,
and a mixed mail with the approval word in English approved despite a
Spanish negation.

- negation list adds: nunca, jamas, tampoco, ningun/ninguna, nadie,
  nada, no podemos, no es posible, no aceptamos, rechaz*, descartad*,
  lamentablemente, no ha sido aprobad*, no procede
- new Spanish REJECTION rule (rechaz*, solicitud denegada, no cumple
  los requisitos) and Spanish APPROVAL rule (aprob*, aceptad*,
  bienvenid*, encantados de trabajar)
- veto rule unchanged: any negation anywhere -> needs_review
- tests: 15 new (12 veto cases incl. 2 mixed-language, pure-Spanish
  approval still approves, Spanish rejection classifies rejection)

Gate: ruff 0 · 96 unit + 1 PG integration · gitleaks clean.
2026-10-28 19:59:44 +00:00
cayce 60300a2f1a Merge PR #10: cadena de auditoría a prueba de dueño
ci / lint-test-leaks (push) Canceled after 0s
2026-10-28 19:56:57 +00:00
cayce 09fe9199b3 audit-3: append-only audit log + strict legacy-prefix rule + anchor cross-check
ci / lint-test-leaks (pull_request) Canceled after 0s
- verify_audit_chain(): rows with entry_hash IS NULL are only tolerated
  BEFORE the first hashed row (documented legacy prefix). A NULL hash
  AFTER the chain started is tampering and fails verification at that
  row id — previously any NULL row anywhere was silently skipped, so an
  attacker with UPDATE could erase intermediate evidence undetected.
- migration 0004: audit_log becomes append-only. SQLite: BEFORE UPDATE/
  DELETE triggers RAISE(ABORT). Postgres: BEFORE triggers RAISE EXCEPTION.
  Closes the hole for the table owner and direct SQL connections.
- monitor.verify_chain_against_anchors(): compares the chain tail against
  anchors published to Nostr. Explicit statuses: no_anchors (never
  anchored / notes missing on relays), local_only (no fetcher), ok
  (relay note attests the tail or the last local anchor), mismatch.
- tests: NULL-before-genesis ok, NULL-in-middle fails at that id, NULL
  late fails, UPDATE rejected, DELETE rejected, INSERT still allowed,
  no-anchors explicit, local-only explicit, mismatch detected, match ok.

Gate: ruff 0 · 81 unit + 1 PG integration · alembic 0001→0004 on PG ·
gitleaks clean · make check exit 0.
2026-10-28 19:56:50 +00:00
cayce 029e4f77d7 Merge PR #9: D16 — desacoplar clave de operador de identidad de agente
ci / lint-test-leaks (push) Canceled after 0s
2026-10-28 19:51:21 +00:00
12 changed files with 847 additions and 11 deletions
+8 -5
View File
@@ -1,10 +1,13 @@
# Cayce Ops # Cayce Ops
Cayce Ops is the autonomous operations stack of **Cayce** — an autonomous entity CayceOps es el stack de operaciones de Cayce, un experimento de operaciones
operating under the Próspera ZEDE agentic-companies jurisdiction pilot. Cayce runs agénticas de Cayce Co. Toda acción con efectos externos requiere aprobación
[bitcoinhotels.org](https://bitcoinhotels.org) (Bitcoin-native travel guides) and humana firmada criptográficamente, con una clave que nunca reside en la
[hardcap.news](https://hardcap.news) (Bitcoin security intelligence). The infraestructura de Cayce.
experiment is managed by Cayce itself.
Cayce operates [bitcoinhotels.org](https://bitcoinhotels.org)
(Bitcoin-native travel guides) and [hardcap.news](https://hardcap.news)
(Bitcoin security intelligence).
Five decoupled components share one Postgres database. Each is a CLI command, Five decoupled components share one Postgres database. Each is a CLI command,
driven by cron/systemd timers. Nothing runs without an audit trail. driven by cron/systemd timers. Nothing runs without an audit trail.
@@ -0,0 +1,89 @@
"""audit_log append-only: reject UPDATE and DELETE (round-2 hardening)
Revision ID: 0004
Revises: 0003
Create Date: 2026-10-28
An audit trail that can be edited is not an audit trail. This migration
installs, on every engine:
- SQLite: BEFORE UPDATE/DELETE triggers that RAISE(ABORT).
- Postgres: BEFORE UPDATE/DELETE triggers that RAISE EXCEPTION.
The writer roles never had UPDATE/DELETE grants in roles.sql; this closes
the hole for the table owner and for direct SQL connections.
"""
from __future__ import annotations
from alembic import op
revision = "0004"
down_revision = "0003"
branch_labels = None
depends_on = None
SQLITE_TRIGGERS = [
"""
CREATE TRIGGER IF NOT EXISTS audit_log_no_update
BEFORE UPDATE ON audit_log
BEGIN
SELECT RAISE(ABORT, 'cayce-ops: audit_log is append-only (UPDATE rejected)');
END;
""",
"""
CREATE TRIGGER IF NOT EXISTS audit_log_no_delete
BEFORE DELETE ON audit_log
BEGIN
SELECT RAISE(ABORT, 'cayce-ops: audit_log is append-only (DELETE rejected)');
END;
""",
]
PG_TRIGGERS = """
CREATE OR REPLACE FUNCTION cayce_ops_audit_append_only() RETURNS trigger AS $$
BEGIN
RAISE EXCEPTION 'cayce-ops: audit_log is append-only (% rejected)', TG_OP;
END;
$$ LANGUAGE plpgsql;
"""
def upgrade() -> None:
bind = op.get_bind()
if bind.dialect.name == "postgresql":
op.execute(PG_TRIGGERS)
op.execute(
"DROP TRIGGER IF EXISTS audit_log_no_update ON audit_log;"
)
op.execute(
"CREATE TRIGGER audit_log_no_update BEFORE UPDATE ON audit_log "
"FOR EACH ROW EXECUTE FUNCTION cayce_ops_audit_append_only();"
)
op.execute(
"DROP TRIGGER IF EXISTS audit_log_no_delete ON audit_log;"
)
op.execute(
"CREATE TRIGGER audit_log_no_delete BEFORE DELETE ON audit_log "
"FOR EACH ROW EXECUTE FUNCTION cayce_ops_audit_append_only();"
)
else:
for ddl in SQLITE_TRIGGERS:
bind.execute(sa_text(ddl))
def sa_text(sql: str):
import sqlalchemy as sa
return sa.text(sql)
def downgrade() -> None:
bind = op.get_bind()
if bind.dialect.name == "postgresql":
op.execute("DROP TRIGGER IF EXISTS audit_log_no_update ON audit_log;")
op.execute("DROP TRIGGER IF EXISTS audit_log_no_delete ON audit_log;")
op.execute("DROP FUNCTION IF EXISTS cayce_ops_audit_append_only();")
else:
bind.execute(sa_text("DROP TRIGGER IF EXISTS audit_log_no_update;"))
bind.execute(sa_text("DROP TRIGGER IF EXISTS audit_log_no_delete;"))
+22 -2
View File
@@ -19,10 +19,17 @@ from cayceops.models import MailClassification
_APPROVAL_WORDS = re.compile(r"\b(approv|accept|welcome)", re.I) _APPROVAL_WORDS = re.compile(r"\b(approv|accept|welcome)", re.I)
# any of these tokens anywhere in the mail vetoes an `approval` result # any of these tokens anywhere in the mail vetoes an `approval` result.
# English and Spanish (merchants write in both; mixed mails are common).
_NEGATIONS = re.compile( _NEGATIONS = re.compile(
r"\b(not|no|never|cannot|can'?t|won'?t|hasn'?t|isn'?t|" r"\b(not|no|never|cannot|can'?t|won'?t|hasn'?t|isn'?t|"
r"no\s+longer|unable|decline\w*|regret\w*|unfortunately)\b", r"no\s+longer|unable|decline\w*|regret\w*|unfortunately|"
# spanish: standalone negation words
r"nunca|jam[aá]s|tampoco|ning[uú]n(?:a|o)?|nadie|nada|"
# spanish: multi-word negations and rejection stems
r"no\s+podemos|no\s+es\s+posible|no\s+acept(?:amos|ar[aá])|"
r"rechaz\w*|lamentablemente|descartad\w*|"
r"no\s+ha\s+sido\s+aprobad\w*|no\s+procede)\b",
re.I, re.I,
) )
@@ -34,11 +41,24 @@ _RULES: list[tuple[MailClassification, re.Pattern[str], float]] = [
re.compile(r"\b(declin|reject|not able to|unable to|we regret|won.t be able)\w*", re.I), # noqa: E501 re.compile(r"\b(declin|reject|not able to|unable to|we regret|won.t be able)\w*", re.I), # noqa: E501
0.9, 0.9,
), ),
(
MailClassification.REJECTION,
# spanish rejections (strong stems, not just negation words)
re.compile(r"\b(rechaz\w*|no\s+vamos\s+a\s+pod(?:er|emos)|su\s+solicitud\s+ha\s+sido\s+denegada|no\s+cumple\s+los\s+requisitos)\b", re.I), # noqa: E501
0.9,
),
( (
MailClassification.APPROVAL, MailClassification.APPROVAL,
re.compile(r"\b(approv|accept|welcome to the|you.re (in|approved)|live|activated|glad to)\w*", re.I), # noqa: E501 re.compile(r"\b(approv|accept|welcome to the|you.re (in|approved)|live|activated|glad to)\w*", re.I), # noqa: E501
0.85, 0.85,
), ),
(
MailClassification.APPROVAL,
# spanish approvals (standalone positive stems; the veto still
# downgrades if ANY negation appears elsewhere in the mail)
re.compile(r"\b(aprob\w*|aceptad\w*|bienvenid\w*|ha\s+sido\s+aceptada|encantados?\s+de\s+(trabajar|colaborar))\b", re.I), # noqa: E501
0.85,
),
( (
MailClassification.QUESTION, MailClassification.QUESTION,
re.compile(r"\b(could you|can you|please (provide|clarify|confirm)|what (is|are)|how (do|does|can)|need more (info|details))\b", re.I), # noqa: E501 re.compile(r"\b(could you|can you|please (provide|clarify|confirm)|what (is|are)|how (do|does|can)|need more (info|details))\b", re.I), # noqa: E501
+117
View File
@@ -251,3 +251,120 @@ def build(cfg: Config | None = None) -> int:
return 0 return 0
finally: finally:
engine().dispose() engine().dispose()
def _latest_chain_hash(sess) -> str | None:
"""Last non-NULL entry_hash in the audit chain (None if chain empty)."""
from cayceops.models import AuditEntry
last = (
sess.query(AuditEntry)
.filter(AuditEntry.entry_hash.isnot(None))
.order_by(AuditEntry.id.desc())
.first()
)
return last.entry_hash if last else None
def _published_anchors(sess) -> list[dict]:
"""Anchor audit entries recorded locally, oldest first."""
from cayceops.models import AuditEntry
rows = (
sess.query(AuditEntry)
.filter(AuditEntry.action == "anchor")
.order_by(AuditEntry.id.asc())
.all()
)
return [
{
"event_id": (r.detail or {}).get("event_id"),
"last_hash": (r.detail or {}).get("last_hash"),
"last_entry": (r.detail or {}).get("last_entry"),
"ts": r.ts.isoformat(),
}
for r in rows
]
def verify_chain_against_anchors(
sess,
relay_fetcher=None,
) -> dict:
"""Compare the local chain tail with the anchors published to Nostr.
relay_fetcher(deployment) -> list of note contents (strings) found on
the relays for this deployment's anchor tag. When None, only local
coherence is checked and the result says so.
Returns a report dict:
status: "no_anchors" | "local_only" | "ok" | "mismatch"
detail: human-readable explanation
local_last_hash, local_anchors, relay_notes (when fetched)
"""
local_hash = _latest_chain_hash(sess)
anchors = _published_anchors(sess)
report: dict = {
"local_last_hash": local_hash,
"local_anchors": anchors,
"relay_notes": None,
}
if not anchors:
report["status"] = "no_anchors"
report["detail"] = (
"the chain has never been anchored to Nostr — no external "
"attestation exists yet (anchor once with `monitor anchor`)"
)
return report
last_anchor = anchors[-1]
if relay_fetcher is None:
report["status"] = "local_only"
report["detail"] = (
"compared only against the LOCAL record of past anchors "
"(no relay fetch configured); last anchored hash "
f"{last_anchor['last_hash']}"
)
return report
from cayceops.config import load as _load
deployment = _load().deployment_id
notes = relay_fetcher(deployment) or []
report["relay_notes"] = notes
if not notes:
report["status"] = "no_anchors"
report["detail"] = (
"local anchor records exist but no anchor notes were found on "
"the relays — the external attestation is missing or was deleted"
)
return report
# the freshest relay note must attest the current chain tail
expected_fragment = f"last_hash={local_hash}"
latest_note = notes[-1]
if expected_fragment in latest_note:
report["status"] = "ok"
report["detail"] = "latest relay anchor matches the current chain tail"
return report
# maybe the chain advanced after the last anchor — that's fine ONLY if
# the note matches the hash recorded in the last local anchor entry
if last_anchor["last_hash"] and last_anchor["last_hash"] in latest_note:
report["status"] = "ok"
report["detail"] = (
"relay anchor matches the last local anchor; the chain has "
"advanced since (anchor again to attest the new tail)"
)
return report
report["status"] = "mismatch"
report["detail"] = (
f"relay anchor does not attest chain tail {local_hash} nor the last "
f"local anchor {last_anchor['last_hash']} — the published anchor "
"does not correspond to this database"
)
return report
+11 -4
View File
@@ -209,14 +209,21 @@ def _build_entry(
def verify_audit_chain(sess) -> tuple[bool, int | None]: def verify_audit_chain(sess) -> tuple[bool, int | None]:
"""Re-walk the chain. Returns (ok, first_broken_entry_id or None). """Re-walk the chain. Returns (ok, first_broken_entry_id or None).
Legacy rows (entry_hash IS NULL, written before migration 0002) are Rows with entry_hash IS NULL are ONLY tolerated BEFORE the first hashed
skipped and documented: the chain begins GENESIS at the first hashed row, row (the documented legacy prefix from before migration 0002). A NULL
so a migrated database verifies clean without any tampering. hash AFTER the chain has started is tampering — an attacker erasing the
hash of an intermediate row to hide an edit — and fails verification at
that row's id.
""" """
prev_hash: str | None = None # None = still in legacy prefix prev_hash: str | None = None # None = still in legacy prefix
chain_started = False
for entry in sess.query(AuditEntry).order_by(AuditEntry.id).all(): for entry in sess.query(AuditEntry).order_by(AuditEntry.id).all():
if entry.entry_hash is None: if entry.entry_hash is None:
continue # legacy row: outside the chain by documented genesis if chain_started:
# tampering: a hash was erased in the middle of the chain
return False, entry.id
continue # legacy prefix row: outside the chain by genesis
chain_started = True
if prev_hash is None: if prev_hash is None:
expected_prev = "GENESIS" # first hashed row anchors the chain expected_prev = "GENESIS" # first hashed row anchors the chain
else: else:
+145
View File
@@ -0,0 +1,145 @@
# RESTORE.md — Restauración de CayceOps desde backups cifrados
**Modelo:** la VPS genera backups cifrados (age) en `/var/backups/cayce/YYYY-MM-DD/`;
tu Mac los descarga con el usuario `backup-pull` (solo lectura). La clave
**privada** de age vive SOLO en tu Mac. Este runbook asume una VPS nueva
(Ubuntu + IP apuntada por DNS) y tus backups descifrados a mano.
---
## 0. Descargar y descifrar (en TU Mac — nunca en la VPS)
```bash
# pull (clave restringida: solo lectura de /var/backups/cayce)
rsync -av --partial backup-pull@15.235.3.239:/ ~/backups/cayce/
# integridad (sin clave privada; manifest lista los artefactos CIFRADOS)
cd ~/backups/cayce/2026-XX-XX && shasum -a 256 -c manifest.sha256
# (Linux: sha256sum -c manifest.sha256)
# descifrar (clave privada age del operador — la que NO está en la VPS)
age -d -i ~/keys/cayce-backup.txt -o cayce-ops-db.dump cayce-ops-db.dump.age
age -d -i ~/keys/cayce-backup.txt -o gitea-dump.zip gitea-dump.zip.age
age -d -i ~/keys/cayce-backup.txt -o config.tar.gz config.tar.gz.age
```
Artefactos:
| Archivo | Contenido | Formato |
|---|---|---|
| `cayce-ops-db.dump` | BD de CayceOps | `pg_dump` custom (restaurar con `pg_restore`) |
| `gitea-dump.zip` | repos + sqlite + adjuntos + config Gitea | zip de `gitea dump` |
| `config.tar.gz` | `caddy/` (Caddyfile+Caddyfile.d), `gitea/` (app.ini + internal_token), `systemd/` (unidades cayce-*), `cayce-ops/dot-env` | tar.gz |
---
## 1. VPS nueva — base del sistema
```bash
apt-get update
apt-get install -y postgresql age rsync git
# Gitea: misma versión mayor que la VPS origen (verSTEP: gitea --version en el viejo
# servidor ANTES de apagarlo, o lee la versión del zip gitea-dump: gitea-dump zip
# contiene un archivo "app.ini" con la versión en comentarios... anota la versión
# manualmente; el binario se baja de https://dl.gitea.com/gitea/)
adduser --system --group --home /var/lib/gitea git
mkdir -p /etc/gitea /var/lib/gitea /etc/cayce-backup
curl -L "https://dl.gitea.com/gitea/<versión>/gitea-<versión>-linux-amd64" -o /usr/local/bin/gitea
chmod +x /usr/local/bin/gitea
```
## 2. Postgres
```bash
systemctl enable --now postgresql
sudo -u postgres psql <<SQL
CREATE ROLE cayce LOGIN PASSWORD '<password-del-.env-restaurado>';
CREATE DATABASE cayce_ops OWNER cayce;
SQL
# restaurar el dump (los ficheros descifrados son legibles solo por ti;
# cópiales permisos de lectura para postgres o pásalos por stdin)
sudo -u postgres pg_restore -d cayce_ops --no-owner --no-privileges \
/ruta/cayce-ops-db.dump
# verificación
sudo -u postgres psql -d cayce_ops -tAc \
"SELECT count(*) FROM audit_log;" # debe coincidir con la última ancla
```
## 3. Gitea
```bash
# config restaurada desde config.tar.gz
tar -xzf config.tar.gz -C /tmp/restored
cp /tmp/restored/gitea/app.ini /etc/gitea/app.ini
chown root:git /etc/gitea/app.ini && chmod 660 /etc/gitea/app.ini
cp /tmp/restored/gitea/internal_token /etc/gitea/ && chown git:git /etc/gitea/internal_token
# NOTA sobre ROOT_URL: app.ini restaurado apunta a https://git.cayce-co.xyz/ —
# el DNS de ese subdominio debe existir ANTES de iniciar Gitea o las URLs
# absolutas (webhooks, notificaciones) quedarán mal.
# dump de Gitea: contiene data/, repos/ y custom/. Restaurar SOBRE el dump
# de Gitea exige la misma versión del binario; descomprimir dentro de WORK_PATH
systemctl stop gitea 2>/dev/null || true
unzip -o gitea-dump.zip -d /tmp/gitea-restore
rsync -a /tmp/gitea-restore/data/ /var/lib/gitea/data/
rsync -a /tmp/gitea-restore/repos/ /var/lib/gitea/data/gitea-repositories/ 2>/dev/null || true
rsync -a /tmp/gitea-restore/custom/ /var/lib/gitea/custom/ 2>/dev/null || true
chown -R git:git /var/lib/gitea
sudo -u git /usr/local/bin/gitea migrate -c /etc/gitea/app.ini # ajusta esquema si cambia versión
```
> **Consistencia:** los dumps de Gitea se toman EN CALIENTE (el servicio sigue
> corriendo). Riesgo teórico de inconsistencia sqlite/repos si un push ocurre
> en el mismo instante. Para un backup garantizado-consistente: `systemctl stop
> gitea` antes de `gitea dump` y volver a levantar después (no automatizado a
> propósito: no queremos ventanas de downtime diarias).
## 4. Config del sistema (Caddy, unidades systemd, .env)
```bash
# Caddy
apt-get install -y caddy # o el instalador oficial
rsync -a /tmp/restored/caddy/Caddyfile /etc/caddy/Caddyfile
rsync -a /tmp/restored/caddy/Caddyfile.d/ /etc/caddy/Caddyfile.d/
systemctl enable --now caddy # emitirá certificados si el DNS ya apunta aquí
# unidades systemd de cayce (DESHABILITADAS por diseño — ver riesgos)
rsync -a /tmp/restored/systemd/ /etc/systemd/system/
systemctl daemon-reload
# NO habilitar cayce-ops@*.timer sin el operador
# .env de cayce-ops
install -d -m 755 /home/ubuntu
rsync -a /tmp/restored/cayce-ops/dot-env /home/ubuntu/cayce-ops/.env
chown ubuntu:ubuntu /home/ubuntu/cayce-ops/.env && chmod 660 /home/ubuntu/cayce-ops/.env
```
## 5. Código del repo
```bash
sudo -u ubuntu git clone http://127.0.0.1:3100/cayce/cayce-ops.git /home/ubuntu/cayce-ops \
|| git clone https://git.cayce-co.xyz/cayce/cayce-ops.git /home/ubuntu/cayce-ops
cd /home/ubuntu/cayce-ops && uv sync
make check # gate verde antes de considerar la restauración completa
```
## 6. Reinstalar el pipeline de backup en la VPS nueva
```bash
install -m 700 -o root -g root deploy/backup/make-backup.sh /usr/local/bin/cayce-make-backup
install -m 644 deploy/systemd/cayce-backup.{service,timer} /etc/systemd/system/
systemctl daemon-reload && systemctl enable --now cayce-backup.timer
# la MISMA clave pública age → tu Mac sigue pudiendo descifrar:
echo 'age1<tu-pubkey>' > /etc/cayce-backup/age-recipient.pub
```
## 7. Checklist de verificación post-restauración
- [ ] `sha256sum -c manifest.sha256` OK en tu Mac (antes de descifrar)
- [ ] `pg_restore --list` del dump sin errores
- [ ] `verify_audit_chain(sess)` → `(True, None)` contra la BD restaurada
- [ ] Gitea responde por TLS en `git.cayce-co.xyz` y `git rev-parse main` da el commit esperado
- [ ] `make check` verde en el repo restaurado
- [ ] Primer backup nuevo generado y verificado desde la Mac
+43
View File
@@ -0,0 +1,43 @@
# backup-pull — usuario de solo lectura para el pull desde la Mac
La VPS genera los backups cifrados en `/var/backups/cayce/YYYY-MM-DD/`.
La Mac los descarga por rsync con una clave SSH **restringida**. La VPS
nunca tiene credenciales para escribir ni borrar nada fuera de sí misma.
## Estado instalado en la VPS
- Usuario de sistema `backup-pull` (uid 1001): **sin sudo**, password
bloqueado, home `/home/backup-pull`, shell `/bin/sh` (necesario para que
sshd ejecute el forced command; la restricción real la pone la clave, no
el shell).
- `/home/backup-pull/.ssh/authorized_keys` (600, backup-pull:backup-pull) —
**vacío** hasta que el operador añada su clave pública.
- `/var/backups/cayce` → `root:backup-pull`, `750`; artefactos `640`.
## Formato de la línea authorized_keys (una sola línea)
restrict,command="/usr/bin/rrsync -ro /var/backups/cayce" ssh-ed25519 AAAA... comentario
`restrict` = no-pty, no-port-forwarding, no-agent-forwarding, no-X11-forwarding
(y sin sftp). `rrsync -ro` = solo rsync de LECTURA, confinado a
`/var/backups/cayce`. Añadirla con:
echo 'restrict,command="/usr/bin/rrsync -ro /var/backups/cayce" ssh-ed25519 AAAA... cayce-mac-pull' \
| sudo tee -a /home/backup-pull/.ssh/authorized_keys
## Pull desde la Mac
rsync -av --partial backup-pull@15.235.3.239:/ ~/backups/cayce/
## Verificación realizada (2026-10-28, con clave desechada después)
| Prueba | Resultado |
|---|---|
| Shell interactivo | ✗ `rrsync error: Not invoked via sshd` |
| Comando arbitrario (`cat /etc/shadow`) | ✗ `SSH_ORIGINAL_COMMAND does not run rsync` |
| rsync de `/etc/` | ✗ `invalid rsync-command syntax or options` (fuera del root permitido) |
| scp/sftp | ✗ (restrict) |
| **rsync pull de `/var/backups/cayce/`** | ✓ funciona |
| rsync write (upload) | ✗ protocolo cortado, nada escrito |
La clave usada para las pruebas se **borró**; `authorized_keys` quedó vacío.
+119
View File
@@ -0,0 +1,119 @@
#!/usr/bin/env bash
# cayce-backup — encrypted daily backup of CayceOps + Gitea + config.
#
# Design contract (Tanda B):
# - EVERY artifact is encrypted with age BEFORE it leaves the tmpdir.
# No plaintext is ever written outside the tmpdir, which is removed
# by a trap even on failure.
# - The age recipient lives in /etc/cayce-backup/age-recipient.pub
# (public key only). The PRIVATE key never touches this server:
# decryption happens on the operator's machine after a pull.
# - Any step failure => non-zero exit => journald entry (run via systemd).
# - Retention: 7 daily directories under /var/backups/cayce/.
#
# The restore procedure lives in deploy/backup/RESTORE.md.
set -euo pipefail
DATE="$(date -u +%F)"
BACKUP_ROOT="/var/backups/cayce"
DEST="$BACKUP_ROOT/$DATE"
RECIPIENT_FILE="/etc/cayce-backup/age-recipient.pub"
CAYCE_DIR="/home/ubuntu/cayce-ops"
ENV_FILE="$CAYCE_DIR/.env"
RETENTION_DAYS=7
log() { echo "[cayce-backup] $(date -u +%FT%TZ) $*"; }
die() { echo "[cayce-backup] FATAL: $*" >&2; exit 1; }
# --- preconditions -------------------------------------------------------------
[[ $EUID -eq 0 ]] || die "must run as root (systemd)"
command -v age >/dev/null || die "age not installed"
[[ -r "$RECIPIENT_FILE" ]] || die "age recipient not configured: create $RECIPIENT_FILE (public key only)"
RECIPIENT="$(tr -d "[:space:]" < "$RECIPIENT_FILE")"
[[ "$RECIPIENT" == age1* ]] || die "$RECIPIENT_FILE does not look like an age1... public key"
# --- workspace: everything plaintext lives here and dies here ------------------
TMP="$(mktemp -d /tmp/cayce-backup.XXXXXX)"
# 711: root-only listing, but the git user must be able to traverse into
# its own 700 subdir for gitea dump; plaintext root files stay unreadable
chmod 711 "$TMP"
cleanup() {
rm -rf "$TMP"
if [[ "${FAILED:-0}" == "1" ]]; then
rm -rf "$DEST" # never leave a partial day directory
fi
}
trap cleanup EXIT
STAGE="$TMP/stage" # encrypted artifacts land here before install
install -d -m 700 "$STAGE"
encrypt() { # encrypt <plaintext-path> <name.age>
age -r "$RECIPIENT" -o "$STAGE/$2" "$1"
}
# --- 1. Postgres dump (cayce-ops DB) --------------------------------------------
DB_URL="${CAYCE_OPS_DATABASE_URL:-}"
if [[ -z "$DB_URL" && -r "$ENV_FILE" ]]; then
# shellcheck disable=SC1090
DB_URL="$(grep -E "^CAYCE_OPS_DATABASE_URL=" "$ENV_FILE" | head -1 | cut -d= -f2-)"
fi
[[ -n "$DB_URL" ]] || die "no CAYCE_OPS_DATABASE_URL in env or $ENV_FILE"
PG_URL="$(printf "%s" "$DB_URL" | sed -E "s#postgresql\+[a-z]+://#postgresql://#")"
log "dumping database from ${PG_URL%%@*@}***@..."
pg_dump "$PG_URL" --format=custom --file "$TMP/cayce-ops-db.dump"
encrypt "$TMP/cayce-ops-db.dump" "cayce-ops-db.dump.age"
log "database dump: $(du -h "$TMP/cayce-ops-db.dump" | cut -f1) plaintext, encrypted ok"
# --- 2. Gitea dump (repos + sqlite + attachments) -------------------------------
GITEA_BIN="$(command -v gitea || echo /usr/local/bin/gitea)"
[[ -x "$GITEA_BIN" ]] || die "gitea binary not found"
install -d -o git -g git -m 700 "$TMP/gitea-work"
log "running gitea dump (live; see RESTORE.md for consistency notes)..."
sudo -u git "$GITEA_BIN" dump -c /etc/gitea/app.ini \
--file "$TMP/gitea-work/gitea-dump.zip" --tempdir "$TMP/gitea-work" --quiet
encrypt "$TMP/gitea-work/gitea-dump.zip" "gitea-dump.zip.age"
log "gitea dump: $(du -h "$TMP/gitea-work/gitea-dump.zip" | cut -f1) plaintext, encrypted ok"
# --- 3. Config tar (Caddy, Gitea app.ini + internal_token, units, .env) ----------
CONF="$TMP/config"
install -d -m 700 "$CONF"
mkdir -p "$CONF/caddy" "$CONF/gitea" "$CONF/systemd" "$CONF/cayce-ops"
cp -a /etc/caddy/Caddyfile "$CONF/caddy/" 2>/dev/null || true
cp -a /etc/caddy/Caddyfile.d "$CONF/caddy/" 2>/dev/null || true
cp -a /etc/gitea/app.ini "$CONF/gitea/" 2>/dev/null || true
cp -a /etc/gitea/internal_token "$CONF/gitea/" 2>/dev/null || true
find /etc/systemd/system -maxdepth 1 -name "cayce-*" \
-exec cp -a {} "$CONF/systemd/" \; 2>/dev/null || true
[[ -r "$ENV_FILE" ]] && cp -a "$ENV_FILE" "$CONF/cayce-ops/dot-env" \
|| log "WARNING: $ENV_FILE not present — config backup continues without it"
# at least Caddy or Gitea config must exist, else the tar is useless
[[ -e "$CONF/caddy/Caddyfile" || -e "$CONF/gitea/app.ini" ]] \
|| die "no config sources found (Caddyfile/app.ini missing)"
tar -C "$CONF" -czf "$TMP/config.tar.gz" .
encrypt "$TMP/config.tar.gz" "config.tar.gz.age"
log "config tar: $(du -h "$TMP/config.tar.gz" | cut -f1) plaintext, encrypted ok"
# --- 4. install + manifest -------------------------------------------------------
install -d -m 750 -o root -g backup-pull "$BACKUP_ROOT" 2>/dev/null \
|| install -d -m 700 "$BACKUP_ROOT" # backup-pull group may not exist yet
install -d -m 750 -o root -g backup-pull "$DEST" 2>/dev/null \
|| install -d -m 700 "$DEST"
install -m 640 -o root -g backup-pull "$STAGE"/*.age "$DEST/" 2>/dev/null \
|| install -m 600 "$STAGE"/*.age "$DEST/"
( cd "$DEST" && sha256sum ./*.age > "$TMP/manifest.sha256" )
install -m 640 -o root -g backup-pull "$TMP/manifest.sha256" "$DEST/manifest.sha256" 2>/dev/null \
|| install -m 600 "$TMP/manifest.sha256" "$DEST/manifest.sha256"
log "manifest written: $(wc -l < "$DEST/manifest.sha256") artifacts"
# --- 5. retention -----------------------------------------------------------------
PRUNED=0
while IFS= read -r old; do
rm -rf "$old"
PRUNED=$((PRUNED + 1))
log "retention: removed $old"
done < <(find "$BACKUP_ROOT" -mindepth 1 -maxdepth 1 -type d -name "2*-*-*" -mtime +"$RETENTION_DAYS")
log "retention: kept last $RETENTION_DAYS days (pruned $PRUNED)"
log "BACKUP OK → $DEST ($(du -sh "$DEST" | cut -f1))"
+14
View File
@@ -0,0 +1,14 @@
[Unit]
Description=Cayce — encrypted daily backup (CayceOps DB + Gitea + config)
After=network-online.target postgresql.service
Wants=network-online.target
[Service]
Type=oneshot
ExecStart=/usr/local/bin/cayce-make-backup
# hardening
NoNewPrivileges=true
ProtectHome=read-only
ReadWritePaths=/var/backups/cayce /tmp
ProtectSystem=strict
PrivateTmp=false
+10
View File
@@ -0,0 +1,10 @@
[Unit]
Description=Cayce — daily backup schedule (03:15 UTC)
[Timer]
OnCalendar=*-*-* 03:15:00 UTC
Persistent=true
RandomizedDelaySec=300
[Install]
WantedBy=timers.target
+189
View File
@@ -0,0 +1,189 @@
"""Round-3 hardening tests: append-only audit log + strict legacy-prefix rule
+ anchor cross-verification (issue: operator key WAS the agent key; the
audit trail must now be tamper-evident against the DB owner itself)."""
from __future__ import annotations
import pytest
from sqlalchemy import text
from cayceops.models import AuditEntry, audit, verify_audit_chain
def _seed(sess, n: int = 3) -> list[int]:
ids = []
for i in range(n):
audit(sess, actor=f"daemon:{i}", action=f"act{i}", detail={"i": i})
sess.flush()
ids.append(sess.query(AuditEntry).order_by(AuditEntry.id.desc()).first().id)
return ids
# --- 1. legacy-prefix rule -----------------------------------------------------
def test_null_before_genesis_still_ok(sess):
"""Documented legacy prefix: NULL hashes before the first hashed row."""
sess.add(AuditEntry(actor="legacy", action="old", detail={}, ts=__import__(
"datetime").datetime.now(__import__("datetime").UTC)))
sess.flush()
_seed(sess, 2)
sess.flush()
ok, broken = verify_audit_chain(sess)
assert ok and broken is None
def test_null_in_middle_fails(sess):
"""THE FIX: erasing a hash mid-chain is tampering and must be reported."""
ids = _seed(sess, 3)
middle = ids[1]
sess.execute(
text("UPDATE audit_log SET entry_hash = NULL WHERE id = :i"), {"i": middle}
)
sess.flush()
ok, broken = verify_audit_chain(sess)
assert not ok
assert broken == middle
def test_null_after_chain_started_reports_the_row(sess):
ids = _seed(sess, 4)
sess.execute(
text("UPDATE audit_log SET entry_hash = NULL WHERE id = :i"), {"i": ids[3]}
)
sess.flush()
ok, broken = verify_audit_chain(sess)
assert not ok and broken == ids[3]
# --- 2/3. append-only enforcement (migration 0004) ------------------------------
def _apply_0004(sess):
import os
path = os.path.join(
os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
"alembic",
"versions",
"0004_audit_append_only.py",
)
spec = {}
with open(path) as fh:
exec(compile(fh.read(), path, "exec"), spec)
# reuse the sqlite trigger DDL directly (dialect-aware inside the module)
for ddl in spec["SQLITE_TRIGGERS"]:
sess.execute(text(ddl))
sess.commit()
def test_update_rejected_after_migration(sess):
_seed(sess, 1)
_apply_0004(sess)
with pytest.raises(Exception, match="append-only"):
sess.execute(text("UPDATE audit_log SET actor = 'evil'"))
sess.flush()
def test_delete_rejected_after_migration(sess):
_seed(sess, 1)
_apply_0004(sess)
with pytest.raises(Exception, match="append-only"):
sess.execute(text("DELETE FROM audit_log"))
sess.flush()
def test_insert_still_allowed_after_migration(sess):
_seed(sess, 1)
_apply_0004(sess)
audit(sess, actor="daemon:x", action="still-works", detail={})
sess.flush()
ok, broken = verify_audit_chain(sess)
assert ok and broken is None
# --- 4. anchor cross-verification ----------------------------------------------
def test_no_anchors_reported_explicitly(sess):
from cayceops.components.monitor.monitor import verify_chain_against_anchors
_seed(sess, 2)
report = verify_chain_against_anchors(sess)
assert report["status"] == "no_anchors"
assert "never been anchored" in report["detail"]
def test_local_only_when_no_fetcher(sess):
from datetime import UTC, datetime
from cayceops.components.monitor.monitor import verify_chain_against_anchors
_seed(sess, 2)
last = sess.query(AuditEntry).order_by(AuditEntry.id.desc()).first()
sess.add(
AuditEntry(
actor="daemon:monitor",
action="anchor",
detail={
"event_id": "deadbeef",
"last_hash": last.entry_hash,
"last_entry": last.id,
},
ts=datetime.now(UTC),
)
)
sess.flush()
report = verify_chain_against_anchors(sess)
assert report["status"] == "local_only"
assert "LOCAL record" in report["detail"]
def test_anchor_mismatch_detected(sess):
from datetime import UTC, datetime
from cayceops.components.monitor.monitor import verify_chain_against_anchors
_seed(sess, 2)
last = sess.query(AuditEntry).order_by(AuditEntry.id.desc()).first()
sess.add(
AuditEntry(
actor="daemon:monitor",
action="anchor",
detail={"event_id": "beef", "last_hash": last.entry_hash, "last_entry": last.id},
ts=datetime.now(UTC),
)
)
sess.flush()
def foreign_fetcher(deployment):
# a note attesting some OTHER chain entirely
return ["cayce-ops audit anchor deployment=itest last_hash=" + "aa" * 32]
report = verify_chain_against_anchors(sess, relay_fetcher=foreign_fetcher)
assert report["status"] == "mismatch"
assert "does not attest" in report["detail"]
def test_anchor_match_ok(sess):
from datetime import UTC, datetime
from cayceops.components.monitor.monitor import verify_chain_against_anchors
_seed(sess, 2)
last = sess.query(AuditEntry).order_by(AuditEntry.id.desc()).first()
sess.add(
AuditEntry(
actor="daemon:monitor",
action="anchor",
detail={"event_id": "beef", "last_hash": last.entry_hash, "last_entry": last.id},
ts=datetime.now(UTC),
)
)
sess.flush()
def honest_fetcher(deployment):
return [f"cayce-ops audit anchor deployment={deployment} last_hash={last.entry_hash}"]
report = verify_chain_against_anchors(sess, relay_fetcher=honest_fetcher)
assert report["status"] == "ok"
+80
View File
@@ -0,0 +1,80 @@
"""Spanish negation tests (round-3 point 4): merchants write in Spanish;
negations like 'nunca', 'lamentablemente' or 'no podemos' must veto
`approval` exactly like their English counterparts. Veto rule unchanged:
when in doubt -> needs_review."""
from __future__ import annotations
import pytest
from cayceops.components.maild import classify as clf
from cayceops.models import MailClassification
APPROVAL_ONLY = MailClassification.APPROVAL
def _never_approval(cfg_factory, text: str) -> None:
kind, _ = clf.classify(text, cfg_factory())
assert kind is not APPROVAL_ONLY, f"{text!r} classified as APPROVAL"
def test_english_baseline_still_works(cfg):
config = cfg()
kind, _ = clf.classify("Your application was approved, welcome to the program", config)
assert kind is APPROVAL_ONLY
@pytest.mark.parametrize(
"text",
[
# 1. rejection stem
"Lamentamos informarle que su solicitud fue rechazada",
# 2. nunca
"Nunca aceptamos negocios de criptomonedas",
# 3. jamás
"Jamás hemos trabajado con affiliates externos",
# 4. tampoco
"No podemos aceptar su sitio, tampoco podemos ofrecerle otro acuerdo",
# 5. ningún/ninguna
"En este momento no tenemos ningún hueco para nuevos partners",
# 6. no podemos (multi-word)
"Su propuesta es interesante pero no podemos proceder",
# 7. no es posible (multi-word)
"Gracias por escribir, pero no es posible colaborar este trimestre",
# 8. lamentablemente
"Lamentablemente su aplicación no ha sido aprobada",
# 9. no aceptamos
"Por política interna no aceptamos sitios de viajes",
# 10. mixed spanish/english: negation in spanish, approval word in english
"Thanks for reaching out! Lamentablemente we cannot approve your site",
# 11. mixed: approval in spanish, negation in english
"Apreciamos su interés, but we are not accepting new partners",
# 12. nadie / nada
"Aquí nadie decidiría eso, así que no hay nada que aprobar",
],
)
def test_spanish_and_mixed_negations_veto_approval(cfg, text):
_never_approval(cfg, text)
def test_spanish_pure_approval_still_approves(cfg):
"""A clean Spanish approval WITHOUT any negation must still classify
approval — the veto must not over-fire."""
config = cfg()
kind, confidence = clf.classify(
"Su solicitud fue aprobada. Bienvenido al programa, estamos encantados "
"de trabajar con ustedes.",
config,
)
assert kind is APPROVAL_ONLY
assert confidence >= 0.5
def test_spanish_rejection_classifies_rejection(cfg):
config = cfg()
kind, conf = clf.classify(
"Lamentamos informarle que su solicitud fue rechazada por el comité",
config,
)
assert kind is MailClassification.REJECTION
assert conf >= 0.9