Compare commits

..
Author SHA1 Message Date
coder-agent 91480ebb1a politica de DMs r33: urgent-watch solo 4 clases + tope 3/dia + dedup tipo:sujeto 24h + backup 2 dias; reporte diario con conteo de DMs y diferidos
ci / lint-test (pull_request) Successful in 1s
isolation-test / probe (pull_request) Successful in 1s
2026-11-03 16:23:08 +00:00
ci-bot 6e16965e9b Merge pull request 'autodeploy r33: identidad hardcap condicional + DM unico por incidente' (#65) from r33-autodeploy-identidad-dedup into main
ci / lint-test (push) Successful in 2s
2026-11-03 16:20:03 +00:00
coder-agent 50f1e5e1b7 autodeploy r33: identidad de hardcap solo si el layout sirve la meta build-id + DM unico por incidente (dm_once/dm_clear)
ci / lint-test (pull_request) Successful in 7s
isolation-test / probe (pull_request) Successful in 0s
- 34 rollbacks (15:10-16:09 UTC) grep-eando un build-id que main no servia
  (gh#10 anade la meta; mientras, no se exige identidad y el deploy avanza).
- dm_once(subject): un DM por incidente; deploy exitoso limpia el subject.
  El journal sigue completo (say por ciclo).
2026-11-03 16:16:28 +00:00
ci-bot d55d8e91a2 Merge pull request 'autodeploy: rollback de hardcap en subshell (no bloquea bitcoinhotels)' (#64) from r32-autodeploy-subshell into main
ci / lint-test (push) Successful in 2s
2026-11-03 15:55:04 +00:00
coder-agent b9dcacd76e autodeploy: cierre de subshell antes del header de bitcoinhotels (cosmetico)
ci / lint-test (pull_request) Successful in 4s
isolation-test / probe (pull_request) Successful in 1s
2026-11-03 15:50:44 +00:00
coder-agent c17ac080cb autodeploy: seccion hardcap en subshell — su rollback no aborta los deploys de bitcoinhotels
ci / lint-test (pull_request) Successful in 7s
isolation-test / probe (pull_request) Successful in 1s
2026-11-03 15:50:11 +00:00
ci-bot fae57e3b4f Merge pull request 'reporte diario: PRs pendientes de merge + contenido nuevo por sitio + semanal AdSense' (#63) from r32-report into main
ci / lint-test (push) Successful in 2s
2026-11-03 15:20:03 +00:00
coder-agent 3ec9b2cfb3 reporte diario: 3b PRs esperando tu merge + contenido nuevo por sitio con enlaces + línea semanal AdSense (lunes)
ci / lint-test (pull_request) Successful in 12s
isolation-test / probe (pull_request) Successful in 1s
2026-11-03 15:18:12 +00:00
pattern f9e54ad675 Merge pull request 's1: flock autodeploy+auto-merge versionados + guard CI/no-/srv' (#58) from r31-s1-lock-ci into main
ci / lint-test (push) Successful in 23s
Reviewed-on: #58
2026-11-03 14:57:23 +00:00
ci-bot 29225fc293 Merge pull request 'r31 doc: estados finales (verdes) de los PRs' (#61) from r31-doc-final into main
ci / lint-test (push) Successful in 2s
2026-11-03 04:10:03 +00:00
coder-agent 6b1b2f83b7 ci: retrigger (workdir_parent absoluto en el runner)
ci / lint-test (pull_request) Successful in 1s
isolation-test / probe (pull_request) Successful in 0s
2026-11-03 03:10:36 +00:00
coder-agent 5cc3fd6419 s1: autodeploy+auto-merge con flock compartido (versionados) + guard de workflow
ci / lint-test (pull_request) Failing after 1s
isolation-test / probe (pull_request) Failing after 0s
- Incidente 02:06:29Z: el rollback del autodeploy (git reset --hard $OLD)
  pisó la rama e2-fuentes-y-sellos porque el árbol cambió de branch a mitad
  de deploy. Causa: ninguna exclusión mutua entre autodeploy, auto-merger y
  ops manuales sobre los mismos árboles.
- Lock: /run/lock/cayce-git-ops.lock (tmpfiles.d, 0664 root:ubuntu).
  autodeploy flock -n (salta ciclo), auto-merge flock -w 240 (espera).
- Guard en ci.yml: un job cuyo workspace caiga bajo /srv/* revienta.
2026-11-03 03:03:03 +00:00
6 changed files with 627 additions and 0 deletions
+6
View File
@@ -15,6 +15,12 @@ jobs:
run:
shell: bash --noprofile --norc -eo pipefail {0}
steps:
- name: "r31 guard: CI nunca opera en /srv/*"
run: |
case "$PWD" in
/srv/*) echo "VIOLACION: workspace de CI en $PWD"; exit 1 ;;
*) echo "workspace OK: $PWD" ;;
esac
- uses: actions/checkout@v4
with:
fetch-depth: 1
+11
View File
@@ -0,0 +1,11 @@
# /usr/local/bin versionado
Copias exactas de los scripts de infra que viven en /usr/local/bin del VPS.
Actualizar: PR aquí → merge → copiar al VPS (sudo cp) + daemon-reload si toca unit.
- cayce-autodeploy (r31: flock compartido /run/lock/cayce-git-ops.lock)
- cayce-auto-merge (r31: mismo lock, espera hasta 240s)
El lockfile lo crea tmpfiles.d: /etc/tmpfiles.d/cayce-locks.conf
(f /run/lock/cayce-git-ops.lock 0664 root ubuntu -). Ops manuales con git
sobre /srv/*: flock /run/lock/cayce-git-ops.lock <comando>.
+46
View File
@@ -0,0 +1,46 @@
#!/bin/bash
# Auto-merge (ronda-9): SOLO PRs con todos los checks en success y que NO
# toquen paths protegidos. Nunca btcpay-events. Nunca credenciales del operador.
# r19: el token persiste en /etc/cayce/ci-bot.token (600). Regenerar con nombre
# ya usado da ERROR y dejaba BOT vacio (bug historico: el script no hacia nada).
# r31 (S1): lock compartido con cayce-autodeploy — nunca compiten sobre los
# mismos arboles/branches (incidente 02:06:29Z del 2026-11-03).
LOCK=/run/lock/cayce-git-ops.lock
exec 9>"$LOCK" || { echo 'FATAL: no se puede abrir lockfile'; exit 1; }
if ! flock -w 240 9; then
echo "[auto-merge] autodeploy mantiene el lock >240s — paso de este ciclo (el proximo mergea)"
exit 0
fi
BOT=$(cat /etc/cayce/ci-bot.token 2>/dev/null | tr -d '[:space:]')
if [ -z "$BOT" ]; then
BOT=$(sudo -u git /usr/local/bin/gitea --config /etc/gitea/app.ini admin user generate-access-token -u ci-bot -t "auto-merge-$(date +%s)" --scopes write:repository,write:issue 2>/dev/null | grep -aoE "[0-9a-f]{40}" | head -1)
[ -n "$BOT" ] && printf '%s' "$BOT" > /etc/cayce/ci-bot.token && chmod 600 /etc/cayce/ci-bot.token
fi
[ -z "$BOT" ] && { echo 'FATAL: sin token ci-bot'; exit 1; }
G="http://127.0.0.1:3100/api/v1"
# paths protegidos (grep -E contra la lista de ficheros modificados del PR)
PROTECTED="SPEC.md|MANDATE.md|prompts/cayce-identity.md|guard[^/]*\.|relay-watcher|outbox\.mjs|guard-blocklist|check-single-outbox|\.gitea/workflows|\.github/workflows|cayce-auto-merge|act_runner|docker"
for REPO in cayce-ops entity bitcoinhotels; do
curl -s -H "Authorization: token $BOT" "$G/repos/cayce/$REPO/pulls?state=open&limit=20" | python3 -c "
import json,sys
for p in json.load(sys.stdin):
if p.get('title','').upper().startswith('WIP'): continue
print(p['number'], p['head']['sha'])" 2>/dev/null | while read N SHA; do
FILES=$(curl -s -H "Authorization: token $BOT" "$G/repos/cayce/$REPO/pulls/$N/files?limit=100" | python3 -c "
import json,sys
for f in json.load(sys.stdin): print(f.get('filename',''))" 2>/dev/null)
if echo "$FILES" | grep -qE "$PROTECTED"; then
echo "$REPO #$N: TOCA PATH PROTEGIDO — no se auto-mergea"
continue
fi
# r19: /status (combinado) en vez de /statuses — la lista incluye los
# 'pending' historicos y hacia que un PR verde pareciera no verde
OK=$(curl -s -H "Authorization: token $BOT" "$G/repos/cayce/$REPO/commits/$SHA/status" | python3 -c "
import json,sys
d = json.load(sys.stdin)
print('yes' if isinstance(d, dict) and d.get('state') == 'success' else 'no')" 2>/dev/null)
if [ "$OK" = "yes" ]; then
curl -s -X POST -H "Authorization: token $BOT" -H "Content-Type: application/json" -d '{"Do":"merge"}' "$G/repos/cayce/$REPO/pulls/$N/merge" -o /dev/null -w "$REPO #$N (green, sin paths protegidos) merge: %{http_code}\n"
fi
done
done
+200
View File
@@ -0,0 +1,200 @@
#!/bin/bash
# cayce-autodeploy — r18: despliega main nuevo SIN intervención del operador.
# ~/cayce-ops → git pull de main (docs, prompts; el despliegue lee el tree)
# /srv/cayce-core → checkout de cayce/entity main, con PUBLISH_OFF activo
# durante el swap. Solo se retira si: tests del guard
# 27+/27 y watcher sin VIOLACION. Rollback automatico.
# Timer: cada 3 min. Fallo → DM al supervisor + logger. Nunca toca un tree sucio.
say() { echo "[autodeploy] $*"; logger -t cayce-autodeploy "$*"; }
# r33: un rollback repetido avisa UNA vez por incidente (subject = repo:causa);
# el journal sigue completo (say por ciclo). Un deploy exitoso limpia el subject.
dm_once() { # $1 subject, $2 mensaje
local flag="/var/lib/cayce/autodeploy-alerts/$1"
if [ -e "$flag" ]; then say "DM suprimido (incidente ya avisado): $1"; return 0; fi
mkdir -p /var/lib/cayce/autodeploy-alerts 2>/dev/null || true
touch "$flag" 2>/dev/null || true
dm "$2"
}
dm_clear() { rm -f "/var/lib/cayce/autodeploy-alerts/$1" 2>/dev/null || true; }
dm() {
SUP=$(grep -aoE 'SUPERVISOR_NPUB_HEX=[0-9a-f]+' /srv/cayce-nostr/.env 2>/dev/null | cut -d= -f2)
[ -n "$SUP" ] && curl -s -X POST http://127.0.0.1:3011/dm -H "Content-Type: application/json" \
-d "{\"to_hex\":\"$SUP\",\"content\":\"AUTODEPLOY: $1\"}" >/dev/null 2>&1
}
# r31 (S1): lock compartido con cayce-auto-merge — nunca compiten sobre los
# mismos arboles/branches (incidente 02:06:29Z del 2026-11-03: un rollback
# `git reset --hard $OLD` pisó una rama de trabajo porque el árbol cambió de
# branch a mitad de deploy). Las ops manuales con git sobre /srv/* tambien
# lo toman: flock /run/lock/cayce-git-ops.lock <comando>
LOCK=/run/lock/cayce-git-ops.lock
exec 9>"$LOCK" || { say 'FATAL: no se puede abrir lockfile'; exit 1; }
if ! flock -n 9; then
say "auto-merge (u otra op) mantiene el lock — paso de este ciclo"
exit 0
fi
# ── 1. cayce-ops (main) ──────────────────────────────────────────────────────
cd /home/ubuntu/cayce-ops || { say "FATAL: sin ~/cayce-ops"; exit 1; }
git fetch -q origin 2>/dev/null
if [ "$(git branch --show-current)" = "main" ] && ! git merge-base --is-ancestor origin/main HEAD 2>/dev/null; then
if [ -n "$(git status --porcelain)" ]; then
say "ABORT: cayce-ops tree sucio"; dm "abortado: cayce-ops dirty"
elif git pull -q --ff-only origin main; then
say "cayce-ops → $(git rev-parse --short HEAD)"
# r21: sin DM (no urgente) — dm "cayce-ops actualizado a $(git rev-parse --short HEAD)"
else
say "ERROR: pull cayce-ops"; dm "error pull cayce-ops"
fi
fi
# ── 2. entity → /srv/cayce-core (con regla de urgencia) ──────────────────────
cd /srv/cayce-core || { say "FATAL: sin /srv/cayce-core"; exit 1; }
git fetch -q origin 2>/dev/null
# desplegar solo si main tiene commits que NO estan en el checkout
# r20: sin exit 0 — mataba el script y la sección bitcoinhotels nunca corría
if ! git merge-base --is-ancestor origin/main HEAD 2>/dev/null; then
# r25: lessons.md lo muta el ciclo retro en produccion — preservar (stash/pop).
# Cualquier OTRO fichero sucio si aborta.
DIRTY_OTHER=$(git status --porcelain | grep -v " lessons.md" | head -3)
if [ -n "$DIRTY_OTHER" ]; then
say "ABORT: /srv/cayce-core tree sucio: $DIRTY_OTHER"; dm "abortado: /srv/cayce-core dirty ($DIRTY_OTHER) — deploy pendiente"
exit 1
fi
git stash push -q -m "autodeploy-lessons" -- lessons.md 2>/dev/null || true
OLD=$(git rev-parse HEAD)
echo "autodeploy $(date -u +%FT%TZ)" > PUBLISH_OFF # r21: firmado para que urgent-watch no lo tome por manual
say "main nuevo detectado: $(git rev-parse --short HEAD) → $(git rev-parse --short origin/main) (PUBLISH_OFF activo)"
if ! git checkout -q main || ! git reset -q --hard origin/main; then
git reset -q --hard "$OLD" 2>/dev/null; rm -f PUBLISH_OFF
say "FATAL: checkout entity"; dm "FATAL checkout entity — deploy cancelado, sin cambios"
exit 1
fi
# verificación 1: tests del guard
OUT=$(cd /srv/cayce-core && node --test guard.test.mjs 2>&1)
PASS=$(echo "$OUT" | grep -c '^ok ')
FAIL=$(echo "$OUT" | grep -c '^not ok ')
if [ "$FAIL" != "0" ] || [ "$PASS" -lt 27 ]; then
git reset -q --hard "$OLD"
git stash pop -q 2>/dev/null || true
say "ROLLBACK: tests $PASS ok / $FAIL fail (<27 o fail>0)"
dm "ROLLBACK en deploy entity: tests $PASS/$FAIL. main NO desplegado; runtime en $(git rev-parse --short HEAD)"
exit 1
fi
# verificación 2: daemon + watcher limpio
sudo -n systemctl restart cayce-nostr.service
sudo -n systemctl start cayce-relay-watcher.service
W=0
for i in 1 2 3 4 5 6; do
sleep 5
[ "$(systemctl is-active cayce-relay-watcher.service)" = "inactive" ] && break
done
if systemctl is-failed cayce-relay-watcher.service >/dev/null 2>&1 \
|| journalctl -u cayce-relay-watcher -n 8 --no-pager 2>/dev/null | grep -q VIOLACION; then
git reset -q --hard "$OLD"
sudo -n systemctl restart cayce-nostr.service
say "ROLLBACK: watcher con VIOLACION — PUBLISH_OFF queda ACTIVO"
dm "ROLLBACK watcher en deploy entity. PUBLISH_OFF QUEDA ACTIVO — revision manual"
exit 1 # ojo: PUBLISH_OFF se queda (fail-safe deliberado)
fi
rm -f PUBLISH_OFF
git stash pop -q 2>/dev/null || true # lessons.md del ciclo retro
say "entity → $(git rev-parse --short HEAD) (tests $PASS/27, watcher OK, live)"
# r21: sin DM (no urgente) — dm "entity desplegado: $(git rev-parse --short HEAD) — tests $PASS/27, watcher OK"
fi # fin sección entity
# r32: la seccion hardcap corre en SUBSHELL — un ROLLBACK (exit 1) ya no
# aborta los deploys de bitcoinhotels (bug: rollback de hardcap llevaba
# bloqueando bh desde las 15:10 mientras gh#10 espera merge)
(
# ── 2b. hardcap-news → /srv/hardcap-news (GitHub; web, sin PUBLISH_OFF) ─────
cd /srv/hardcap-news || { say "FATAL: sin /srv/hardcap-news"; exit 1; }
git fetch -q origin 2>/dev/null
if ! git merge-base --is-ancestor origin/main HEAD 2>/dev/null && [ -z "$(git status --porcelain)" ]; then
OLD=$(git rev-parse HEAD)
if ! git checkout -q main || ! git reset -q --hard origin/main; then
git reset -q --hard "$OLD" 2>/dev/null; say "FATAL: checkout hardcap-news"; exit 1
fi
if git diff --name-only "$OLD" HEAD 2>/dev/null | grep -q "^frontend/package-lock.json$"; then
(cd frontend && npm ci --ignore-scripts >/dev/null 2>&1) || say "WARN: npm ci hardcap"
fi
if ! (cd frontend && npm run build >/tmp/hc-autodeploy-build.log 2>&1); then
git reset -q --hard "$OLD"; (cd frontend && npm run build >/dev/null 2>&1); sudo -n systemctl restart hardcap-frontend.service
say "ROLLBACK hardcap-news: fallo el build"; dm_once "hardcap:rollback-build" "ROLLBACK hardcap-news (build) — main NO desplegado, rollback activo (aviso unico)"
exit 1
fi
sudo -n systemctl restart hardcap-frontend.service
sleep 6
CODE=$(curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:3003/)
if [ "$CODE" != "200" ]; then
git reset -q --hard "$OLD"; (cd frontend && npm run build >/dev/null 2>&1); sudo -n systemctl restart hardcap-frontend.service
say "ROLLBACK hardcap-news: verificacion (code=$CODE)"; dm_once "hardcap:rollback-code" "ROLLBACK hardcap-news verificacion: code=$CODE — rollback activo (aviso unico)"
exit 1
fi
# r24: identidad por build-id (cuando el despliegue trae la función)
# r33: la identidad de hardcap solo se exige si ESTE main renderiza la meta
# build-id (gh#10 la anade al layout). Mientras no exista, no exigir identidad
# (loop del 2026-11-03: 34 rollbacks grep-eando un build-id que main no servia).
if git grep -q "name=.build-id." -- frontend/src/app/layout.tsx 2>/dev/null; then
BUILT=$(curl -s -m 8 http://127.0.0.1:3003/ | grep -oE 'name="build-id" content="[a-f0-9]{8}"' | grep -oE '[a-f0-9]{8}' | head -1)
WANT=$(git rev-parse --short=8 HEAD)
if [ "$BUILT" != "$WANT" ]; then
git reset -q --hard "$OLD"; (cd frontend && npm run build >/dev/null 2>&1); sudo -n systemctl restart hardcap-frontend.service
say "ROLLBACK hardcap-news: build-id '$BUILT' != '$WANT'"; dm_once "hardcap:build-id" "ROLLBACK hardcap-news: servia build '$BUILT' pero lo desplegado era '$WANT'. Hasta mergear gh#10 (meta build-id) el check de identidad no se exige; aviso unico."
exit 1
fi
dm_clear "hardcap:build-id"; dm_clear "hardcap:rollback-build"; dm_clear "hardcap:rollback-code"
say "identidad hardcap OK: servido=$BUILT desplegado=$WANT"
fi
dm_clear "hardcap:build-id"; dm_clear "hardcap:rollback-build"; dm_clear "hardcap:rollback-code"
say "hardcap-news → $(git rev-parse --short HEAD) (200)"
fi
)
# ── 3. bitcoinhotels → /srv/bitcoinhotels (web; sin PUBLISH_OFF) ────────────
cd /srv/bitcoinhotels || { say "FATAL: sin /srv/bitcoinhotels"; exit 1; }
git fetch -q origin 2>/dev/null
if ! git merge-base --is-ancestor origin/main HEAD 2>/dev/null && [ -z "$(git status --porcelain)" ]; then
OLD=$(git rev-parse HEAD)
if ! git checkout -q main || ! git reset -q --hard origin/main; then
git reset -q --hard "$OLD" 2>/dev/null; say "FATAL: checkout bitcoinhotels"; dm "FATAL checkout bitcoinhotels — sin cambios"; exit 1
fi
if git diff --name-only "$OLD" HEAD 2>/dev/null | grep -q "^package-lock.json$"; then
npm ci --ignore-scripts >/dev/null 2>&1 || { say "WARN: npm ci bitcoinhotels"; }
fi
if ! npm run build >/tmp/bh-autodeploy-build.log 2>&1; then
git reset -q --hard "$OLD"; npm run build >/dev/null 2>&1; systemctl restart cayce-bitcoinhotels
say "ROLLBACK bitcoinhotels: fallo el build"; dm "ROLLBACK bitcoinhotels (build) — main NO desplegado, rollback activo"
exit 1
fi
sudo -n systemctl restart cayce-bitcoinhotels
sleep 6
CODE=$(curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:3002/)
UMAMI=$(curl -s http://127.0.0.1:3002/ | grep -c umami)
MARCAS=$(curl -s http://127.0.0.1:3002/guides/how-to-pay-with-lightning-abroad | grep -cE "Trezor|Ledger|BitBox|Coldcard")
if [ "$CODE" != "200" ] || [ "$UMAMI" = "0" ] || [ "$MARCAS" != "0" ]; then
git reset -q --hard "$OLD"; npm run build >/dev/null 2>&1; systemctl restart cayce-bitcoinhotels
say "ROLLBACK bitcoinhotels: verificacion (code=$CODE umami=$UMAMI marcas=$MARCAS)"
dm "ROLLBACK bitcoinhotels verificacion: code=$CODE umami=$UMAMI marcas=$MARCAS — rollback activo"
exit 1
fi
say "bitcoinhotels → $(git rev-parse --short HEAD) (200 + umami + 0 marcas)"
# r22: verificación de IDENTIDAD — lo servido debe ser lo desplegado
if git grep -q "NEXT_PUBLIC_BUILD_ID" -- src/app/layout.tsx 2>/dev/null; then
BUILT=$(curl -s -m 8 http://127.0.0.1:3002/ | grep -oE 'name="build-id" content="[a-f0-9]{8}"' | grep -oE '[a-f0-9]{8}' | head -1)
WANT=$(git rev-parse --short=8 HEAD)
if [ "$BUILT" != "$WANT" ]; then
git reset -q --hard "$OLD"; npm run build >/dev/null 2>&1; sudo -n systemctl restart cayce-bitcoinhotels
say "ROLLBACK bitcoinhotels: build-id servido '$BUILT' != desplegado '$WANT'"
dm "ROLLBACK bitcoinhotels: el sitio servia build '$BUILT' pero lo desplegado era '$WANT' — rollback activo."
exit 1
fi
say "identidad OK: servido=$BUILT desplegado=$WANT"
fi
fi
+226
View File
@@ -0,0 +1,226 @@
#!/bin/bash
# cayce-daily-report (r20) — DM diario al supervisor:
# 1) publicado por canal con enlaces 2) rechazos del guard y por qué
# 3) PRs auto-mergeados + deploys 4) métricas Umami (ambos sitios)
# 5) costo estimado de Gemini del día
set -uo pipefail
OUT=$(mktemp)
PSQL="sudo -n -u postgres psql -d cayce -tA -f-"
DAY=$(date -u +%F)
{
echo "CAYCE — reporte diario $DAY (UTC)"
echo ""
echo "== 1. Publicado (24h) =="
$PSQL <<'EOF'
SELECT 'nostr/' || type || ' — ' || left(regexp_replace(content, E'[\\n\\r]+', ' ', 'g'), 80)
|| ' — ev:' || coalesce(meta->>'event_id', '(sin id)')
FROM outbound_drafts
WHERE status='published' AND channel='nostr'
AND type NOT IN ('profile','contact_list','inbox_relays','deletion')
AND ts > now() - interval '24 hours'
ORDER BY ts DESC LIMIT 6;
SELECT 'hardcap/' || type || ' — ' || left(regexp_replace(content, E'[\\n\\r]+', ' ', 'g'), 80)
FROM outbound_drafts
WHERE status='published' AND channel='hardcap' AND ts > now() - interval '24 hours'
ORDER BY ts DESC LIMIT 4;
EOF
sudo -n -u postgres psql -d cayce -tA -f- <<'EOF'
SELECT platform || ' — ' || theme || ' — ' || coalesce(post_url, '(sin url)')
FROM posts_log WHERE ts > now() - interval '24 hours' AND status='ok' ORDER BY ts DESC LIMIT 6;
EOF
echo ""
echo "== 2. Guard: rechazos (24h) =="
sudo -n -u postgres psql -d cayce -tA -f- <<'EOF'
SELECT count(*) || 'x ' || left(reasons, 100) FROM guard_rejections
WHERE ts > now() - interval '24 hours'
GROUP BY reasons ORDER BY count(*) DESC LIMIT 6;
EOF
echo ""
echo "== 2b. DMs urgentes (politica r33) =="
NDM=$(journalctl -u cayce-nostr --since "-24 hours" --no-pager 2>/dev/null | grep -c "\[dm\] envio a")
UCOUNT=$(python3 -c "import json; print(json.load(open('/srv/cayce-core/.dm-urgent-count.json')).get('$(date -u +%F)', 0))" 2>/dev/null || echo 0)
echo " DMs NIP-04 enviados (24h, todas las causas): $NDM · urgentes de hoy (tope 3): $UCOUNT"
if [ -s /srv/cayce-core/.dm-deferred.log ]; then
echo " diferidos por tope (van aqui en vez de DM):"
tail -5 /srv/cayce-core/.dm-deferred.log | sed "s/^/ /"
fi
if [ -f /srv/cayce-core/PUBLISH_OFF ]; then
echo " kill switch: ACTIVO ($(head -c 20 /srv/cayce-core/PUBLISH_OFF 2>/dev/null))"
fi
echo ""
echo "== 3. PRs y deploys =="
BTOK=$(cat /etc/cayce/ci-bot.token 2>/dev/null)
if [ -n "$BTOK" ]; then
for REPO in cayce-ops entity bitcoinhotels; do
curl -s -m 10 -H "Authorization: token $BTOK" \
"http://127.0.0.1:3100/api/v1/repos/cayce/$REPO/pulls?state=closed&limit=20" \
| python3 -c "
import json, sys, datetime
cutoff = (datetime.datetime.utcnow() - datetime.timedelta(days=1)).strftime('%Y-%m-%dT%H:%M')
for p in json.load(sys.stdin):
m = (p.get('merged_at') or '')
if not p.get('merged') or not m or m < cutoff: continue
print(f\"$REPO#{p['number']} — {p['title'][:60]}\")" 2>/dev/null
done
fi
journalctl -u cayce-autodeploy --since "-24 hours" -o cat --no-pager 2>/dev/null | grep -E "^\[autodeploy\].*(→|ROLLBACK|ABORT|FATAL)" | tail -6
echo ""
echo "== 3b. PRs que esperan tu merge =="
PROTECTED="SPEC.md|MANDATE.md|prompts/cayce-identity.md|guard[^/]*\.|relay-watcher|outbox\.mjs|guard-blocklist|check-single-outbox|\.gitea/workflows|\.github/workflows|cayce-auto-merge|act_runner|docker"
if [ -n "${BTOK:-}" ]; then
for REPO in cayce-ops entity bitcoinhotels; do
curl -s -m 10 -H "Authorization: token $BTOK" \
"http://127.0.0.1:3100/api/v1/repos/cayce/$REPO/pulls?state=open&limit=20" \
| python3 -c "
import json, sys, urllib.request, os
tok = os.environ.get('BTOK')
for p in json.load(sys.stdin):
req = urllib.request.Request(f'http://127.0.0.1:3100/api/v1/repos/cayce/$REPO/pulls/{p["number"]}/files?limit=100', headers={'Authorization': 'token ' + tok})
files = [f.get('filename','') for f in json.load(urllib.request.urlopen(req, timeout=10))]
import re
prot = [f for f in files if re.search(r'$PROTECTED', f)]
motivo = 'toca paths protegidos: ' + ', '.join(prot[:3]) if prot else 'CI pendiente o esperando revisión'
print(f" $REPO#{p['number']} — {p['title'][:56]} — {motivo}")" 2>/dev/null
done
fi
echo " GitHub patternn/hardcap-news: sin token en VPS — revisar https://github.com/patternn/hardcap-news/pulls"
echo ""
echo "== 4. Umami (24h) =="
sudo -n -u postgres psql -d umami -tA -f- <<'EOF'
SELECT w.name || ': ' || count(DISTINCT e.session_id) || ' visitas · ' || count(*) || ' páginas'
FROM website_event e JOIN website w ON w.website_id = e.website_id
WHERE e.created_at > now() - interval '24 hours' AND e.event_type = 1
GROUP BY w.name ORDER BY count(*) DESC;
SELECT 'top: ' || url_path || ' (' || count(*) || ')' FROM website_event
WHERE created_at > now() - interval '24 hours' AND event_type = 1
GROUP BY url_path ORDER BY count(*) DESC LIMIT 4;
SELECT 'refs: ' || coalesce(nullif(referrer_domain, ''), '(direct)') || ' (' || count(DISTINCT session_id) || ')'
FROM website_event
WHERE created_at > now() - interval '24 hours' AND event_type = 1
GROUP BY referrer_domain ORDER BY count(DISTINCT session_id) DESC LIMIT 4;
EOF
echo ""
echo "== Contenido nuevo publicado hoy por sitio =="
# bitcoinhotels: guias/fichas nuevas o actualizadas en el arbol DESPLEGADO (link vivo)
cd /srv/bitcoinhotels 2>/dev/null && git log --since="24 hours" --format="%H" -- src/data/guides.json | while read -r C; do
git show "$C" -- src/data/guides.json 2>/dev/null | grep -oE '^\+\s+"slug": "[a-z0-9-]+"' | sed 's/.*: "//;s/"//' \
| while read -r SL; do echo " bitcoinhotels.org/guides/$SL"; done
done | sort -u
# hardcap: notas publicadas hoy (link por article_id del meta)
sudo -n -u postgres psql -d cayce -tA -f- <<'EOF2' | while read -r L; do [ -n "$L" ] && echo " hardcap.news$L"; done
SELECT '/news/' || (meta->>'article_id') || '/summary — ' || left(regexp_replace(content, E'[\n\r]+', ' ', 'g'), 60)
FROM outbound_drafts
WHERE channel='hardcap' AND status='published' AND ts > now() - interval '24 hours'
AND meta->>'article_id' IS NOT NULL
ORDER BY ts DESC LIMIT 6;
EOF2
echo ""
echo "== Ciclos: piezas producidas (24h) =="
sudo -n -u postgres psql -d cayce -tA -f- <<'EOF'
SELECT 'posts: ' || count(*) || ' piezas nostr (opinion/reflection/events/iniciativa)'
FROM outbound_drafts WHERE channel='nostr'
AND type IN ('opinion','reflection','events_post','initiative_note')
AND status='published' AND ts > now() - interval '24 hours';
SELECT 'editoriales: ' || count(*) || ' opiniones publicadas · ' || (SELECT count(*) FROM actions_log WHERE action='editorials_cycle' AND ts > now() - interval '24 hours') || ' corridas'
FROM outbound_drafts WHERE channel='nostr' AND type='opinion' AND status='published' AND ts > now() - interval '24 hours';
SELECT 'notas hardcap: ' || count(*) || ' publicadas'
FROM outbound_drafts WHERE channel='hardcap' AND status='published' AND ts > now() - interval '24 hours';
SELECT 'curacion hardcap: ' || count(*) || ' corridas del ciclo core · ' || (SELECT count(*) FROM digests WHERE ts > now() - interval '24 hours') || ' digests'
FROM actions_log WHERE action='cycle_complete' AND ts > now() - interval '24 hours';
SELECT 'promo: ' || count(*) FILTER (WHERE status='ok') || ' publicadas · ' || count(*) FILTER (WHERE status LIKE 'rejected%') || ' rechazadas · ' || count(*) FILTER (WHERE status LIKE 'error%') || ' con error'
FROM posts_log WHERE ts > now() - interval '24 hours';
SELECT 'iniciativa: ' || count(*) || ' drafts creados'
FROM actions_log WHERE action='draft_created' AND ts > now() - interval '24 hours';
EOF
echo "== Checklist de postulacion (verificable en vivo) =="
chk() { # $1 nombre, $2 condicion (0=verde)
if [ "$2" = "0" ]; then echo " $1: verde"; else echo " $1: pendiente"; fi
}
for SITE in bh hc; do
[ "$SITE" = "bh" ] && P=3002 && D=bitcoinhotels.org || { P=3003; D=hardcap.news; }
N=$(curl -s -m 6 http://127.0.0.1:$P/sitemap.xml | grep -c "<loc>")
TR=0; for PG in about contact privacy terms; do C=$(curl -s -o /dev/null -w "%{http_code}" -m 5 http://127.0.0.1:$P/$PG); [ "$C" = "200" ] && TR=$((TR+1)); done
HWW=$(curl -s -o /dev/null -w "%{http_code}" -m 5 http://127.0.0.1:$P/how-we-work)
ADS=$(curl -s -o /dev/null -w "%{http_code}" -m 5 http://127.0.0.1:$P/ads.txt)
PC=$(curl -s -o /dev/null -w "%{http_code}" -m 5 http://127.0.0.1:$P/privacy)
PRIV=$(curl -s -m 5 http://127.0.0.1:$P/privacy | grep -c -i "adsense"); [ "$PC" != "200" ] && PRIV=0
echo "$D: sitemap=$N URLs · confianza $TR/4 · how-we-work $([ "$HWW" = "200" ] && echo SI || echo NO) · ads.txt $([ "$ADS" = "200" ] && echo SI || echo NO) · privacy-AdSense $([ "$PRIV" -ge 1 ] && echo SI || echo NO)"
done
if [ "$(date -u +%u)" = "1" ]; then
echo "== Semanal (lunes): checklist AdSense =="
for SITE in bh hc; do
[ "$SITE" = "bh" ] && P=3002 && N=bitcoinhotels.org || { P=3003; N=hardcap.news; }
OK=0; TOTAL=5
for PG in sitemap.xml ads.txt privacy terms how-we-work; do
C=$(curl -s -o /dev/null -w "%{http_code}" -m 5 http://127.0.0.1:$P/$PG); [ "$C" = "200" ] && OK=$((OK+1))
done
echo " $N: $OK/$TOTAL (sitemap · ads.txt · privacy · terms · how-we-work)"
done
fi
echo "== Pushes (24h) por autor =="
for R in cayce-ops entity bitcoinhotels; do
A=$(cd /srv/$([ "$R" = "cayce-ops" ] && echo home/ubuntu/cayce-ops || { [ "$R" = "entity" ] && echo cayce-core || echo bitcoinhotels; }) 2>/dev/null && git log --since="24 hours" --format="%an" | sort | uniq -c | tr "\n" ";" )
echo " $R: ${A:-sin commits}"
done
echo "== Avance semanal FASE B (piezas vs objetivo) =="
# convencion: los commits de piezas FASE B llevan prefijo "pieza:"
SEM=$(( ($(date -u +%s) / 86400 + 3) / 7 )) # semana ISO aprox
for R in cayce-ops entity bitcoinhotels; do
D=/srv/$([ "$R" = "cayce-ops" ] && echo home/ubuntu/cayce-ops || { [ "$R" = "entity" ] && echo cayce-core || echo bitcoinhotels; })
N=$(cd "$D" 2>/dev/null && git log --since="7 days" --format="%s" | grep -c "^pieza:")
echo " $R: $N piezas-commits esta semana"
done
echo " objetivos: bitcoinhotels 2/sem · hardcap ~2,5/sem (hub+watch+guia alterna)"
echo "== Persistencias (caidos ahora mismo) =="
for U in cayce-nostr.service hardcap-backend.service hardcap-frontend.service cayce-bitcoinhotels.service postgresql@18-main.service caddy.service; do
A=$(systemctl is-active $U 2>/dev/null)
[ "$A" != "active" ] && echo " $U: $A"
done
true
echo "== 5. Gemini del día =="
python3 - /srv/cayce-core/.llm-usage.jsonl <<'PY' 2>/dev/null || echo "(sin uso registrado aún)"
import json, sys, os
IN = OUT_ = CACHED = CALLS = 0
if os.path.exists(sys.argv[1]):
for line in open(sys.argv[1]):
try:
d = json.loads(line)
if d.get("ts", "")[:10] == __import__("datetime").datetime.utcnow().date().isoformat():
CALLS += 1; IN += d.get("in", 0); OUT_ += d.get("out", 0); CACHED += d.get("cached", 0)
except Exception:
pass
rin = float(os.environ.get("GEMINI_USD_PER_MTOK_IN", "0.15"))
rout = float(os.environ.get("GEMINI_USD_PER_MTOK_OUT", "0.60"))
cost = IN/1e6*rin + OUT_/1e6*rout
print(f"{CALLS} llamadas · in {IN:,} tok · out {OUT_:,} tok · cached {CACHED:,} · costo estimado ${cost:.3f} (rates {rin}/{rout} USD/Mtok)")
PY
} > "$OUT" 2>&1
# DM al supervisor + copia de auditoría
SUP=$(grep -aoE 'SUPERVISOR_NPUB_HEX=[0-9a-f]+' /srv/cayce-nostr/.env 2>/dev/null | cut -d= -f2)
BODY=$(head -c 6000 "$OUT")
if [ -n "$SUP" ]; then
curl -s -m 20 -X POST http://127.0.0.1:3011/dm -H "Content-Type: application/json" \
-d "$(python3 -c "import json,sys; print(json.dumps({'to_hex': '$SUP', 'content': open('$OUT').read()[:6000]}))")" \
>/dev/null && logger -t cayce-daily-report "DM enviado ($(wc -c < "$OUT") bytes)"
else
logger -t cayce-daily-report "SIN supervisor — reporte solo en log"
fi
cat "$OUT"
cp "$OUT" /var/log/cayce-daily-report.log 2>/dev/null || true
rm -f "$OUT"
+138
View File
@@ -0,0 +1,138 @@
#!/bin/bash
# cayce-urgent-watch (r21 → r33) — política de DMs del operador:
# DM inmediato SOLO si:
# · kill switch activado o fuga detectada
# · sitio/servicio público caído >15 min (no rollbacks que se recuperan)
# · backup fallido 2 días seguidos
# · gasto de Gemini por encima del umbral
# Todo lo demás (initiative, rollbacks recuperados, ciclos, publicaciones,
# rechazos, merges CODEOWNERS) va SOLO al resumen diario de las 23:50 UTC.
# Ciclos pausados a propósito (p.ej. promo) no generan alertas.
# Dedup por tipo de incidente + sujeto (24 h). Tope: 3 DMs urgentes/día;
# a partir de ahí, una línea en el resumen diario (.dm-deferred.log).
# DRYRUN=1 imprime qué haría sin enviar ni contar.
DEDUP=/srv/cayce-core/.dm-dedup.json
DOWNST=/srv/cayce-core/.critical-down.json
COUNTF=/srv/cayce-core/.dm-urgent-count.json
DEFER=/srv/cayce-core/.dm-deferred.log
USAGE=${LLM_USAGE_LOG:-/srv/cayce-core/.llm-usage.jsonl}
GEMINI_MAX_USD=${GEMINI_MAX_USD:-1.0}
export TTL_MS=$((24 * 3600 * 1000))
DRY=${DRYRUN:-0}
dm_urgente() { # $1 clave (tipo:sujeto), $2 mensaje
python3 - "$1" "$2" "$DEDUP" "$TTL_MS" "$COUNTF" "$DEFER" "$DRY" <<'PY'
import json, sys, subprocess, datetime, os
key, msg, dedup, ttl, countf, defer, dry = sys.argv[1:8]
ttl, dry = int(ttl), int(dry)
st = {}
try: st = json.load(open(dedup))
except Exception: pass
now = datetime.datetime.utcnow().timestamp() * 1000
if now - st.get(key, 0) < ttl:
print(f"[urgent] DM dedup ({key}) — ya alertado en <24h"); sys.exit(0)
cnt = {}
today = datetime.datetime.utcnow().date().isoformat()
try: cnt = json.load(open(countf))
except Exception: pass
n = cnt.get(today, 0)
sup = ""
for l in open("/srv/cayce-nostr/.env"):
if l.startswith("SUPERVISOR_NPUB_HEX="): sup = l.strip().split("=",1)[1]
if not sup:
print("[urgent] sin supervisor — solo log"); sys.exit(0)
if n >= 3:
with open(defer, "a") as f:
f.write(f"{datetime.datetime.utcnow().isoformat()} DEFERIDO (tope 3/dia) {key}: {msg}\n")
print(f"[urgent] TOPE 3/dia alcanzado — diferido al resumen diario ({key})")
sys.exit(0)
if dry:
print(f"[urgent] DRYRUN: enviaria DM ({key}): {msg[:100]}")
sys.exit(0)
subprocess.run(["curl","-s","-m","20","-X","POST","http://127.0.0.1:3011/dm",
"-H","Content-Type: application/json","-d",json.dumps({"to_hex":sup,"content":msg})],
capture_output=True)
st[key] = now
cnt[today] = n + 1
json.dump(st, open(dedup, "w"))
json.dump(cnt, open(countf, "w"))
print(f"[urgent] DM enviado ({key}) — {cnt[today]}/3 hoy")
PY
}
# 1) kill switch manual o fuga: PUBLISH_OFF sin violación reciente del watcher,
# o violación del relay-watcher (fuga) en la última hora.
if [ -f /srv/cayce-core/PUBLISH_OFF ]; then
SIG=$(head -c 9 /srv/cayce-core/PUBLISH_OFF 2>/dev/null)
AGE=$(( $(date -u +%s) - $(stat -c %Y /srv/cayce-core/PUBLISH_OFF 2>/dev/null || echo 0) ))
if [ "$SIG" != "autodeploy" ] || [ "$AGE" -ge 600 ]; then
V=$(sudo -n -u postgres psql -d cayce -tAc "SELECT count(*) FROM guard_rejections WHERE type='watcher_violation' AND ts > now() - interval '15 minutes'" 2>/dev/null || echo 1)
[ "$V" = "0" ] && dm_urgente "killswitch:manual" "URGENTE: PUBLISH_OFF activo (kill switch MANUAL) — publicación en draft global. Sin violación del watcher que lo explique."
fi
fi
FUGA=$(sudo -n -u postgres psql -d cayce -tAc "SELECT count(*) FROM guard_rejections WHERE type='watcher_violation' AND ts > now() - interval '60 minutes'" 2>/dev/null || echo 0)
[ "$FUGA" != "0" ] && dm_urgente "fuga:watcher" "URGENTE: $FUGA violaciones del watcher en 60 min — posible fuga de publicación. PUBLISH_OFF recomendado."
# 2) servicios críticos caídos >15 min (estado persistente de primera caída)
while IFS=: read -r TAG UNIT MINS; do
[ "$TAG" = "DOWN" ] && dm_urgente "down:$UNIT" "URGENTE: servicio crítico caído ${MINS} min: $UNIT"
done < <(python3 - "$DOWNST" <<'PY'
import json, subprocess, sys, time
UNITS = ["cayce-nostr.service","hardcap-backend.service","hardcap-frontend.service",
"cayce-bitcoinhotels.service","postgresql@18-main.service","caddy.service"]
st = {}
try: st = json.load(open(sys.argv[1]))
except Exception: pass
now = time.time()
for u in UNITS:
active = subprocess.run(["systemctl","is-active",u], capture_output=True, text=True).stdout.strip() == "active"
if active:
st.pop(u, None)
else:
st.setdefault(u, now)
if now - st[u] > 15*60:
print(f"DOWN:{u}:{int((now-st[u])/60)}")
json.dump(st, open(sys.argv[1], "w"))
PY
)
# 3) backup fallido 2 días seguidos (r33: un fallo puntual no es urgente)
if systemctl is-failed cayce-backup.service >/dev/null 2>&1; then
FAILS=$(journalctl -u cayce-backup.service --since "48 hours ago" --no-pager 2>/dev/null | grep -c "Failed with result")
if [ "${FAILS:-0}" -ge 2 ]; then
dm_urgente "backup:2días" "URGENTE: cayce-backup fallido ${FAILS} veces en 48h — revisar /var/backups/cayce y el journal."
else
echo "[urgent] backup failed hoy ($FAILS en 48h <2) — va al resumen diario"
fi
fi
# 4) gasto de Gemini hoy > umbral (tokens reales)
read -r COST CALLS TIN TOUT < <(python3 - "$USAGE" <<'PY'
import json, sys, os, datetime
rin = float(os.environ.get("GEMINI_USD_PER_MTOK_IN", "0.15"))
rout = float(os.environ.get("GEMINI_USD_PER_MTOK_OUT", "0.60"))
today = datetime.datetime.utcnow().date().isoformat()
IN = OUT_ = CALLS = 0
if os.path.exists(sys.argv[1]):
for line in open(sys.argv[1]):
try:
d = json.loads(line)
if d.get("ts","")[:10] == today:
CALLS += 1; IN += d.get("in",0); OUT_ += d.get("out",0)
except Exception: pass
print(f"{IN/1e6*rin + OUT_/1e6*rout:.4f} {CALLS} {IN} {OUT_}")
PY
)
if python3 -c "exit(0 if float('${COST:-0}') > float('$GEMINI_MAX_USD') else 1)" 2>/dev/null; then
dm_urgente "gasto:gemini" "URGENTE: gasto Gemini de hoy \$${COST} (> \$${GEMINI_MAX_USD}) — ${CALLS} llamadas, ${TIN} in / ${TOUT} out tokens."
fi
# r33: lo que YA NO manda DM (solo resumen diario de 23:50):
# · ciclos mudos / pausados a propósito (promo hoy) — el resumen diario ya
# trae la sección "Ciclos: piezas producidas"
# · merges que tocan CODEOWNERS — el resumen diario (sección 3) lista los
# merges del día; la sección 3b lista los PRs abiertos
# · rollbacks recuperados — el autodeploy usa dm_once (un aviso por
# incidente) y el resumen diario lista los deploys/rollbacks del journal
logger -t cayce-urgent-watch "tick ok (gemini=\$${COST:-0})"