89 lines
6.1 KiB
Markdown
89 lines
6.1 KiB
Markdown
# 2026-06-29 01 — Scheduled Remote Conflict Detection
|
||
|
||
**Status:** Shipped
|
||
**Scope:** Backend, frontend Settings UI, ops env, docs
|
||
|
||
---
|
||
|
||
## What we set out to do
|
||
|
||
Resolve the "Dead Code — `YouTubeSyncService.detectConflict()`" backlog entry. Before this session, the method was fully implemented but never called; conflict handling as a user-facing flow was effectively broken:
|
||
|
||
- **Detection** only happened on manual "Refresh from YouTube" per video.
|
||
- **Resolution** either meant running a full channel refresh (clobbers *every* video's local fields) or re-pushing local (silently overwrites remote). Neither offered a per-video accept/reject choice and neither showed a diff.
|
||
|
||
Decision: wire `detectConflict()` up as a scheduled sweep AND build a proper per-video accept-remote path. Direction "push local" reuses the existing `POST /videos/:id/sync`.
|
||
|
||
## Design decisions (locked in during the session)
|
||
|
||
| Question | Decision |
|
||
|---|---|
|
||
| Granularity | Per-video accept/reject (v1). Field-level merge deferred. |
|
||
| Settings location | Per-team, with cadence global via env var |
|
||
| Skip already-conflicted videos in the sweep? | Yes — no point re-detecting an unresolved conflict |
|
||
| Add a `POST /videos/:id/force-local` endpoint? | No — reuse existing `POST /videos/:id/sync` |
|
||
| Scheduler mechanism | BullMQ repeatable job (no new dependency) |
|
||
| Selection strategy | Stalest-first per team, filtered by `lastSyncedAt < now - minAgeDays`, `remoteConflict = false`, `youtubeDeletedAt = null` |
|
||
|
||
## What changed
|
||
|
||
### Backend
|
||
|
||
- **Schema** — new migration `20260629000000_add_conflict_detection_settings`:
|
||
- `Team.conflictDetectionEnabled Boolean @default(false)`
|
||
- `Team.conflictDetectionBatchSize Int @default(50)`
|
||
- `Team.conflictDetectionMinAgeDays Int @default(7)`
|
||
- `Video.pendingRemoteSnapshot Json?`
|
||
- `Video.pendingRemoteDescription String?`
|
||
- **`YouTubeSyncService.detectConflict()`** (`backend/src/modules/youtube-sync/youtube-sync.service.ts`) — on hash mismatch now persists `pendingRemoteSnapshot` (mirrors `youtubeSnapshot` shape) and `pendingRemoteDescription`. On hash match with a stale flag, self-heals (clears flag + pending fields). No more dead code.
|
||
- **New queue** `CONFLICT_DETECTION` in `backend/src/queues/queues.constants.ts`.
|
||
- **New processor** `backend/src/queues/processors/conflict-detection.processor.ts` — iterates enabled teams, stalest-first selection, per-call quota guard, per-team batch cap. Logs summary. Stops the whole sweep on quota exhaustion.
|
||
- **New scheduler** `backend/src/queues/schedulers/conflict-detection.scheduler.ts` — `OnModuleInit` registers a BullMQ repeatable driven by `CONFLICT_DETECTION_CRON` (default `0 3 * * *`), gated by `CONFLICT_DETECTION_ENABLED`. Wipes stale repeatables on boot so config changes take effect.
|
||
- **Wired both** into `backend/src/worker.module.ts`.
|
||
- **New endpoint** `POST /videos/:id/accept-remote` (EDITOR role) in `backend/src/modules/videos/videos.controller.ts` + `.service.ts`. Zero YouTube API calls — promotes stored `pendingRemoteSnapshot` into live columns and `youtubeSnapshot`, sets `renderedDescription = youtubeDescription = pendingRemoteDescription`, recomputes `lastSyncedHash`, sets `lastSyncedAt = now`, clears pending + flag. Emits audit log entry `action: 'accept-remote'`.
|
||
- **Team settings** — three new fields added to `GET`/`PATCH /teams/:teamId/settings` with bounds validation (batch 1–500, min-age ≥ 0).
|
||
|
||
### Frontend
|
||
|
||
- `TeamSettings` interface (`frontend/src/lib/api.ts`) extended with the three new fields.
|
||
- New "Remote Conflict Detection" section on `/settings`: Enable toggle, "Videos per run" (number, 1–500), "Only check videos older than (days)" (number, ≥0). Admin-only edit. Own Save button (kept separate from the schedule section to avoid mixed-scope saves).
|
||
|
||
### Ops / env
|
||
|
||
- `CONFLICT_DETECTION_ENABLED` and `CONFLICT_DETECTION_CRON` added to:
|
||
- `backend/.env` (locally set to `true` / default cron)
|
||
- `backend/.env.example` (default `false`)
|
||
- `infrastructure/.env.example`
|
||
- `infrastructure/docker-compose.yml` worker service (uses `:-` defaults)
|
||
|
||
### Documentation
|
||
|
||
- **Backlog entries removed** from `06 - Backlog/01 - Technical Debt and Future Work.md`:
|
||
- "Dead Code — `YouTubeSyncService.detectConflict()`"
|
||
- "Improvements Worth Considering — Automatic Remote Conflict Detection"
|
||
- **Docs brought back in sync** (11 findings across 8 files identified by an audit agent, all applied):
|
||
- Rewrote the two aggressively-wrong sections (Gotchas `remoteConflict Is Not Automatically Detected`, Schema `remoteConflict` paragraph) which literally claimed the opposite of the new behavior.
|
||
- Filled content gaps in `04 - Database Schema`, `05 - Queue System`, `13 - Team Settings`, `02 - Videos API`, `09 - Teams API`, `02 - Environment Variables`, `05 - Deployment and Operations`.
|
||
- Vault gained a new `07 - Daily Notes/` section (this note is the first entry).
|
||
|
||
## Verification
|
||
|
||
- Backend `tsc --noEmit` clean after `prisma generate`.
|
||
- Frontend `tsc --noEmit` clean for the touched files (`settings/page.tsx`, `api.ts`); pre-existing errors elsewhere untouched.
|
||
- Migration file present but not yet applied on any prod DB — user ran `prisma migrate` locally.
|
||
|
||
## Follow-ups worth flagging
|
||
|
||
- **UI for the conflict itself** — the video editor should show a per-field diff between the current local state and `pendingRemoteSnapshot` when `remoteConflict === true`, with "Accept remote" and "Keep local" buttons. Backend is ready; frontend diff view doesn't exist yet.
|
||
- **Field-level merge** — declined for v1; if requested later, the storage layer already supports it (fields are individually addressable in `pendingRemoteSnapshot`).
|
||
- **Multi-team frequency** — if teams later want different cadences, `frequency` needs to move from env-var to a per-team column and the scheduler must manage a repeatable-per-team.
|
||
|
||
## Related
|
||
|
||
- [[04 - Database Schema]] — new Team + Video columns
|
||
- [[05 - Queue System]] — processor details, quota model
|
||
- [[13 - Team Settings]] — per-team knobs, validation
|
||
- [[02 - Videos API]] — `accept-remote` endpoint
|
||
- [[04 - Gotchas]] — resolution options and self-heal
|
||
- [[02 - Environment Variables]] — `CONFLICT_DETECTION_*`
|