56 lines
3.7 KiB
Markdown
56 lines
3.7 KiB
Markdown
# 2026-07-04 01 — Batched Conflict Detection
|
||
|
||
**Status:** Shipped
|
||
**Scope:** Backend refactor + docs
|
||
|
||
---
|
||
|
||
## What we set out to do
|
||
|
||
The scheduled conflict-detection sweep from [[2026-06-29 01 Scheduled Remote Conflict Detection]] was doing one `videos.list` API call per video. YouTube's `videos.list` accepts up to 50 IDs per call and costs the same 1 quota unit either way — the per-video pattern was wasting ~98% of the quota it consumed.
|
||
|
||
Goal: batch, without changing observable behavior for users or the API surface.
|
||
|
||
## Design decisions locked in
|
||
|
||
| Question | Decision |
|
||
|---|---|
|
||
| Refactor `detectConflict` in place or add a new method? | Add `detectConflictsForVideos(videoIds[])` and delete the single-video `detectConflict()` — nothing else called it |
|
||
| Grouping strategy for batches | Group by `channelId` first (batch API needs a per-channel OAuth client), then chunks of 50 within each channel |
|
||
| Where does the quota check live? | Moved from the processor's per-video loop into the service's per-batch loop — `canSpend(1)` before each `getVideosBatch` call |
|
||
| Extend `getVideosBatch` to fetch `recordingDetails`? | Yes — the batch endpoint previously fetched only `snippet` + `status`, which would have dropped `recordingDate` from the hash and produced false positives. Adding `recordingDetails` is free (quota is per method-call, not per part) |
|
||
|
||
## What changed
|
||
|
||
- **`backend/src/modules/youtube-sync/youtube-api.client.ts:106`** — added `recordingDetails` to `getVideosBatch()`'s `part` array. No behavior change for existing callers (channel-import already falls back to DB values for missing fields).
|
||
- **`backend/src/modules/youtube-sync/youtube-sync.service.ts`** — removed `detectConflict(videoId)`, added `detectConflictsForVideos(videoIds[])`, extracted the per-video hash+persist logic into a private `applyConflictDetection(video, remoteItem)` helper. The batch method returns `{ scanned, conflicts, quotaExhausted }` so the processor can stop the sweep cleanly.
|
||
- **`backend/src/queues/processors/conflict-detection.processor.ts`** — inner loop replaced with a single service call per team. `QuotaService` no longer injected (moved into the service). Error handling now per-team (not per-video), which changes granularity of the `errors` counter — an API failure aborts one team but doesn't leak quota.
|
||
|
||
## Cost math — before vs after
|
||
|
||
| Team `batchSize` | Old quota per team per run | New quota per team per run |
|
||
|---:|---:|---:|
|
||
| 50 | 50 | 1 |
|
||
| 250 | 250 | 5 |
|
||
| 500 | 500 | 10 |
|
||
|
||
Numbers assume all videos on one channel. Extra API call per channel boundary within a batch, so multi-channel teams pay slightly more (still ~50× cheaper than before).
|
||
|
||
## Follow-ups worth flagging
|
||
|
||
- **Default `conflictDetectionBatchSize = 50` is now overly conservative.** Existing team settings unchanged out of caution. Users could safely raise to 250–500. Worth a mention in release notes if you ship an announcement.
|
||
- **`applyConflictDetection` is `private` on the service** — if a manual `POST /videos/:id/detect-conflict` endpoint ever becomes a thing, it should call `detectConflictsForVideos([id])` rather than making a single-video sibling method reappear.
|
||
- **Batch API doesn't return `contentDetails`.** Not needed by conflict detection but referenced by `refreshFromYouTube`. That path still uses the single-video `getVideoMetadata` — no change needed.
|
||
|
||
## Verification
|
||
|
||
- Backend `tsc --noEmit` clean
|
||
- No migration required — this is a service-level refactor only
|
||
- Frontend untouched
|
||
|
||
## Related
|
||
|
||
- [[2026-06-29 01 Scheduled Remote Conflict Detection]] — original feature
|
||
- [[05 - Queue System]] — updated processor description with new cost model
|
||
- [[04 - Gotchas]] — updated `remoteConflict` cost section
|