3.7 KiB
3.7 KiB
2026-07-04 01 — Batched Conflict Detection
Status: Shipped Scope: Backend refactor + docs
What we set out to do
The scheduled conflict-detection sweep from 2026-06-29 01 Scheduled Remote Conflict Detection was doing one videos.list API call per video. YouTube's videos.list accepts up to 50 IDs per call and costs the same 1 quota unit either way — the per-video pattern was wasting ~98% of the quota it consumed.
Goal: batch, without changing observable behavior for users or the API surface.
Design decisions locked in
| Question | Decision |
|---|---|
Refactor detectConflict in place or add a new method? |
Add detectConflictsForVideos(videoIds[]) and delete the single-video detectConflict() — nothing else called it |
| Grouping strategy for batches | Group by channelId first (batch API needs a per-channel OAuth client), then chunks of 50 within each channel |
| Where does the quota check live? | Moved from the processor's per-video loop into the service's per-batch loop — canSpend(1) before each getVideosBatch call |
Extend getVideosBatch to fetch recordingDetails? |
Yes — the batch endpoint previously fetched only snippet + status, which would have dropped recordingDate from the hash and produced false positives. Adding recordingDetails is free (quota is per method-call, not per part) |
What changed
backend/src/modules/youtube-sync/youtube-api.client.ts:106— addedrecordingDetailstogetVideosBatch()'spartarray. No behavior change for existing callers (channel-import already falls back to DB values for missing fields).backend/src/modules/youtube-sync/youtube-sync.service.ts— removeddetectConflict(videoId), addeddetectConflictsForVideos(videoIds[]), extracted the per-video hash+persist logic into a privateapplyConflictDetection(video, remoteItem)helper. The batch method returns{ scanned, conflicts, quotaExhausted }so the processor can stop the sweep cleanly.backend/src/queues/processors/conflict-detection.processor.ts— inner loop replaced with a single service call per team.QuotaServiceno longer injected (moved into the service). Error handling now per-team (not per-video), which changes granularity of theerrorscounter — an API failure aborts one team but doesn't leak quota.
Cost math — before vs after
Team batchSize |
Old quota per team per run | New quota per team per run |
|---|---|---|
| 50 | 50 | 1 |
| 250 | 250 | 5 |
| 500 | 500 | 10 |
Numbers assume all videos on one channel. Extra API call per channel boundary within a batch, so multi-channel teams pay slightly more (still ~50× cheaper than before).
Follow-ups worth flagging
- Default
conflictDetectionBatchSize = 50is now overly conservative. Existing team settings unchanged out of caution. Users could safely raise to 250–500. Worth a mention in release notes if you ship an announcement. applyConflictDetectionisprivateon the service — if a manualPOST /videos/:id/detect-conflictendpoint ever becomes a thing, it should calldetectConflictsForVideos([id])rather than making a single-video sibling method reappear.- Batch API doesn't return
contentDetails. Not needed by conflict detection but referenced byrefreshFromYouTube. That path still uses the single-videogetVideoMetadata— no change needed.
Verification
- Backend
tsc --noEmitclean - No migration required — this is a service-level refactor only
- Frontend untouched
Related
- 2026-06-29 01 Scheduled Remote Conflict Detection — original feature
- 05 - Queue System — updated processor description with new cost model
- 04 - Gotchas — updated
remoteConflictcost section