Files
youtube-studio-flow/documentation/YouTube Studio Flow/07 - Daily Notes/2026-07-04 01 Batched Conflict Detection.md
T

3.7 KiB
Raw Blame History

2026-07-04 01 — Batched Conflict Detection

Status: Shipped Scope: Backend refactor + docs


What we set out to do

The scheduled conflict-detection sweep from 2026-06-29 01 Scheduled Remote Conflict Detection was doing one videos.list API call per video. YouTube's videos.list accepts up to 50 IDs per call and costs the same 1 quota unit either way — the per-video pattern was wasting ~98% of the quota it consumed.

Goal: batch, without changing observable behavior for users or the API surface.

Design decisions locked in

Question Decision
Refactor detectConflict in place or add a new method? Add detectConflictsForVideos(videoIds[]) and delete the single-video detectConflict() — nothing else called it
Grouping strategy for batches Group by channelId first (batch API needs a per-channel OAuth client), then chunks of 50 within each channel
Where does the quota check live? Moved from the processor's per-video loop into the service's per-batch loop — canSpend(1) before each getVideosBatch call
Extend getVideosBatch to fetch recordingDetails? Yes — the batch endpoint previously fetched only snippet + status, which would have dropped recordingDate from the hash and produced false positives. Adding recordingDetails is free (quota is per method-call, not per part)

What changed

  • backend/src/modules/youtube-sync/youtube-api.client.ts:106 — added recordingDetails to getVideosBatch()'s part array. No behavior change for existing callers (channel-import already falls back to DB values for missing fields).
  • backend/src/modules/youtube-sync/youtube-sync.service.ts — removed detectConflict(videoId), added detectConflictsForVideos(videoIds[]), extracted the per-video hash+persist logic into a private applyConflictDetection(video, remoteItem) helper. The batch method returns { scanned, conflicts, quotaExhausted } so the processor can stop the sweep cleanly.
  • backend/src/queues/processors/conflict-detection.processor.ts — inner loop replaced with a single service call per team. QuotaService no longer injected (moved into the service). Error handling now per-team (not per-video), which changes granularity of the errors counter — an API failure aborts one team but doesn't leak quota.

Cost math — before vs after

Team batchSize Old quota per team per run New quota per team per run
50 50 1
250 250 5
500 500 10

Numbers assume all videos on one channel. Extra API call per channel boundary within a batch, so multi-channel teams pay slightly more (still ~50× cheaper than before).

Follow-ups worth flagging

  • Default conflictDetectionBatchSize = 50 is now overly conservative. Existing team settings unchanged out of caution. Users could safely raise to 250500. Worth a mention in release notes if you ship an announcement.
  • applyConflictDetection is private on the service — if a manual POST /videos/:id/detect-conflict endpoint ever becomes a thing, it should call detectConflictsForVideos([id]) rather than making a single-video sibling method reappear.
  • Batch API doesn't return contentDetails. Not needed by conflict detection but referenced by refreshFromYouTube. That path still uses the single-video getVideoMetadata — no change needed.

Verification

  • Backend tsc --noEmit clean
  • No migration required — this is a service-level refactor only
  • Frontend untouched
  • 2026-06-29 01 Scheduled Remote Conflict Detection — original feature
  • 05 - Queue System — updated processor description with new cost model
  • 04 - Gotchas — updated remoteConflict cost section