Initial commit: YouTube Studio Flow (backend, frontend, infrastructure, docs)
This commit is contained in:
@@ -0,0 +1,65 @@
|
||||
# System Overview
|
||||
|
||||
YouTube Studio Flow is a multi-tenant SaaS tool for YouTube channel management. Teams connect their YouTube channels, import their video library, manage video metadata using a description template and block system, and push changes back to YouTube via the API.
|
||||
|
||||
## Tech Stack
|
||||
|
||||
| Layer | Technology |
|
||||
|---|---|
|
||||
| Frontend | Next.js 15 App Router, TypeScript, CSS Modules |
|
||||
| Data fetching | TanStack Query v5 |
|
||||
| Auth state | Zustand |
|
||||
| Backend | NestJS 10, TypeScript |
|
||||
| ORM | Prisma (PostgreSQL) |
|
||||
| Queue | BullMQ + Redis |
|
||||
| Auth | Google OAuth → JWT (access + refresh tokens) |
|
||||
| YouTube integration | `googleapis` package, quota-tracked |
|
||||
|
||||
## Process Architecture
|
||||
|
||||
The backend runs as **two separate Node.js processes** from the same build:
|
||||
|
||||
| Process | Entry point | Role |
|
||||
|---|---|---|
|
||||
| API server | `src/main.ts` | HTTP API on port 3001 |
|
||||
| Queue worker | `src/worker.ts` | BullMQ job processor (no HTTP) |
|
||||
|
||||
Both must be running for full functionality. The API enqueues jobs; the worker executes them.
|
||||
|
||||
## System Diagram
|
||||
|
||||
```
|
||||
Browser (Next.js)
|
||||
│
|
||||
│ HTTP/REST (NEXT_PUBLIC_API_URL)
|
||||
▼
|
||||
NestJS API (:3001)
|
||||
│
|
||||
├──── PostgreSQL (Prisma)
|
||||
├──── Redis (BullMQ queues)
|
||||
│
|
||||
└──── BullMQ worker process
|
||||
│
|
||||
├──── PostgreSQL (reads/writes)
|
||||
└──── YouTube API (googleapis)
|
||||
```
|
||||
|
||||
## Request Scoping
|
||||
|
||||
Every authenticated request is scoped to a team. The JWT payload contains `userId` and `teamId`. All database queries must filter by `channel: { teamId }` or direct `teamId` field. See [[02 - Backend]] for the auth pattern.
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
- **Denormalized `lintStatus`**: `Video.lintStatus` is a cached column. Any code path that deletes `LintResult` rows must recompute it manually — it is not computed on-the-fly.
|
||||
- **YouTube token encryption**: OAuth tokens are AES-256 encrypted in the DB using `TOKEN_ENCRYPTION_KEY`. If this key changes all channels need re-authentication.
|
||||
- **Queue deduplication**: BullMQ job IDs are used for deduplication. The pattern is `lint-{videoId}` for lint-once and `lint-{videoId}-{timestamp}` for forced reruns.
|
||||
- **No shared code**: Backend and frontend are fully independent — no shared packages, no shared types. The frontend maintains its own API interface definitions in `api.ts`.
|
||||
- **`hasPendingChanges` null-hash fallback**: `Video.lastSyncedHash` is set at import time to establish the YouTube baseline. For videos that existed before hashing was introduced (i.e. `lastSyncedHash IS NULL`), `findAll` falls back to computing a synthetic baseline from `youtubeDescription` and `recordingDate: null` and compares that against the current state hash. This means pre-existing videos correctly show "push pending" only when local changes diverge from the YouTube state, rather than always appearing as pending. Re-importing a channel resets the hash baseline and will clear any locally-pending changes that have not yet been pushed.
|
||||
|
||||
## Related
|
||||
|
||||
- [[02 - Backend]] — Module structure, patterns, shared services
|
||||
- [[03 - Frontend]] — Routing, state management, API layer
|
||||
- [[04 - Database Schema]] — All Prisma models
|
||||
- [[05 - Queue System]] — BullMQ queues and processors
|
||||
- [[06 - Authentication]] — OAuth flow, JWT, guards
|
||||
@@ -0,0 +1,99 @@
|
||||
# Backend Architecture
|
||||
|
||||
## Entry Points
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `src/main.ts` | Starts the NestJS HTTP API on port 3001 |
|
||||
| `src/worker.ts` | Starts the BullMQ worker (no HTTP server) |
|
||||
|
||||
## Module Map
|
||||
|
||||
```
|
||||
src/modules/
|
||||
auth/ Google OAuth, JWT strategy, guards
|
||||
videos/ Video CRUD, bulk ops, sync enqueue, findAll with filters
|
||||
video-configs/ Per-video description config (block order, overrides, variables)
|
||||
blocks/ DescriptionBlock CRUD + versioning
|
||||
templates/ Template CRUD, preview, apply-to-video
|
||||
collaborators/ Collaborator CRUD
|
||||
teams/ Team management, members, channels, settings, publishing schedule
|
||||
team-variables/ Global team-level key/value variables
|
||||
saved-views/ Saved filter presets for the video list
|
||||
campaigns/ Campaign date windows (CAMPAIGN blocks auto-include when active)
|
||||
playlists/ YouTube playlist sync + video↔playlist management
|
||||
linting/ Rule-based lint checks on video metadata
|
||||
bulk-jobs/ Bulk metadata operations (preview, execute, rollback)
|
||||
youtube-sync/ Channel import (YouTubeApiClient, ChannelImportService)
|
||||
calendar/ Scheduled video calendar view
|
||||
imports/ CSV + JSON workspace import
|
||||
exports/ CSV + JSON workspace export
|
||||
quota/ YouTube API quota tracking and history
|
||||
|
||||
src/shared/
|
||||
prisma/ PrismaService singleton
|
||||
render-engine/ VideoRenderService + RenderEngineService + hashMetadata
|
||||
audit/ AuditService — logs all user mutations
|
||||
quota/ QuotaService — tracks and enforces YouTube API quota
|
||||
system-variables/ Built-in {video.*} and {collab.*} token registry
|
||||
|
||||
src/queues/processors/
|
||||
youtube-sync.processor.ts Renders description + pushes all fields to YouTube
|
||||
render.processor.ts Background render without pushing
|
||||
lint.processor.ts Runs all lint rules against a video
|
||||
bulk-metadata.processor.ts Processes bulk job items
|
||||
import.processor.ts CSV import processing
|
||||
```
|
||||
|
||||
## Auth Pattern
|
||||
|
||||
Every controller method receives the authenticated user via `@Req() req`. Extract:
|
||||
- `req.user.id` — actorId (for audit logs, ownership checks)
|
||||
- `req.user.teamId` — always scope DB queries to this team
|
||||
|
||||
All routes are protected by `JwtAuthGuard`. Role-restricted routes additionally use `RolesGuard` with `@Roles(TeamRole.EDITOR)` or `@Roles(TeamRole.ADMIN)`.
|
||||
|
||||
## Team Scoping Rule
|
||||
|
||||
**Always** filter database queries by team. The ownership chain is:
|
||||
|
||||
```
|
||||
Team → Channel → Video
|
||||
```
|
||||
|
||||
To scope a video query: `where: { channel: { teamId: req.user.teamId } }`
|
||||
|
||||
## Audit Logging
|
||||
|
||||
`AuditService.log(actorId, entityType, entityId, action, before, after)` must be called for every user-facing mutation (create/update/delete). Background/system operations do not log.
|
||||
|
||||
Tracked entity types: `Video`, `VideoConfig`, `DescriptionBlock`, `Template`, `Collaborator`, `TeamVariable`, `SavedView`, `TeamMember`.
|
||||
|
||||
Modules that need audit logging must import `AuditModule` in their `@Module` imports array.
|
||||
|
||||
## Shared Services
|
||||
|
||||
### VideoRenderService
|
||||
Single source of truth for description rendering. Fetches all required data (blocks, team variables, campaign blocks, collaborators, team date format, playlists) and delegates to `RenderEngineService`. Used by all three render paths:
|
||||
- `youtube-sync.processor` — render + push
|
||||
- `render.processor` — render only
|
||||
- `videos.service.ts renderDescription()` — on-demand
|
||||
|
||||
**Never duplicate the data-fetching logic.** Always go through `VideoRenderService`.
|
||||
|
||||
### RenderEngineService
|
||||
Pure computation. Takes `RenderInput`, returns `{ rendered: string, hash: string }`. Used directly only by `video-configs.service.ts renderPreview()` and `templates.service.ts renderPreview()` (which already have all data).
|
||||
|
||||
### QuotaService
|
||||
`canSpend(units)` and `spend(units, operation, meta)` must be called before any YouTube API write. Quota resets at midnight Pacific Time. `videos.update` costs 50 units; daily limit is 10,000.
|
||||
|
||||
### AuditService
|
||||
See Audit Logging above.
|
||||
|
||||
## Related
|
||||
|
||||
- [[01 - System Overview]]
|
||||
- [[04 - Database Schema]]
|
||||
- [[05 - Queue System]]
|
||||
- [[07 - Render Engine]]
|
||||
- [[03 - Metadata Linting]] (features)
|
||||
@@ -0,0 +1,99 @@
|
||||
# Frontend Architecture
|
||||
|
||||
## Framework
|
||||
|
||||
Next.js 15 App Router with TypeScript and CSS Modules. All pages under `src/app/(dashboard)/` are protected by `AuthGuard` and wrapped in the dashboard layout (sidebar + header).
|
||||
|
||||
## Route Map
|
||||
|
||||
| Route | Page |
|
||||
|---|---|
|
||||
| `/` | Redirects to `/overview` |
|
||||
| `/overview` | Dashboard overview |
|
||||
| `/videos` | Video list — paginated, sortable, filterable, with tabs |
|
||||
| `/videos/[id]` | Video editor — metadata + description config |
|
||||
| `/blocks` | Description block management |
|
||||
| `/templates` | Template management |
|
||||
| `/variables` | Team global variables |
|
||||
| `/collaborators` | Collaborator management |
|
||||
| `/settings` | Team members + connected channels |
|
||||
| `/saved-views` | Saved filter preset management |
|
||||
| `/linting` | Lint results browser — filter by severity/rule |
|
||||
| `/bulk-jobs` | Bulk operation history + rollback |
|
||||
| `/calendar` | Scheduled video calendar (month/week/agenda) |
|
||||
| `/audit` | Change history / audit log |
|
||||
| `/quota-history` | YouTube API quota usage history |
|
||||
| `/io` | CSV / JSON import and export |
|
||||
| `/login` | Google OAuth login |
|
||||
| `/auth/callback` | OAuth callback handler |
|
||||
|
||||
## Key Files
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `src/lib/api.ts` | All API call functions + TypeScript interfaces for every API response |
|
||||
| `src/lib/api-client.ts` | Axios instance — sets `baseURL`, attaches JWT, handles 401 refresh |
|
||||
| `src/store/useAuthStore.ts` | Zustand — `user`, `teamId`, `token`, `clearAuth` |
|
||||
| `src/store/useUIStore.ts` | Zustand (persisted) — `sidebarCollapsed`, `toggleSidebar` |
|
||||
| `src/middleware.ts` | Route protection — redirects unauthenticated users to `/login` |
|
||||
| `src/styles/globals.css` | Design tokens (CSS variables), global utility classes |
|
||||
| `src/hooks/useTheme.ts` | Dark/light theme toggle with localStorage persistence |
|
||||
|
||||
## State Management
|
||||
|
||||
| Concern | Tool |
|
||||
|---|---|
|
||||
| Server data (videos, blocks, etc.) | TanStack Query v5 `useQuery` / `useMutation` |
|
||||
| Auth state | Zustand `useAuthStore` |
|
||||
| UI state (sidebar) | Zustand `useUIStore` (localStorage-persisted) |
|
||||
| Page-level UI state | `useState` / `useReducer` |
|
||||
|
||||
## TanStack Query Patterns
|
||||
|
||||
- `useQueryClient()` must be called at component level, never inside callbacks
|
||||
- After mutations, invalidate related queries in `onSuccess`:
|
||||
- After saving a video: invalidate `['video', id]` and `['videos']`
|
||||
- After changing blocks: invalidate `['blocks']` and affected video queries
|
||||
- Use `placeholderData: (prev) => prev` to keep stale data visible during page transitions
|
||||
- Query keys are arrays: `['videos']`, `['video', id]`, `['preferences']`
|
||||
|
||||
## API Layer
|
||||
|
||||
All API calls are in `src/lib/api.ts`. The file exports:
|
||||
- TypeScript interfaces for all API response shapes
|
||||
- Named functions for each endpoint (`fetchVideos`, `updateVideo`, `lintVideo`, etc.)
|
||||
|
||||
The Axios client (`api-client.ts`) handles:
|
||||
- Base URL from `NEXT_PUBLIC_API_URL`
|
||||
- JWT `Authorization: Bearer` header injection
|
||||
- 401 → automatic token refresh → retry
|
||||
|
||||
## Component Structure
|
||||
|
||||
```
|
||||
src/components/
|
||||
video-table/
|
||||
VideoTable.tsx TanStack Table — video list with sorting, pagination
|
||||
VideoTable.module.css
|
||||
video-config/
|
||||
VideoConfigEditor.tsx forwardRef component — exposes save() and isDirty()
|
||||
VideoConfigEditor.module.css
|
||||
shared/
|
||||
Sidebar.tsx Collapsible navigation sidebar
|
||||
Header.tsx Top bar — search, sync status, bulk change, theme, user
|
||||
Modal.tsx Generic modal wrapper
|
||||
FormField.module.css Shared form field utility classes
|
||||
Providers.tsx TanStack Query provider wrapper
|
||||
AuthGuard.tsx Redirects unauthenticated users
|
||||
SortableSection.tsx DnD-kit sortable wrapper for video editor sections
|
||||
```
|
||||
|
||||
## Video Editor Layout
|
||||
|
||||
The video editor (`/videos/[id]`) uses a combined horizontal header card (thumbnail + meta + actions) above a two-column sortable editing area. Sections (Basic Info, Language, Audience, Playlists, Description) can be dragged between the two columns. Column layout is persisted to `User.preferences` via `PATCH /users/me/preferences`.
|
||||
|
||||
## Related
|
||||
|
||||
- [[01 - System Overview]]
|
||||
- [[01 - Visual Design]] (design guidelines)
|
||||
- [[02 - CSS Conventions]] (design guidelines)
|
||||
@@ -0,0 +1,453 @@
|
||||
# Database Schema
|
||||
|
||||
PostgreSQL via Prisma. Schema at `backend/prisma/schema.prisma`.
|
||||
|
||||
## Model Overview
|
||||
|
||||
```
|
||||
User
|
||||
└── TeamMember[] (many-to-many with Team)
|
||||
|
||||
Team
|
||||
├── TeamMember[]
|
||||
├── Channel[]
|
||||
├── DescriptionBlock[]
|
||||
├── Template[]
|
||||
├── Collaborator[]
|
||||
├── BulkJob[]
|
||||
├── SavedView[]
|
||||
├── ImportJob[]
|
||||
├── ExportJob[]
|
||||
├── Campaign[]
|
||||
└── TeamVariable[]
|
||||
|
||||
Channel
|
||||
├── Video[]
|
||||
├── Playlist[]
|
||||
└── QuotaLog[]
|
||||
|
||||
Video
|
||||
├── VideoConfig (1:1)
|
||||
├── VideoPlaylist[] (many-to-many with Playlist)
|
||||
├── LintResult[]
|
||||
├── BulkJobItem[]
|
||||
└── QuotaLog[]
|
||||
|
||||
DescriptionBlock
|
||||
└── BlockVersion[]
|
||||
|
||||
Template
|
||||
└── TemplateVersion[]
|
||||
|
||||
BulkJob
|
||||
└── BulkJobItem[]
|
||||
|
||||
Playlist
|
||||
└── VideoPlaylist[] (many-to-many with Video)
|
||||
```
|
||||
|
||||
## Key Models
|
||||
|
||||
### User
|
||||
```
|
||||
id, email, name, googleId, isAppAdmin, preferences (Json), createdAt, updatedAt
|
||||
```
|
||||
`preferences` is a freeform JSON field used to persist per-user UI state (column visibility, column order, video editor section layout).
|
||||
|
||||
### Team
|
||||
```
|
||||
id, name, slug,
|
||||
dateFormat (String?),
|
||||
timezone (String, default "UTC"),
|
||||
publishingSchedule (Json?),
|
||||
showCanvaLink (Boolean),
|
||||
disabledLintRules (String[]),
|
||||
showDeletedVideos (Boolean),
|
||||
conflictDetectionEnabled (Boolean, default false),
|
||||
conflictDetectionBatchSize (Int, default 50),
|
||||
conflictDetectionMinAgeDays (Int, default 7)
|
||||
```
|
||||
The three `conflictDetection*` fields configure the scheduled remote-conflict sweep. `Enabled` is per-team opt-in; `batchSize` caps how many videos are scanned per run (1–500); `minAgeDays` skips videos synced more recently than N days. Global on/off and cron pattern live in env vars (`CONFLICT_DETECTION_ENABLED`, `CONFLICT_DETECTION_CRON`).
|
||||
|
||||
### Channel
|
||||
```
|
||||
id, teamId, youtubeChannelId (unique), name, uploadsPlaylistId,
|
||||
supplementalVideoIds (String[]),
|
||||
youtubeAccessToken (encrypted), youtubeRefreshToken (encrypted), youtubeTokenExpiry,
|
||||
connectedBy
|
||||
```
|
||||
OAuth tokens are AES-256 encrypted using `TOKEN_ENCRYPTION_KEY`. Changing this key breaks all existing channel connections.
|
||||
|
||||
`supplementalVideoIds` is an array of YouTube video IDs that are always imported during `channel-import` and `channel-full-refresh` runs, regardless of whether they appear in the uploads playlist. This exists for unlisted or otherwise playlist-excluded videos that the team still wants to manage. On each import the list is filtered against the playlist results so only IDs not already discovered via the playlist are fetched as extras. New IDs are added via `POST /youtube-sync/channels/:channelId/supplemental-ids`.
|
||||
|
||||
### Video
|
||||
```
|
||||
id, youtubeVideoId (unique), channelId, title,
|
||||
youtubeDescription, renderedDescription, thumbnailUrl,
|
||||
tags (String[]), categoryId, privacyStatus (PUBLIC/PRIVATE/UNLISTED),
|
||||
publishedAt, scheduledAt, templateId,
|
||||
selfDeclaredMadeForKids, embeddable, license, defaultLanguage, defaultAudioLanguage,
|
||||
recordingDate, gameTitle,
|
||||
collaboratorIds (Json, default "[]"),
|
||||
youtubeSnapshot (Json?),
|
||||
lintStatus (OK/WARNING/ERROR),
|
||||
lastSyncedAt, lastSyncedHash,
|
||||
remoteConflict (Boolean),
|
||||
pendingRemoteSnapshot (Json?), pendingRemoteDescription (String?),
|
||||
youtubeDeletedAt
|
||||
```
|
||||
|
||||
**Important fields:**
|
||||
- `lintStatus` — denormalized cache. Must be recomputed by any code that deletes `LintResult` rows
|
||||
- `lastSyncedHash` — hash of YouTube-side state. Compared against current state to compute `hasPendingChanges`
|
||||
- `youtubeSnapshot` — the *last intentionally-synced* YouTube metadata (written on push and on refresh). Used for field-level diff
|
||||
- `collaboratorIds` — JSON array of Collaborator IDs used in description rendering
|
||||
- `remoteConflict` — set to `true` when YouTube-side metadata diverges from `lastSyncedHash`, meaning something changed on YouTube outside of this tool since the last push. Detected via two paths: (1) manual `POST /videos/:id/refresh`, or (2) the scheduled `CONFLICT_DETECTION` queue when the team opts in (`Team.conflictDetectionEnabled`). Cleared on a successful YouTube push, on `POST /videos/:id/accept-remote`, or on a self-heal pass where the remote hash matches `lastSyncedHash` again.
|
||||
- `pendingRemoteSnapshot` / `pendingRemoteDescription` — written by `detectConflict()` when a mismatch is found. Snapshot mirrors the `youtubeSnapshot` shape and holds the freshly fetched remote state; description holds the raw remote description. Consumed by `POST /videos/:id/accept-remote` (no extra YouTube call). Cleared when the conflict is resolved.
|
||||
|
||||
### VideoConfig
|
||||
```
|
||||
id, videoId (unique), templateId?,
|
||||
blockOrder (Json, String[]),
|
||||
blockOverrides (Json),
|
||||
variableValues (Json),
|
||||
version, renderHash, renderedAt
|
||||
```
|
||||
`blockOrder` may contain both real block IDs and `freetext:{uuid}` IDs. Freetext content is stored in `blockOverrides[id].content`.
|
||||
|
||||
### DescriptionBlock
|
||||
```
|
||||
id, teamId, name, type (BlockType), content, language,
|
||||
campaignId?, version, active, compact,
|
||||
tags (String[]), variableDefinitions (Json), condition (Json?)
|
||||
```
|
||||
|
||||
**BlockType enum values:**
|
||||
- `STATIC` — raw output, no token substitution
|
||||
- `VARIABLE` — resolves `{custom_var}`, `{video.*}`, `{collab.*}` tokens
|
||||
- `CAMPAIGN` — auto-included at end of render when linked campaign is active
|
||||
- `CONDITIONAL`, `COLLABORATOR` — contextual blocks
|
||||
- `GLOBAL`, `REPEATABLE` — deprecated (kept in enum, hidden in UI)
|
||||
|
||||
### Template
|
||||
```
|
||||
id, teamId, name, description?,
|
||||
defaultBlocks (Json, String[]),
|
||||
defaultOverrides (Json),
|
||||
rules (Json),
|
||||
variables (Json),
|
||||
videoFields (Json?),
|
||||
version, active
|
||||
```
|
||||
`rules.requiredLinks` is used by the `DESC_REQUIRED_LINK_MISSING` lint rule.
|
||||
`videoFields` contains default values for video metadata fields applied when the template is assigned.
|
||||
|
||||
### TeamMember
|
||||
```
|
||||
userId (PK), teamId (PK), role (TeamRole, default EDITOR), createdAt
|
||||
```
|
||||
Composite primary key `(userId, teamId)`. Represents a user's membership in a team.
|
||||
|
||||
**TeamRole enum:** `OWNER`, `ADMIN`, `EDITOR`, `REVIEWER`, `READONLY`
|
||||
|
||||
`OWNER` is set at team creation and cannot be assigned via the invite API. See [[06 - Authentication]] for role hierarchy and permission levels.
|
||||
|
||||
### TeamVariable
|
||||
```
|
||||
id, teamId, name, value, createdAt, updatedAt
|
||||
```
|
||||
Unique constraint on `(teamId, name)` — variable names must be unique per team. The `name` is used as the token key: `{name}` in `VARIABLE`-type description blocks. Team variables form the baseline layer; video-level `variableValues` override them per-video.
|
||||
|
||||
Any mutation (create / update / delete) triggers a background re-render of all team videos.
|
||||
|
||||
### Collaborator
|
||||
```
|
||||
id, teamId, name,
|
||||
youtubeLink?, twitchLink?, instagramLink?, tiktokLink?,
|
||||
twitterLink?, blueskyLink?, discordHandle?,
|
||||
aliases (String[]),
|
||||
active (Boolean, default true),
|
||||
notes?,
|
||||
createdAt, updatedAt
|
||||
```
|
||||
Platform link fields are full URLs except `discordHandle` which is a handle string. All are optional. `aliases` is an array of alternative names used in search and token display.
|
||||
|
||||
Collaborators are assigned to a video via `Video.collaboratorIds` (JSON string array). This is the single source used by the render engine to resolve `{collab.*}` tokens and by the calendar and filter queries to check collaborator assignment. There is no separate join table.
|
||||
|
||||
### Campaign
|
||||
```
|
||||
id, teamId, name, startAt, endAt?, status (String, default "active"), notes?
|
||||
```
|
||||
`endAt` is optional — `null` means the campaign is open-ended. `status` is a free-form string (not an enum), default `"active"`.
|
||||
|
||||
At render time, any `CAMPAIGN`-type `DescriptionBlock` linked to this campaign (via `campaignId`) is automatically appended to the description if `startAt ≤ now ≤ endAt`. Campaign CRUD is not exposed via the REST API — managed directly in the database.
|
||||
|
||||
### BlockVersion
|
||||
```
|
||||
id, blockId, version (Int), contentSnapshot (Json), createdAt, createdBy?
|
||||
```
|
||||
Append-only history table. A new row is written each time a `DescriptionBlock` is saved. `contentSnapshot` stores the full block content at that version. `createdBy` is the user ID of the actor who saved it (nullable for system/migration writes).
|
||||
|
||||
`version` mirrors `DescriptionBlock.version` at the time of the snapshot. Versions are used to show change history in the block editor.
|
||||
|
||||
### TemplateVersion
|
||||
```
|
||||
id, templateId, version (Int), snapshot (Json), createdAt
|
||||
```
|
||||
Same append-only pattern as `BlockVersion`. `snapshot` stores the full template state (including `defaultBlocks`, `defaultOverrides`, `rules`, `variables`, `videoFields`) at the time of save.
|
||||
|
||||
### LintResult
|
||||
```
|
||||
id, videoId, ruleCode, severity (INFO/WARNING/ERROR),
|
||||
targetField?, message, fixSuggestion?, resolvedAt?
|
||||
```
|
||||
Unresolved results have `resolvedAt: null`. Resolving marks `resolvedAt = now()` and triggers `lintStatus` recomputation on the video.
|
||||
|
||||
### SavedView
|
||||
```
|
||||
id, teamId, name, description?,
|
||||
isGlobal, ownerId?,
|
||||
queryJson, columnsJson, sortJson?,
|
||||
pinnedAsTab (Boolean), tabOrder (Int?)
|
||||
```
|
||||
|
||||
### BulkJob
|
||||
```
|
||||
id, teamId, type, initiatedBy, filterSnapshot, targetIds,
|
||||
status (PENDING/DRY_RUN/CONFIRMED/RUNNING/DONE/FAILED/ROLLED_BACK),
|
||||
dryRunResult?, rollbackData?, totalCount, successCount, errorCount
|
||||
```
|
||||
|
||||
### BulkJobItem
|
||||
```
|
||||
id, bulkJobId, videoId, beforeSnapshot (Json?), afterSnapshot (Json?),
|
||||
status (String, default "pending"), errorMessage?
|
||||
createdAt
|
||||
```
|
||||
One row per video in a bulk job. `status` values: `"pending"`, `"done"`, `"error"`. `beforeSnapshot` and `afterSnapshot` store the YouTube metadata before and after the operation — used for rollback and the change diff display in the UI.
|
||||
|
||||
### Playlist
|
||||
```
|
||||
id, channelId, youtubePlaylistId (unique), title, description?,
|
||||
itemCount (Int, default 0), privacyStatus (String, default "public"),
|
||||
createdAt, updatedAt
|
||||
```
|
||||
Synced from YouTube. `itemCount` is cached and used to detect playlist changes without fetching all items on every import.
|
||||
|
||||
### VideoPlaylist
|
||||
```
|
||||
videoId (PK), playlistId (PK), position (Int?)
|
||||
```
|
||||
Composite primary key join table. `position` is the video's position within the playlist as returned by YouTube, if available.
|
||||
|
||||
### ImportJob
|
||||
```
|
||||
id, teamId, type (String), sourceName, mappingJson (Json?),
|
||||
validationReport (Json?), commitStatus (String, default "pending"),
|
||||
createdBy, createdAt, committedAt?
|
||||
```
|
||||
Tracks a CSV import session. `type` identifies the import variant (e.g. `"csv-videos"`). `mappingJson` stores the column-to-field mapping chosen during the import wizard. `validationReport` stores per-row validation results. `commitStatus` transitions: `"pending"` → `"committed"`.
|
||||
|
||||
### ExportJob
|
||||
```
|
||||
id, teamId, type (String), scopeJson (Json),
|
||||
fileReference?, createdBy, createdAt
|
||||
```
|
||||
Records each export operation. `scopeJson` captures the filter/selection that was exported. `fileReference` stores the path or identifier of the generated export file.
|
||||
|
||||
### QuotaLog
|
||||
```
|
||||
id, datePt, units, operation, entityId?, channelId?, videoId?, bulkJobId?, actionId?, actionType?
|
||||
```
|
||||
`actionId` groups related quota entries (e.g. all API calls for a single video sync). Used for quota history grouping in the frontend.
|
||||
|
||||
### AuditLog
|
||||
```
|
||||
id, actorId, entityType, entityId, action, beforeJson?, afterJson?, requestId?
|
||||
```
|
||||
|
||||
## JSON Field Schemas
|
||||
|
||||
### Team.publishingSchedule
|
||||
|
||||
```typescript
|
||||
type PublishingSlot = {
|
||||
days: number[]; // weekdays: 0 = Sunday … 6 = Saturday. Empty array = every day.
|
||||
time: string; // "HH:MM" in 24h format, interpreted in the team's IANA timezone
|
||||
};
|
||||
|
||||
// Stored as: PublishingSlot[] | null
|
||||
```
|
||||
|
||||
Used by `GET /teams/:teamId/next-publish-slot`. The algorithm walks up to 90 days forward, checks each slot against already-scheduled videos (±30 min collision window), and returns the first free slot.
|
||||
|
||||
---
|
||||
|
||||
### DescriptionBlock.condition
|
||||
|
||||
Only present on `CONDITIONAL`-type blocks. If absent or `rules` is empty, the block always renders.
|
||||
|
||||
```typescript
|
||||
type Condition = {
|
||||
combinator?: 'and' | 'or'; // default: 'and'
|
||||
rules: ConditionRule[];
|
||||
};
|
||||
|
||||
type ConditionRule =
|
||||
| { type: 'variable_filled'; variable: string } // true if variable has a non-empty value
|
||||
| { type: 'variable_empty'; variable: string } // true if variable is absent or empty
|
||||
| { type: 'collab_count';
|
||||
operator: 'eq' | 'gt' | 'lt' | 'gte' | 'lte';
|
||||
value: number; // compared against number of assigned collaborators
|
||||
};
|
||||
```
|
||||
|
||||
`variable` refers to the resolved variable name (after team + video-level override merge). An unknown variable evaluates as empty. Unknown rule `type` values default to `true`.
|
||||
|
||||
---
|
||||
|
||||
### DescriptionBlock.variableDefinitions
|
||||
|
||||
Declares which custom variable tokens this block expects. Used by the UI to render per-block variable inputs.
|
||||
|
||||
```typescript
|
||||
type BlockVariableDefinition = {
|
||||
name: string; // token name (without braces), e.g. "sponsor_name"
|
||||
label: string; // human-readable label shown in the UI
|
||||
description?: string; // optional tooltip text
|
||||
defaultValue?: string; // pre-filled default
|
||||
};
|
||||
|
||||
// Stored as: BlockVariableDefinition[]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### BulkJob.filterSnapshot
|
||||
|
||||
Stores the original request that created the job. Shape varies by job type:
|
||||
|
||||
**Metadata bulk jobs** (`SET_PRIVACY`, `SET_TEMPLATE`, `ADD_TAGS`, `REMOVE_TAGS`, `SEARCH_REPLACE_TITLE`):
|
||||
```typescript
|
||||
{
|
||||
type: string; // job type, e.g. "SET_PRIVACY"
|
||||
videoIds?: string[]; // explicit video IDs, or...
|
||||
savedViewId?: string; // ...a saved view to resolve IDs from
|
||||
payload: {
|
||||
// SET_PRIVACY:
|
||||
privacyStatus?: 'PUBLIC' | 'PRIVATE' | 'UNLISTED';
|
||||
// SET_TEMPLATE:
|
||||
templateId?: string;
|
||||
// ADD_TAGS / REMOVE_TAGS:
|
||||
tags?: string[];
|
||||
// SEARCH_REPLACE_TITLE:
|
||||
search?: string;
|
||||
replace?: string;
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
**Sync-push bulk jobs** (`SYNC_PUSH`):
|
||||
```typescript
|
||||
{
|
||||
type: 'SYNC_PUSH';
|
||||
videoIds: string[]; // already-resolved and ownership-verified video IDs
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### BulkJob.rollbackData
|
||||
|
||||
**Unused.** This field exists in the schema but is never written or read by the application. Rollback is implemented entirely through `BulkJobItem.beforeSnapshot` — each item stores the pre-operation video snapshot, and the rollback endpoint restores it per-item. `rollbackData` is a vestigial field.
|
||||
|
||||
---
|
||||
|
||||
### BulkJobItem.beforeSnapshot / afterSnapshot
|
||||
|
||||
Stores a snapshot of the video's mutable fields before and after the bulk operation. Shape matches the fields relevant to the job type:
|
||||
|
||||
```typescript
|
||||
{
|
||||
title: string;
|
||||
tags: string[];
|
||||
privacyStatus: string;
|
||||
templateId: string | null;
|
||||
}
|
||||
```
|
||||
|
||||
Used by the rollback endpoint: `beforeSnapshot` is written back to `Video` directly via `prisma.video.update({ data: item.beforeSnapshot })`.
|
||||
|
||||
For SYNC_PUSH jobs, these hold the `youtubeSnapshot` before and after the push.
|
||||
|
||||
---
|
||||
|
||||
### ImportJob.validationReport
|
||||
|
||||
Shape differs by import type.
|
||||
|
||||
**CSV import:**
|
||||
```typescript
|
||||
{
|
||||
validCount: number;
|
||||
errorCount: number;
|
||||
errors: Array<{
|
||||
row: number; // 1-based row number in the CSV
|
||||
field: string; // field name that failed validation
|
||||
message: string; // zod validation message
|
||||
}>;
|
||||
}
|
||||
```
|
||||
|
||||
**JSON workspace import:**
|
||||
```typescript
|
||||
{ valid: true }
|
||||
```
|
||||
|
||||
JSON validation only checks that the top-level shape is correct (`version`, optional `videos`/`blocks`/`templates`/`collaborators` arrays). No per-item validation is performed during preview.
|
||||
|
||||
---
|
||||
|
||||
### ImportJob.mappingJson
|
||||
|
||||
Optional. Stores the column-to-field mapping the user selected during the CSV import wizard. Free-form shape — passed through from the frontend and not parsed by the backend during commit.
|
||||
|
||||
---
|
||||
|
||||
### JSON workspace import payload (`POST /imports/json/preview` + commit)
|
||||
|
||||
The workspace JSON format (used for both import and export):
|
||||
|
||||
```typescript
|
||||
{
|
||||
version: string; // schema version, currently "1.0"
|
||||
exportedAt?: string; // ISO datetime (present on exports, ignored on import)
|
||||
videos?: Video[]; // full Video rows
|
||||
videoConfigs?: VideoConfig[];
|
||||
blocks?: DescriptionBlock[];
|
||||
templates?: Template[];
|
||||
collaborators?: Collaborator[];
|
||||
savedViews?: SavedView[];
|
||||
}
|
||||
```
|
||||
|
||||
On commit, `collaborators`, `blocks`, and `templates` are upserted by `id`. The `teamId` from the current session overwrites whatever `teamId` is in the payload. `videos`, `videoConfigs`, and `savedViews` are **not** processed during JSON import commit — only the three content types above.
|
||||
|
||||
---
|
||||
|
||||
### ExportJob.scopeJson
|
||||
|
||||
**Unused.** `ExportJob` rows are never created by the application. The export service (`ExportsService`) returns data directly from the database without recording an export job. This model and field are schema artefacts with no active code path.
|
||||
|
||||
---
|
||||
|
||||
## Schema Rules
|
||||
|
||||
- **Never remove Prisma enum values** — PostgreSQL enum removal requires raw SQL migration and is risky. Mark deprecated values in UI instead.
|
||||
- **After any schema change:** stop backend → `npx prisma generate` → `npx prisma migrate deploy` → restart backend
|
||||
- **JSON columns** use Prisma `Json` type. Query with `array_contains` operator for JSON array fields (e.g. `collaboratorIds`)
|
||||
|
||||
## Related
|
||||
|
||||
- [[01 - System Overview]]
|
||||
- [[02 - Backend]]
|
||||
@@ -0,0 +1,112 @@
|
||||
# Queue System
|
||||
|
||||
BullMQ on Redis. The API process enqueues jobs; the worker process (`src/worker.ts`) consumes them.
|
||||
|
||||
## Critical Redis Requirement
|
||||
|
||||
Redis **must** run with `--maxmemory-policy noeviction`. BullMQ silently loses jobs if Redis uses `allkeys-lru`. This is pre-configured in `infrastructure/docker-compose.yml`.
|
||||
|
||||
## Queue Names
|
||||
|
||||
Defined in `backend/src/queues/queues.constants.ts`:
|
||||
|
||||
| Constant | Queue name |
|
||||
|---|---|
|
||||
| `QUEUES.YOUTUBE_SYNC` | `youtube-sync` |
|
||||
| `QUEUES.RENDER` | `render` |
|
||||
| `QUEUES.LINT` | `lint` |
|
||||
| `QUEUES.BULK_METADATA` | `bulk-metadata` |
|
||||
| `QUEUES.IMPORT` | `import` |
|
||||
| `QUEUES.CONFLICT_DETECTION` | `conflict-detection` |
|
||||
|
||||
## Processors
|
||||
|
||||
### `youtube-sync.processor.ts`
|
||||
**Trigger:** `POST /videos/:id/sync`
|
||||
**Job data:** `{ videoId }`
|
||||
**Behavior:**
|
||||
1. Calls `VideoRenderService` to render description and compute hash
|
||||
2. Skips push if hash matches `lastSyncedHash` (no changes)
|
||||
3. Calls `QuotaService.canSpend(50)` — rejects if quota exceeded
|
||||
4. Pushes all editable fields to YouTube API (`videos.update`)
|
||||
5. Updates `renderedDescription`, `lastSyncedHash`, `lastSyncedAt` on the video row
|
||||
|
||||
### `render.processor.ts`
|
||||
**Trigger:** `PUT /video-configs/:videoId` when the request body includes `"autoRender": true`
|
||||
**Job data:** `{ videoId }`
|
||||
**Behavior:** Renders description without pushing to YouTube. Updates `renderedDescription`.
|
||||
|
||||
The render job is not enqueued automatically on every config save — it is opt-in via the `autoRender` flag. `VideoConfigsService.upsert()` checks `if (dto.autoRender)` and only then calls `renderQueue.add(...)`. The frontend passes `autoRender: true` when saving from the video editor so the preview updates in the background. Saves that don't need an immediate re-render (e.g. bulk template applies) omit the flag and skip the queue.
|
||||
|
||||
### `lint.processor.ts`
|
||||
**Trigger:** `POST /lint/bulk`, `POST /lint/channel/:id`, `POST /lint/team`
|
||||
**Job data:** `{ videoId }`
|
||||
**Behavior:** Calls `LintingService.lintVideo(videoId)`. Replaces all unresolved `LintResult` rows and recomputes `Video.lintStatus`.
|
||||
|
||||
> **Note:** `POST /lint/videos/:id` (single video) is **synchronous** — it calls `LintingService.lintVideo()` directly in the HTTP handler and returns results immediately. It does not use this queue. Only bulk operations go through BullMQ.
|
||||
|
||||
### `bulk-metadata.processor.ts`
|
||||
**Trigger:** Bulk job confirmed by user
|
||||
**Job data:** `{ bulkJobId }`
|
||||
**Behavior:** Processes each `BulkJobItem` in sequence. Updates video fields, saves rollback snapshots, updates job status counters.
|
||||
|
||||
### `import.processor.ts`
|
||||
**Trigger:** CSV import committed (`POST /imports/csv/commit`)
|
||||
**Job data:** `{ importJobId }`
|
||||
**Behavior:** Calls `ImportsService.executeCommit()`, which marks the `ImportJob` as `committed` and writes an audit log entry. **Does not create or update any Video rows.** The preview step does not persist the validated rows, so the processor has no data to act on. CSV import is structurally incomplete — see the backlog.
|
||||
|
||||
### `conflict-detection.processor.ts`
|
||||
**Trigger:** BullMQ repeatable job registered on worker startup by `ConflictDetectionScheduler`. Cron pattern comes from `CONFLICT_DETECTION_CRON` (default `0 3 * * *`). Registration is gated by the global `CONFLICT_DETECTION_ENABLED` env var.
|
||||
**Job data:** `{}`
|
||||
**Behavior:**
|
||||
1. Loads all teams with `conflictDetectionEnabled: true`
|
||||
2. For each team, selects videos where `remoteConflict = false`, `youtubeDeletedAt = null`, and `lastSyncedAt < now - minAgeDays`, stalest-first, capped at `batchSize`
|
||||
3. Hands the selected video IDs to `YouTubeSyncService.detectConflictsForVideos()`, which groups by `channelId` (needed for OAuth) and issues one `videos.list` call per batch of up to 50 IDs. Cost is **1 quota unit per batch**, not per video (YouTube quota is per method-call, invariant to the number of parts or IDs). `QuotaService.canSpend(1)` is checked before each batch.
|
||||
4. Per video within a batch: on hash mismatch, writes `pendingRemoteSnapshot` + `pendingRemoteDescription` and sets `remoteConflict = true`. On hash match with a stale flag, clears the flag and pending fields — self-heals when the creator reverts an out-of-band edit.
|
||||
5. Stops the entire sweep as soon as quota is exhausted; resumes on the next cron tick.
|
||||
|
||||
`Team.conflictDetectionBatchSize` (1–500) and `conflictDetectionMinAgeDays` (≥0) are per-team knobs. Because the API cost is per batch (up to 50 videos each), a `batchSize` of 500 consumes ~10 units per team per run — not 500. Users resolve detected conflicts via `POST /videos/:id/accept-remote` (adopt remote, zero extra quota) or `POST /videos/:id/sync` (push local, overwrite remote).
|
||||
|
||||
## Frontend Job Completion Detection
|
||||
|
||||
There is **no push mechanism** (no SSE, no WebSocket, no EventEmitter). The frontend detects job completion entirely through polling via TanStack Query's `refetchInterval`.
|
||||
|
||||
### Sync queue status — dynamic polling
|
||||
|
||||
The `<Header>` component (`components/shared/Header.tsx`) polls `GET /youtube-sync/queue-status` continuously with an interval that adapts to queue state:
|
||||
|
||||
| Queue state | Poll interval |
|
||||
|---|---|
|
||||
| Active jobs running | 3 s |
|
||||
| Jobs waiting | 8 s |
|
||||
| Idle | 30 s |
|
||||
|
||||
This drives the sync-in-progress indicator in the header. When the queue drains, the indicator clears within one polling cycle.
|
||||
|
||||
### Video editor — scheduled invalidations
|
||||
|
||||
After `POST /videos/:id/sync` is enqueued, the video editor schedules two forced refetches of `['video', id]` — at **3 s** and **8 s** — to pick up the updated `hasPendingChanges` once the processor finishes. If the job takes longer than 8 s, the UI shows stale "push pending" status until the next sync-status poll triggers a broader refresh.
|
||||
|
||||
### Bulk jobs page — fixed polling
|
||||
|
||||
`/bulk-jobs` polls `GET /bulk-jobs` every **5 s** via `refetchInterval: 5000`. Job status transitions (pending → processing → completed) are reflected within one polling cycle.
|
||||
|
||||
### Linting page — fixed polling + immediate invalidation
|
||||
|
||||
`/linting` polls `GET /lint/results` every **30 s**. After enqueuing a bulk lint operation, the mutation's `onSuccess` immediately invalidates `['lintResults']` and `['videos']` for a faster first update.
|
||||
|
||||
### Implication
|
||||
|
||||
Because completion detection is polling-based, the UI does not reflect job results in real time — there is always a latency of up to one polling interval. For sync jobs that take longer than 8 s, the video editor in particular may lag. If tighter feedback is needed in future, SSE on the sync queue endpoint would be the natural addition.
|
||||
|
||||
---
|
||||
|
||||
## Job ID Deduplication
|
||||
|
||||
- Lint-once: `jobId: lint-{videoId}` — BullMQ ignores duplicate job IDs, so submitting the same video twice before the job runs only creates one job
|
||||
- Forced rerun: `jobId: lint-{videoId}-{timestamp}` — always creates a new job
|
||||
|
||||
## Related
|
||||
|
||||
- [[02 - Backend]]
|
||||
- [[05 - Local Setup]] (Redis setup)
|
||||
@@ -0,0 +1,112 @@
|
||||
# Authentication
|
||||
|
||||
## Flow
|
||||
|
||||
1. User clicks "Sign in with Google" → frontend redirects to `GET /auth/google`
|
||||
2. Backend redirects to Google OAuth consent screen
|
||||
3. Google redirects to `GET /auth/google/callback` with auth code
|
||||
4. Backend exchanges code for tokens, upserts `User` record, then branches on team membership (see below)
|
||||
5. Backend issues JWT access token (short-lived) + refresh token (long-lived)
|
||||
6. Frontend stores tokens; `api-client.ts` attaches `Authorization: Bearer <token>` to all requests
|
||||
7. On 401: `api-client.ts` automatically calls `POST /auth/refresh` and retries the original request
|
||||
|
||||
## First Login vs. Returning User
|
||||
|
||||
The OAuth callback handler (`AuthService.upsertGoogleUser`) checks whether the authenticating user already has any `TeamMember` rows.
|
||||
|
||||
### New user (no team memberships)
|
||||
|
||||
A full onboarding sequence runs automatically — no separate onboarding step or UI flow exists:
|
||||
|
||||
1. YouTube API is called with the login OAuth tokens to fetch the user's own channel (`channels.list?mine=true`), retrieving the YouTube channel ID, channel name, and uploads playlist ID.
|
||||
2. A `Team` is created using the user's display name as the team name, with a URL-safe slug derived from it.
|
||||
3. A `Channel` is created and linked to the team, with the YouTube OAuth tokens stored AES-256 encrypted.
|
||||
4. A `TeamMember` row is created linking the user to the team with role `OWNER`.
|
||||
5. The JWT is issued with this new team's ID.
|
||||
|
||||
This means the first Google login is both account creation **and** channel connection in one step. There is no separate "connect your channel" screen.
|
||||
|
||||
### Returning user (already has team memberships)
|
||||
|
||||
1. The user's existing `TeamMember` rows are loaded, ordered by `createdAt` ascending.
|
||||
2. The first membership's `teamId` and `role` are used for the JWT.
|
||||
3. The YouTube access token is refreshed on all `Channel` rows where `connectedBy = user.id`. The refresh token is only overwritten if Google returns a new one (which only happens on first auth or forced re-consent).
|
||||
|
||||
### Invited user
|
||||
|
||||
Invitations (`POST /teams/:teamId/members`) create a `TeamMember` row immediately. **The invitee must have already logged in at least once** — invite lookup is by `User.email`, and if no `User` row exists for that email the invite throws a 404. There is no pending-invite or email-notification system.
|
||||
|
||||
When an invited user subsequently logs in:
|
||||
- They already have at least one `TeamMember` row (the invite).
|
||||
- They fall into the **returning user** path — no new team is created.
|
||||
- If this is their very first login, their `User` record is created by `upsertGoogleUser` before the membership check, so the invite is found correctly.
|
||||
|
||||
## JWT Payload
|
||||
|
||||
```typescript
|
||||
{
|
||||
sub: string; // User.id
|
||||
email: string;
|
||||
teamId: string; // Active team for this session
|
||||
teamRole: TeamRole;
|
||||
}
|
||||
```
|
||||
|
||||
On first login the active team is the user's primary team (earliest `TeamMember.createdAt`). After calling `POST /auth/switch-team` the returned token carries the new team's ID and role.
|
||||
|
||||
## Team Switching
|
||||
|
||||
A user who belongs to multiple teams can switch the active team without logging out:
|
||||
|
||||
1. Call `GET /teams/mine` to get the list of teams the user belongs to.
|
||||
2. Call `POST /auth/switch-team` with `{ "teamId": "<target>" }` — returns `{ "accessToken": "..." }`.
|
||||
3. Replace the stored access token with the new one. All subsequent API calls are now scoped to the new team.
|
||||
|
||||
The backend validates that the user is actually a member of the requested team before issuing the new token.
|
||||
|
||||
**Current limitation:** The frontend has no UI for this. `POST /auth/switch-team` and `GET /teams/mine` are not called anywhere in the frontend codebase. Team switching is currently only possible via direct API calls.
|
||||
|
||||
## Guards
|
||||
|
||||
| Guard | Usage |
|
||||
|---|---|
|
||||
| `JwtAuthGuard` | Applied to all protected routes via `@UseGuards(JwtAuthGuard)` |
|
||||
| `RolesGuard` | Applied alongside JwtAuthGuard; uses `@Roles(TeamRole.EDITOR)` decorator |
|
||||
|
||||
## Team Roles
|
||||
|
||||
| Role | Permissions |
|
||||
|---|---|
|
||||
| `OWNER` | All permissions |
|
||||
| `ADMIN` | All permissions except ownership transfer |
|
||||
| `EDITOR` | Create/update/delete content |
|
||||
| `REVIEWER` | Read only (see note below) |
|
||||
| `READONLY` | Read only |
|
||||
|
||||
`OWNER` is assigned automatically when a team is created (first Google login). It cannot be assigned via the invite or role-update endpoints — both reject `role: OWNER` with `403 Forbidden`. There is no ownership transfer mechanism. The OWNER's role cannot be changed by anyone, and the OWNER cannot be removed from the team.
|
||||
|
||||
No endpoint is gated exclusively to `OWNER` — every `@Roles` guard that lists `OWNER` also lists `ADMIN`. Because the guard uses priority ordering, ADMIN users have identical feature access. If an OWNER's Google account becomes inaccessible, ADMIN members can continue using the application without restriction. The only consequence is that the orphaned `TeamMember` row with `role: OWNER` cannot be removed via the API (it would require direct DB access). Since the YouTube channel is bound to the same Google account as the OWNER, a lost Google account means the channel itself is also inaccessible — an in-app ownership transfer would not recover YouTube access.
|
||||
|
||||
> **REVIEWER and READONLY are currently functionally identical.** The `REVIEWER` role sits at priority 2 in the `RolesGuard` hierarchy (above `READONLY` at 1), but no endpoint specifies `REVIEWER` as a minimum role. The intended "comment" capability does not exist — there is no comment model, no comment endpoints, and no UI for comments. Until a comment system is built, assigning `REVIEWER` grants exactly the same access as `READONLY`. See the backlog.
|
||||
|
||||
## sf_session Cookie
|
||||
|
||||
`sf_session` is a **frontend-only routing flag** — it is not a backend session and carries no auth data. It is set to `1` by the frontend via `document.cookie` after a successful login or token refresh, and cleared on logout or failed refresh.
|
||||
|
||||
Its sole purpose is to give Next.js `middleware.ts` something to check. Middleware runs on the edge before the React app hydrates and cannot access Zustand or `localStorage` (where the real JWT lives). Reading this cookie is the only way the middleware can distinguish authenticated from unauthenticated requests and redirect accordingly.
|
||||
|
||||
The backend does not read or set `sf_session`. The backend sets a `refresh_token` httpOnly cookie for token refresh purposes.
|
||||
|
||||
**Limitation:** `sf_session` is a presence flag, not a validity check. If the cookie persists after a token is revoked (e.g. due to a crash before the clear runs), middleware lets the request through and the client gets 401s from the API, which then redirects to `/login` client-side. This is acceptable given the current architecture.
|
||||
|
||||
## Token Security
|
||||
|
||||
- YouTube OAuth tokens are AES-256 encrypted in the database using `TOKEN_ENCRYPTION_KEY` (must be exactly 32 characters)
|
||||
- JWT signing uses separate `JWT_SECRET` and `JWT_REFRESH_SECRET`
|
||||
- The backend sets a `refresh_token` httpOnly cookie on login and clears it on logout
|
||||
- Access tokens are stored in Zustand (persisted to `localStorage`) and attached to every request as `Authorization: Bearer <token>` by `api-client.ts`
|
||||
|
||||
## Related
|
||||
|
||||
- [[01 - System Overview]]
|
||||
- [[02 - Environment Variables]] (development)
|
||||
@@ -0,0 +1,151 @@
|
||||
# Render Engine
|
||||
|
||||
## Two Services
|
||||
|
||||
### VideoRenderService
|
||||
**Location:** `backend/src/shared/render-engine/video-render.service.ts`
|
||||
|
||||
The **single source of truth** for description rendering. Responsible for fetching all required data:
|
||||
- Description blocks (from `blockOrder`)
|
||||
- Block overrides and variable values (from `VideoConfig`)
|
||||
- Team variables
|
||||
- Campaign blocks (active ones are appended regardless of `blockOrder`)
|
||||
- Team date format and timezone
|
||||
- Playlists (for `{video.playlists}` token)
|
||||
- Collaborators (for `{collab.*}` tokens)
|
||||
|
||||
Then delegates computation to `RenderEngineService`.
|
||||
|
||||
**Used by:** `youtube-sync.processor`, `render.processor`, `videos.service.ts renderDescription()`
|
||||
|
||||
### RenderEngineService
|
||||
**Location:** `backend/src/shared/render-engine/render-engine.service.ts`
|
||||
|
||||
Pure computation. Takes `RenderInput`, returns `{ rendered: string, hash: string }`.
|
||||
|
||||
**Used directly by:** `video-configs.service.ts renderPreview()`, `templates.service.ts renderPreview()` — these callers already have all data fetched.
|
||||
|
||||
## Block Rendering by Type
|
||||
|
||||
| Block Type | Rendering Behavior |
|
||||
|---|---|
|
||||
| `STATIC` | Raw content output, zero token substitution |
|
||||
| `VARIABLE` | Resolves `{custom_var}`, `{video.*}`, `{collab.*}` tokens |
|
||||
| `CAMPAIGN` | Auto-appended at end when `campaign.startAt ≤ now ≤ campaign.endAt` |
|
||||
| `COLLABORATOR` | Cloned once per assigned collaborator, joined with `\n\n` (or `\n` if compact) |
|
||||
| `CONDITIONAL` | Evaluates `condition` JSON against video data |
|
||||
| `GLOBAL`, `REPEATABLE` | Deprecated; render as VARIABLE |
|
||||
|
||||
## Freetext Entries
|
||||
|
||||
`blockOrder` may contain IDs prefixed `freetext:` (e.g. `freetext:abc123`). These have no corresponding `DescriptionBlock` row. Their content comes entirely from `blockOverrides[id].content`. The `compact` flag for freetext entries also comes from `blockOverrides[id].compact`.
|
||||
|
||||
## Token Resolution
|
||||
|
||||
Resolution happens in three distinct passes per block, applied sequentially:
|
||||
|
||||
1. **Custom variable pass** — `resolveVariables(content, effectiveVars)` replaces `{custom_token}` placeholders. `effectiveVars` is built by merging team variables and video-level values: `{ ...teamVariables, ...variableValues }`. **Video-level values win on any name collision** — last-write-wins from the spread. System tokens (`video.*`, `collab.*`) are explicitly skipped here.
|
||||
2. **Video token pass** — `resolveVideoVars()` replaces `{video.*}` tokens using the video's own fields.
|
||||
3. **Collaborator token pass** — `resolveCollaborators()` or `expandCollaboratorBlock()` replaces `{collab.*}` tokens.
|
||||
|
||||
**Effective priority (highest → lowest):**
|
||||
1. System tokens (`{video.*}`, `{collab.*}`) — always win; cannot be shadowed by any variable name
|
||||
2. Video-level variables (`VideoConfig.variableValues`) — override team variables on conflict
|
||||
3. Team variables (`TeamVariable` table) — baseline for custom tokens
|
||||
|
||||
Because system tokens are skipped in pass 1 and resolved in dedicated passes afterwards, naming a team or video variable `video.title` has no effect — it is silently ignored and the actual video title is substituted instead.
|
||||
|
||||
`DescriptionBlock.variableDefinitions` declares which tokens a block expects but is never read by the render engine — it is metadata only (see Backlog).
|
||||
|
||||
Any token in `SYSTEM_VARIABLE_TOKENS` set is **skipped** by `resolveVariables()` and handled by its dedicated resolver. If you add a new `{video.*}` or `{collab.*}` token, it must be registered in the system variables registry.
|
||||
|
||||
## System Variable Tokens
|
||||
|
||||
**Video tokens:**
|
||||
|
||||
| Token | Resolves to |
|
||||
|---|---|
|
||||
| `{video.title}` | Video title |
|
||||
| `{video.tags}` | Comma-separated tag list |
|
||||
| `{video.category}` | YouTube category name (e.g. `Gaming`). Falls back to the raw numeric ID if the category is not in the known map. There is no `{video.categoryId}` token. |
|
||||
| `{video.scheduledAt}` | Scheduled publish date (date token — supports `\|format`) |
|
||||
| `{video.recordingDate}` | Recording date (date token — supports `\|format`) |
|
||||
| `{video.gameTitle}` | Game title field |
|
||||
| `{video.language}` | Default language code |
|
||||
| `{video.playlists}` | Comma-separated list of playlist **titles** (e.g. `Gaming, Tutorials`) |
|
||||
| `{video.playlistLinks}` | Comma-separated list of full YouTube playlist **URLs** (e.g. `https://www.youtube.com/playlist?list=PLxxx`) |
|
||||
|
||||
**Collaborator tokens:**
|
||||
`{collab.name}`, `{collab.youtube}`, `{collab.twitch}`, `{collab.instagram}`, `{collab.tiktok}`, `{collab.twitter}`, `{collab.bluesky}`, `{collab.discord}`, `{collab.aliases}`, `{collab.notes}`
|
||||
|
||||
Note: `{collab.youtube}` resolves to the full URL (`https://www.youtube.com/@handle`), not just the handle.
|
||||
|
||||
## Multi-Collaborator Rendering
|
||||
|
||||
Two different behaviors apply depending on block type:
|
||||
|
||||
### In VARIABLE, CONDITIONAL, CAMPAIGN, and freetext blocks
|
||||
`{collab.*}` tokens resolve against the **first** collaborator in `Video.collaboratorIds` whose record is found in the loaded collaborator list. If no collaborator is assigned, all `{collab.*}` tokens are left unreplaced.
|
||||
|
||||
This means if a video has three collaborators, `{collab.name}` in a VARIABLE block only produces the first collaborator's name — not all three.
|
||||
|
||||
### In COLLABORATOR blocks
|
||||
The block's content template is expanded **once per assigned collaborator**, in the order they appear in `collaboratorIds`. Each expansion resolves all `{collab.*}` tokens against that specific collaborator. The resulting strings are joined with `\n\n` (double newline), or `\n` if the block has `compact: true`.
|
||||
|
||||
**Example** — with two collaborators (Alice, Bob) and a COLLABORATOR block containing:
|
||||
```
|
||||
🎮 {collab.name} — {collab.youtube}
|
||||
```
|
||||
Renders as:
|
||||
```
|
||||
🎮 Alice — https://www.youtube.com/@alice
|
||||
|
||||
🎮 Bob — https://www.youtube.com/@bob
|
||||
```
|
||||
|
||||
If a collaborator has no value for a given token (e.g. no `youtubeLink`), that token resolves to an empty string.
|
||||
|
||||
## Hash Computation
|
||||
|
||||
`hashMetadata()` in `shared/render-engine/hash.ts` hashes these fields: `title`, `description`, `tags`, `categoryId`, `privacyStatus`, `defaultLanguage`, `defaultAudioLanguage`, `selfDeclaredMadeForKids`, `embeddable`, `license`, `recordingDate`.
|
||||
|
||||
The hash is stored as `Video.lastSyncedHash` after a successful YouTube push. `hasPendingChanges` is computed by comparing current state hash against `lastSyncedHash`.
|
||||
|
||||
## Date Format Override Syntax
|
||||
|
||||
Date tokens support an inline format override using `|` as a separator:
|
||||
|
||||
```
|
||||
{video.scheduledAt|DD.MM.YYYY}
|
||||
{video.recordingDate|MMMM D, YYYY}
|
||||
```
|
||||
|
||||
**Only date tokens support this.** The two date tokens are `video.scheduledAt` and `video.recordingDate`. All other `video.*` and `collab.*` tokens are non-date (`kind: 'simple'`) — the regex captures the `|format` portion for all tokens but non-date resolvers ignore it.
|
||||
|
||||
**Format string tokens** (custom implementation, not strftime or moment.js):
|
||||
|
||||
| Token | Output |
|
||||
|---|---|
|
||||
| `YYYY` | 4-digit year (e.g. `2024`) |
|
||||
| `YY` | 2-digit year (e.g. `24`) |
|
||||
| `MMMM` | Full month name (e.g. `January`) |
|
||||
| `MMM` | Short month name (e.g. `Jan`) |
|
||||
| `MM` | Zero-padded month (e.g. `01`) |
|
||||
| `M` | Month without padding (e.g. `1`) |
|
||||
| `DD` | Zero-padded day (e.g. `05`) |
|
||||
| `D` | Day without padding (e.g. `5`) |
|
||||
|
||||
**Priority:** inline token format → team `dateFormat` setting → `YYYY-MM-DD` (hard default).
|
||||
|
||||
**Invalid format strings** are silently passed through as-is. Any character or sequence not matching a known token is output verbatim — no error, no warning.
|
||||
|
||||
**Null date:** If the date field is null or undefined (e.g. `video.recordingDate` not set), the token resolves to an empty string regardless of format.
|
||||
|
||||
## Date Formatting Implementation
|
||||
|
||||
`applyDateFormat()` in `render-engine.service.ts` uses a single-pass regex replacement to avoid re-substitution bugs. Do not convert to chained `.replace()` calls.
|
||||
|
||||
## Related
|
||||
|
||||
- [[02 - Backend]]
|
||||
- [[02 - Description Engine]] (features)
|
||||
Reference in New Issue
Block a user