Compare commits

...
8 Commits
Author SHA1 Message Date
devil 49cfe2d8e0 fix(schema): cascade-delete VideoConfig/LintResult/BulkJobItem/VideoPlaylist with their Video
Channel import crashed on the hard-delete purge path (videos confirmed
gone from YouTube for 30+ days): prisma.video.deleteMany() hit a FK
violation on VideoConfig_videoId_fkey. All four required relations
pointing at Video had no onDelete behavior set, defaulting to RESTRICT
- any of the four would have hit the same crash, not just VideoConfig.
First real channel import against a real, aged dataset is what
surfaced this; nothing had exercised the 30-day hard-delete path before.
2026-08-11 17:14:20 +02:00
devil 0ac1ca3b08 fix(scripts): disable MSYS path translation in migrate-local-db-to-server.sh
pg_dump inside the container failed with "could not open output file
C:/Users/.../Temp/studioflow_migration_....dump" - Git Bash/MSYS was
rewriting the /tmp/... argument into a Windows path before handing it
to docker.exe, even though it's meant to be interpreted inside the
Linux container. MSYS_NO_PATHCONV=1 stops that translation.
2026-08-11 17:08:09 +02:00
devil b8c4cebad5 chore(scripts): add local-to-server DB migration script
One-time tool: dumps the local dev Postgres, scp's it to the server,
and restores it into the remote postgres container over SSH - drops
and recreates the target database, so it prompts for an explicit YES
before touching the remote side. Connection details are placeholders
(SSH_USER/SSH_HOST/etc.) filled in via env vars or direct edits, never
committed with real values.
2026-08-11 16:45:24 +02:00
devil baf827df29 fix(auth): set supplementalVideoIds on first-login channel creation
upsertGoogleUser's nested channels.create (the auto-provisioning path
for a brand new team's first Google login) never set
supplementalVideoIds, a required String[] column with no DB default
(dropped intentionally in 20260608000002 to match the Prisma schema).
channel-import.service.ts already sets it explicitly on its create path
- this path was just missed, and nothing had exercised a real first
login against Postgres until now.
2026-08-11 16:33:22 +02:00
devil ba409ca53b fix(backend): compiled output was nested under dist/src, not dist/
api container crashed on boot: Cannot find module '/app/dist/main.js'.
Root cause: tsconfig.json had no rootDir, and prisma/make-admin.ts (run
separately via ts-node, never part of the Nest build) was swept into
the nest build compilation since there was no exclude either - so tsc
inferred the output root as the project root and nested everything
under dist/src/ instead of the flat dist/main.js the Dockerfile,
package.json scripts, and compose's worker command all assume.

Also excludes tsconfig.tsbuildinfo from the Docker build context - a
stale local incremental-build cache file was leaking in via `COPY . .`
(only dist/node_modules were dockerignored) and made nest build skip
emitting .js files entirely on a from-scratch container build once the
rootDir fix invalidated its cached signature.
2026-08-11 16:10:44 +02:00
devil 811c14ee73 fix(infrastructure): don't let a slow/unhealthy api block deploy
Portainer's compose up blocks on frontend's api: condition:
service_healthy dependency and, on timeout, tears down everything it
just created - so an unhealthy api container never stuck around long
enough to pull logs from. Relaxed to service_started (frontend doesn't
need api ready at container-start), and gave the api healthcheck more
runway (60s start_period, 10 retries @ 10s) in case it's just slow to
boot rather than crashing.
2026-08-11 15:53:06 +02:00
devil ba8c7185a7 fix(infrastructure): use the actual Traefik network name (proxy)
The external network on the target server is named "proxy", not
traefik-network as originally assumed - deploy was failing with
"network traefik-network not found". Updated the compose file and
both the infra README and vault deployment doc to match.
2026-08-11 15:43:06 +02:00
devil b122ab4ac0 chore(infrastructure): force image re-pull on redeploy for Portainer
pull_policy: always on the four studioflow services (migrate, api,
worker, frontend) so redeploying the stack in Portainer actually fetches
the latest pushed image instead of reusing a stale local layer for the
mutable `latest`/IMAGE_TAG reference. Third-party pinned images
(postgres, redis) are left on default pull behavior.
2026-08-11 15:18:04 +02:00
10 changed files with 140 additions and 15 deletions
+1
View File
@@ -3,6 +3,7 @@ node_modules/
# Build output
backend/dist/
backend/tsconfig.tsbuildinfo
frontend/.next/
frontend/tsconfig.tsbuildinfo
+1
View File
@@ -1,5 +1,6 @@
node_modules
dist
tsconfig.tsbuildinfo
.env
.env.development
.env.production
@@ -0,0 +1,23 @@
-- DropForeignKey
ALTER TABLE "BulkJobItem" DROP CONSTRAINT "BulkJobItem_videoId_fkey";
-- DropForeignKey
ALTER TABLE "LintResult" DROP CONSTRAINT "LintResult_videoId_fkey";
-- DropForeignKey
ALTER TABLE "VideoConfig" DROP CONSTRAINT "VideoConfig_videoId_fkey";
-- DropForeignKey
ALTER TABLE "VideoPlaylist" DROP CONSTRAINT "VideoPlaylist_videoId_fkey";
-- AddForeignKey
ALTER TABLE "VideoConfig" ADD CONSTRAINT "VideoConfig_videoId_fkey" FOREIGN KEY ("videoId") REFERENCES "Video"("id") ON DELETE CASCADE ON UPDATE CASCADE;
-- AddForeignKey
ALTER TABLE "LintResult" ADD CONSTRAINT "LintResult_videoId_fkey" FOREIGN KEY ("videoId") REFERENCES "Video"("id") ON DELETE CASCADE ON UPDATE CASCADE;
-- AddForeignKey
ALTER TABLE "BulkJobItem" ADD CONSTRAINT "BulkJobItem_videoId_fkey" FOREIGN KEY ("videoId") REFERENCES "Video"("id") ON DELETE CASCADE ON UPDATE CASCADE;
-- AddForeignKey
ALTER TABLE "VideoPlaylist" ADD CONSTRAINT "VideoPlaylist_videoId_fkey" FOREIGN KEY ("videoId") REFERENCES "Video"("id") ON DELETE CASCADE ON UPDATE CASCADE;
+4 -4
View File
@@ -174,7 +174,7 @@ enum LintStatus {
model VideoConfig {
id String @id @default(cuid())
videoId String @unique
video Video @relation(fields: [videoId], references: [id])
video Video @relation(fields: [videoId], references: [id], onDelete: Cascade)
templateId String?
blockOrder Json
blockOverrides Json
@@ -312,7 +312,7 @@ model SavedView {
model LintResult {
id String @id @default(cuid())
videoId String
video Video @relation(fields: [videoId], references: [id])
video Video @relation(fields: [videoId], references: [id], onDelete: Cascade)
ruleCode String
severity LintSeverity
targetField String?
@@ -369,7 +369,7 @@ model BulkJobItem {
bulkJobId String
bulkJob BulkJob @relation(fields: [bulkJobId], references: [id])
videoId String
video Video @relation(fields: [videoId], references: [id])
video Video @relation(fields: [videoId], references: [id], onDelete: Cascade)
beforeSnapshot Json?
afterSnapshot Json?
status String @default("pending")
@@ -415,7 +415,7 @@ model VideoPlaylist {
videoId String
playlistId String
position Int?
video Video @relation(fields: [videoId], references: [id])
video Video @relation(fields: [videoId], references: [id], onDelete: Cascade)
playlist Playlist @relation(fields: [playlistId], references: [id])
@@id([videoId, playlistId])
+1
View File
@@ -67,6 +67,7 @@ export class AuthService {
youtubeChannelId,
name: channelName,
uploadsPlaylistId,
supplementalVideoIds: [],
youtubeAccessToken: this.encrypt(accessToken),
youtubeRefreshToken: refreshToken ? this.encrypt(refreshToken) : undefined,
youtubeTokenExpiry: new Date(Date.now() + 3600 * 1000),
+3 -1
View File
@@ -9,6 +9,7 @@
"target": "ES2021",
"sourceMap": true,
"outDir": "./dist",
"rootDir": "./src",
"baseUrl": "./",
"incremental": true,
"skipLibCheck": true,
@@ -17,5 +18,6 @@
"strictBindCallApply": false,
"forceConsistentCasingInFileNames": false,
"noFallthroughCasesInSwitch": false
}
},
"exclude": ["node_modules", "dist", "prisma"]
}
@@ -17,7 +17,7 @@ The production setup is a single-host Docker Compose deployment using **Traefik*
| `postgres` | postgres:16-alpine | Database — bound to `127.0.0.1:5432` (not public) |
| `redis` | redis:7-alpine | BullMQ queues — `noeviction` policy, AOF persistence |
**SSL / routing**: Traefik handles TLS termination with automatic Let's Encrypt certificates. Both API (`/api` prefix) and frontend run on the same domain — Traefik routes by path prefix. The `traefik-network` external network must exist before deploy.
**SSL / routing**: Traefik handles TLS termination with automatic Let's Encrypt certificates. Both API (`/api` prefix) and frontend run on the same domain — Traefik routes by path prefix. The `proxy` external network (the host's existing Traefik network) must exist before deploy.
**Build**: Multi-stage Dockerfile (`node:22-alpine`). Builder compiles TypeScript; runner installs prod-only deps. The Prisma CLI is copied from builder stage so the `migrate` service can run schema migrations.
+2 -2
View File
@@ -24,8 +24,8 @@ Registry (`git.devils.zone/devil/youtube-studio-flow-{backend,frontend}`), gebau
6. Container starten: `docker compose up -d`
Für ein Update auf eine neue Version: `IMAGE_TAG` in `.env` anpassen (oder `latest`
belassen), dann Schritte 46 wiederholen. Das externe Docker-Netzwerk `traefik-network`
muss vorher existieren (`docker network create traefik-network`), falls Traefik das nicht
belassen), dann Schritte 46 wiederholen. Das externe Docker-Netzwerk `proxy`
muss vorher existieren (`docker network create proxy`), falls Traefik das nicht
bereits selbst anlegt.
## Daten-Persistenz
+14 -7
View File
@@ -3,6 +3,7 @@ services:
# Runs database migrations once before the API and worker start.
migrate:
image: git.devils.zone/devil/youtube-studio-flow-backend:${IMAGE_TAG:-latest}
pull_policy: always
command: npx prisma migrate deploy
restart: "no"
environment:
@@ -15,6 +16,7 @@ services:
api:
image: git.devils.zone/devil/youtube-studio-flow-backend:${IMAGE_TAG:-latest}
pull_policy: always
restart: unless-stopped
environment:
NODE_ENV: production
@@ -44,16 +46,17 @@ services:
condition: service_completed_successfully
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://localhost:3001/api/v1/health || exit 1"]
interval: 15s
interval: 10s
timeout: 5s
retries: 3
start_period: 30s
retries: 10
start_period: 60s
networks:
- app-network
- traefik-network
- proxy
worker:
image: git.devils.zone/devil/youtube-studio-flow-backend:${IMAGE_TAG:-latest}
pull_policy: always
restart: unless-stopped
command: node dist/worker.js
environment:
@@ -82,6 +85,7 @@ services:
# Dockerfile default, /api/v1) — the relative path works because Traefik serves both
# frontend and API on the same domain.
image: git.devils.zone/devil/youtube-studio-flow-frontend:${IMAGE_TAG:-latest}
pull_policy: always
restart: unless-stopped
labels:
- "traefik.enable=true"
@@ -91,11 +95,14 @@ services:
- "traefik.http.routers.studioflow-frontend.priority=1"
- "traefik.http.services.studioflow-frontend.loadbalancer.server.port=3000"
depends_on:
# service_started, not service_healthy: a slow/unhealthy api shouldn't block the
# whole `compose up` (and Portainer tearing down what it created) - the frontend
# itself doesn't need api to be ready at container-start time.
api:
condition: service_healthy
condition: service_started
networks:
- app-network
- traefik-network
- proxy
postgres:
image: postgres:16-alpine
@@ -142,5 +149,5 @@ volumes:
networks:
app-network:
driver: bridge
traefik-network:
proxy:
external: true
+90
View File
@@ -0,0 +1,90 @@
#!/usr/bin/env bash
# One-time migration: dump the local dev Postgres and restore it into the
# server's Postgres over SSH, replacing whatever is there.
#
# DESTRUCTIVE on the target: drops and recreates the remote database.
#
# Fill in the placeholders below (or export them as env vars before running),
# then run from the project root: bash scripts/migrate-local-db-to-server.sh
#
# Prerequisites:
# - Local Postgres running: cd infrastructure && docker compose up -d postgres redis
# - SSH access to the server
# - The remote postgres container's name (find it with `docker ps` on the
# server, or check the container list in Portainer for your stack - it'll
# be something like <stack-name>-postgres-1)
set -euo pipefail
# Git Bash/MSYS auto-translates POSIX-looking path args (e.g. /tmp/foo) into
# Windows paths before handing them to docker.exe - but these paths are meant
# to be interpreted inside the Linux container, not on the Windows host.
export MSYS_NO_PATHCONV=1
# ── Fill these in ──────────────────────────────────────────────────────────
SSH_USER="${SSH_USER:-dummyuser}"
SSH_HOST="${SSH_HOST:-dummy.server.example}"
SSH_PORT="${SSH_PORT:-22}"
REMOTE_POSTGRES_CONTAINER="${REMOTE_POSTGRES_CONTAINER:-studioflow-postgres-1}"
REMOTE_POSTGRES_USER="${REMOTE_POSTGRES_USER:-studioflow}"
REMOTE_POSTGRES_DB="${REMOTE_POSTGRES_DB:-studioflow}"
# ────────────────────────────────────────────────────────────────────────────
LOCAL_DB_USER="studioflow"
LOCAL_DB_NAME="studioflow"
DUMP_FILE="studioflow_migration_$(date +%Y%m%d_%H%M%S).dump"
echo "==> Finding local Postgres container"
LOCAL_PG_CONTAINER=$(docker ps --format '{{.Names}}' | grep -i postgres | head -1)
if [ -z "$LOCAL_PG_CONTAINER" ]; then
echo "No running local postgres container found. Start it first:"
echo " cd infrastructure && docker compose up -d postgres redis"
exit 1
fi
echo " Using: $LOCAL_PG_CONTAINER"
echo "==> Dumping local database ($LOCAL_DB_NAME)"
docker exec "$LOCAL_PG_CONTAINER" pg_dump -U "$LOCAL_DB_USER" -d "$LOCAL_DB_NAME" -F c -f "/tmp/$DUMP_FILE"
docker cp "${LOCAL_PG_CONTAINER}:/tmp/$DUMP_FILE" "./$DUMP_FILE"
docker exec "$LOCAL_PG_CONTAINER" rm "/tmp/$DUMP_FILE"
echo " Dump size: $(du -h "./$DUMP_FILE" | cut -f1)"
echo ""
echo "About to overwrite the database on ${SSH_HOST} (container: ${REMOTE_POSTGRES_CONTAINER})."
read -p "Type YES to continue: " CONFIRM
if [ "$CONFIRM" != "YES" ]; then
echo "Aborted. Local dump kept at ./$DUMP_FILE"
exit 1
fi
echo "==> Copying dump to server"
scp -P "$SSH_PORT" "./$DUMP_FILE" "${SSH_USER}@${SSH_HOST}:/tmp/$DUMP_FILE"
echo "==> Restoring on server"
ssh -p "$SSH_PORT" "${SSH_USER}@${SSH_HOST}" bash -s <<EOF
set -euo pipefail
echo " Copying dump into container..."
docker cp "/tmp/$DUMP_FILE" "${REMOTE_POSTGRES_CONTAINER}:/tmp/$DUMP_FILE"
echo " Terminating existing connections..."
docker exec "$REMOTE_POSTGRES_CONTAINER" psql -U "$REMOTE_POSTGRES_USER" -d postgres -c \
"SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = '$REMOTE_POSTGRES_DB';" >/dev/null
echo " Dropping and recreating database..."
docker exec "$REMOTE_POSTGRES_CONTAINER" psql -U "$REMOTE_POSTGRES_USER" -d postgres -c \
"DROP DATABASE IF EXISTS $REMOTE_POSTGRES_DB;" >/dev/null
docker exec "$REMOTE_POSTGRES_CONTAINER" psql -U "$REMOTE_POSTGRES_USER" -d postgres -c \
"CREATE DATABASE $REMOTE_POSTGRES_DB OWNER $REMOTE_POSTGRES_USER;" >/dev/null
echo " Restoring data..."
docker exec "$REMOTE_POSTGRES_CONTAINER" pg_restore -U "$REMOTE_POSTGRES_USER" -d "$REMOTE_POSTGRES_DB" --no-owner --no-privileges "/tmp/$DUMP_FILE"
docker exec "$REMOTE_POSTGRES_CONTAINER" rm "/tmp/$DUMP_FILE"
rm "/tmp/$DUMP_FILE"
echo " Done."
EOF
echo ""
echo "==> Migration complete."
echo " Restart api/worker in Portainer (or they'll self-heal on next DB query)."
echo " Local dump kept at ./$DUMP_FILE - delete it once you've confirmed the server looks right."