Video Pipeline

sequenceDiagram participant User participant App participant API participant S3Upload as S3 Uploads participant MC as MediaConvert participant S3Proc as S3 Processed participant Trans as Transcribe participant OS as OpenSearch participant DDB as DynamoDB participant CF as CloudFront User->>App: Record video App->>API: Request upload URL API->>S3Upload: Generate presigned URL S3Upload-->>API: Presigned URL API-->>App: Upload URL App->>S3Upload: Upload video directly S3Upload->>MC: Trigger (S3 event) MC->>MC: Transcode to 720p, 480p, 360p MC->>S3Proc: Store processed videos MC->>API: Job complete event API->>Trans: Start transcription Trans->>Trans: Extract text from audio Trans-->>API: Transcription result API->>OS: Index content + transcript API->>DDB: Update content status API-->>App: Content ready User->>App: View video App->>API: Request video API->>CF: Generate signed URL CF-->>App: Stream video

flowchart LR subgraph Input["Input"] Original["Original Upload\n(any resolution)"] end subgraph Processing["MediaConvert / FFmpeg"] MC["Transcoder"] end subgraph Output["Output Profiles"] P1080["1080p\n1920x1080\n5000 kbps"] P720["720p\n1280x720\n2500 kbps"] P480["480p\n854x480\n1000 kbps"] P360["360p\n640x360\n500 kbps"] Thumb["Thumbnail\n640x360\nJPEG"] end Original --> MC MC --> P1080 MC --> P720 MC --> P480 MC --> P360 MC --> Thumb

flowchart TB subgraph Upload["Upload Phase"] Raw["Raw Upload\nS3 Standard"] end subgraph Active["Active Storage (0-30 days)"] Processed["Processed Videos\nS3 Standard"] end subgraph Warm["Warm Storage (30-90 days)"] IT["S3 Intelligent Tiering"] end subgraph Cold["Cold Storage (90+ days)"] Glacier["S3 Glacier\nTime Capsules"] end subgraph Archive["Deep Archive (1+ year)"] DA["S3 Glacier Deep Archive\nLong-term Time Capsules"] end Raw -->|Processed & deleted| Processed Processed -->|Auto-tier| IT IT -->|Time capsule| Glacier Glacier -->|50+ year capsule| DA Glacier -.->|Unlock request| Processed DA -.->|Unlock request\n12hr restore| Glacier

flowchart TB subgraph App["Application"] VP["VideoProcessor Interface"] end subgraph Local["Local Development"] FFmpeg["FFmpeg Worker"] MinIO["MinIO Storage"] Redis["Redis Queue"] end subgraph AWS["AWS Production"] MC["MediaConvert"] S3["S3 Storage"] SQS["SQS Queue"] end VP -->|Local env| FFmpeg VP -->|AWS env| MC FFmpeg --> MinIO MC --> S3 FFmpeg -.-> Redis MC -.-> SQS