Skill · Version 1.0.0

Video and audio processing

Builds media processing flows that validate uploads, produce playback-ready formats and keep expensive work out of web requests.

video-audio-processing

See how it helps

When is it useful?

Use when uploaded videos load slowly, processing blocks the application or failed conversions are difficult to diagnose.

What does it help you do?

Inspects real streams, dimensions and duration before selecting a conversion path.
Prepares browser-friendly video, adaptive streams, preview frames and audio waveforms as needed.
Bounds background processing and temporary storage, with visible progress and useful failure categories.

Try asking this

Plan a training-video upload flow. Validate the file, generate a playback-ready version and poster image, and show clear progress and recovery when conversion fails. Upload a valid lesson video and a file with unsupported streams. The first should become playable; the second should show a useful failure without consuming unbounded resources.

Detailed documentation

Read the skill’s original instructions, examples and checks in their authored language.

Details…Show less

Video & Audio Processing

Video and audio transcoding are among the most CPU-, disk- and time-intensive operations in a backend system. Design media pipelines around upfront stream probing, safe subprocess execution, streaming-friendly container layouts and resumable background jobs.

1. Upfront Probing & Validation (`ffprobe`)

Before running an expensive ffmpeg transcode, inspect the container and streams with ffprobe (-v error -show_format -show_streams -of json):

Verify actual streams: confirm whether the file contains a valid video stream (codec_type === "video"), audio stream (codec_type === "audio"), duration, dimensions (width, height), frame rate and bitrate.
Enforce media bounds: reject files exceeding maximum duration, resolution or stream count, or files using unsupported/dangerous external reference tracks (such as HLS playlist files containing file:// or internal http:// segments!).
Disable network protocols on untrusted inputs: pass -protocol_whitelist file,pipe (or local file only) when probing or decoding user-uploaded files so a crafted playlist container cannot trigger SSRF or local file inclusion.

2. Web-Optimized Transcoding & Delivery Formats

Progressive MP4 (+faststart):
- When outputting MP4 (H.264/AAC), always pass -movflags +faststart so the moov atom (index table) is moved to the front of the file. Without +faststart, browsers must download the entire multi-hundred-MB file before playback can start!
- Normalize pixel format (-pix_fmt yuv420p) and audio sample rate/channels (-ac 2 -ar 48000) for universal browser and mobile hardware decoder compatibility.
Adaptive Bitrate Streaming (HLS / DASH):
- For longer or high-resolution videos, generate an HLS master playlist (.m3u8) and segmented renditions (360p, 720p, 1080p .ts or fMP4 .m4s segments) so players adapt smoothly to mobile network conditions.
- Remember that HLS produces many small files (manifest + hundreds of segments): track the rendition prefix cleanly in file_derivatives so storage cleanup deletes the entire segment prefix when the parent video is deleted.
Poster Frames, Timeline Sprites & Audio Waveforms:
- Extract a representative poster frame (-ss keyframe + WebP/JPEG output) and, for audio files, precompute normalized peak/RMS waveform JSON arrays during background processing so the UI player renders instantly.

3. Worker Isolation, Progress & Failure Handling

Never transcode on web/API nodes: run media jobs on dedicated worker queues with bounded concurrency and scratch disk limits.
Skip redundant transcoding (Stream Copy): if ffprobe shows an uploaded video is already within target resolution/bitrate in h264/aac, remux with -c copy -movflags +faststart in seconds instead of re-encoding from scratch.
Always clean up scratch files in finally blocks and record clear error classifications (corrupt container vs. missing codec vs. worker timeout).

Included files 1

SKILL.md