The Challenge: Create and scale a highly viral, surreal, and visually consistent cinematic universe (“The Jelly Kingdom”) across major social channels without manual editing, while navigating strict API content filters and maintaining absolute control over complex fluid physics and retro-futuristic aesthetics.
The Solution: Engineered a parametric, multi-model AI video automation pipeline that constructs, renders, and scores multi-scene narrative sequences entirely from a single JSON configuration file. By programmatically orchestrating state-of-the-art AI video generators and audio engines, the system generates high-fidelity, physics-compliant vertical videos on autopilot.
To power this surreal world-building, I designed and developed an advanced generation factory with the following capabilities:
-
Parametric Park-Architect Engine: Implemented a configuration-driven pipeline where park variables—such as themes (e.g., 1950s cinematic realism), character personas, physical material properties (e.g., jelly viscosity and translucent structural density), and attraction types (e.g., “Impossible Slide Loops” or “Syrup Möbius Strip”)—are programmatically generated and updated by a central LLM.
-
Multi-Model API Orchestration: Built a modular backend that dynamically selects and integrates top-tier generative models. It uses cutting-edge video APIs (like Kling) and advanced image models (such as Nanobana a2 and Pro versions) to output high-resolution, photorealistic assets (Kodak Portra 400 aesthetic) in vertical 9:16 aspect ratio.
-
Content Policy Failsafe (Auto-Rewrite): Designed a defensive prompt-engineering layer that automatically intercepts API content policy violations. If a human-related trigger is flagged, the system deploys a “Ghost Motion Protocol”—automatically rewriting descriptions into abstract kinetic physics (“articulated silhouettes,” “elastic vibrations”) to bypass safety filters while preserving the visual intent and original image-to-video seed integrity.
The Video Factory Architecture & Codebase
This automation suite is structured to orchestrate complex narrative structures, seamless scene-to-scene transitions, and dynamic audio overlays in a single automated run.
Key Automation Features:
- Custom Sequence & Frame Control: The pipeline handles precise sequencing, allowing full customization of scene count, individual video durations, and seamless handovers where the final frame of a scene (e.g., Scene 7) is fed as the starting seed for the next (e.g., Scene 8), ensuring continuous first-person POV motion.
- Dual-LLM Writing & Economy Mode: Features an abstract LLM layer that utilizes cost-efficient local processing blocks and caching to minimize API token consumption while generating highly structured narrative intents and prompt outputs.
- Algorithmic Audio & Soundscapes: Integrates ElevenLabs to generate tailored, cinematic soundscapes. The engine dynamically maps and injects audio prompts (e.g., retro-futurist orchestrations, wind roar, and gurgling elastic fluid effects) onto specific scenes within the sequence array.
- Automated Multi-Platform SEO Metadata: Automatically processes and exports optimized high-RPM meta assets—generating hooks, descriptions, and hashtags tailored for TikTok, Instagram, Pinterest, and YouTube Shorts in seconds.
Key Deliverables & Impact:
-
Architectural Consistency: Achieved 100% visual and material consistency across a 9-scene narrative sequence, rendering complex translucent structures and customized high-viscosity fluid dynamics.
-
Automated Safety Bypassing: Successfully processed and completed 100% of generation runs without filter-blocking, thanks to the dynamic abstract-rewrite engine.
-
Optimized Asset Pipeline: Reduced rendering and assembly workflows from hours of manual video editing to minutes of automated processing.
Core Tech Stack: Python (Parametric Pipelines & Asynchronous Video Assembly), Kling & Midjourney APIs, ElevenLabs Audio API, Multi-Agent LLM Orchestrator (with Smart Caching), JSON-Schema Configurations.