Auto-Add Stock Footage B-roll to Generated Shorts

[shreesha345]

Problem

Generated shorts only use original video footage, which can be repetitive and less engaging. Adding relevant B-roll footage would make shorts more professional and increase viewer retention.

Solution

Automatically integrate non-copyrighted stock footage into generated shorts:

1. Analyze the video transcription to understand topics being discussed
2. Search free stock footage APIs (Pexels, Pixabay) for relevant clips
3. Fit footage to 9:16 vertical format using AI-powered smart cropping
4. Integrate B-roll seamlessly into the timeline with smooth transitions

Example

For a 60-second clip about "AI and startups":

• 0:00-0:15 - Original footage
• 0:15-0:25 - B-roll: Tech office workspace (auto-fitted to 9:16)
• 0:25-0:45 - Original footage
• 0:45-0:55 - B-roll: AI visualization graphics (auto-fitted to 9:16)
• 0:55-1:00 - Original footage

Benefits

• More engaging and professional-looking shorts
• Better visual variety keeps viewers watching
• Automated - no manual B-roll searching needed
• Free stock footage - no licensing costs

To-Do

Phase 1: Stock Footage Integration

• Set up Pexels API integration
• Set up Pixabay API integration
• Build search functionality
• Add caching system for downloaded footage
• Track licenses and attribution

Phase 2: Content Analysis

• Extract keywords from transcription using NLP
• Generate relevant search queries
• Rank search results by relevance
• Determine optimal B-roll insertion points

Phase 3: AI-Powered 9:16 Fitting

• Implement object detection (YOLO/MediaPipe)
• Build smart cropping algorithm
• Add subject tracking for dynamic cropping
• Test with various footage types

Phase 4: Video Composition

• Integrate B-roll into video timeline
• Add transition effects (crossfade, dissolve)
• Maintain caption/audio sync
• Render final composite video

Phase 5: Configuration & Polish

• Add user settings (enable/disable, coverage %, style)
• Create config file options
• Add CLI arguments
• Performance optimization
• Testing and bug fixes

Configuration Options

broll_enabled: true/false
broll_coverage: 20-50% of video length
transition_type: crossfade/cut/dissolve
stock_providers: pexels, pixabay
quality: HD/4K

Questions

• What percentage of B-roll feels natural? (suggest 25-35%)
• Enable by default or opt-in?
• Automatic or require user approval before rendering?

Resources Needed

• Pexels API (free)
• Pixabay API (free)
• Object detection model
• FFmpeg for video processing
• NLP library for content analysis

---

Priority: High
Estimated Time: 4-6 weeks

[matijani]

Saw issue auto B-roll integration, the semantic search in Phase 2 is where Pexafy adds real value.
Instead of separate Pexels and Pixabay integrations with keyword matching, one Pexafy call covers both
sources with CLIP semantic search, "AI and startups" returns visually relevant clips, not just keyword matches.
Single endpoint, consistent JSON schema across all sources.
Launching mid-April , https://early.pexafy.com if useful.