← Back to Projects

AI Camera: Raspberry Pi Image-to-Video System

AI Camera: A Raspberry Pi Image-to-Video Generation System

A Raspberry Pi in a red 3D-printed shell: take a photo, or describe a dream, and it turns that into a short film with real camera language.

View code

Role
Course Project · Innovative Experiments in Electronic Science III
Period
Sep. 2025 – Jun. 2026
Institution
SUSTech　Supervised by Dr. Junmin Jiang

Why this matters

Generative models can already turn a still into video, but that happens in a browser tab — the camera itself still only records. This project turns the camera into a creative terminal: it has to understand why you took the shot, turn the story and mood you add into shot language, and produce a finished clip. The hard part was never any single API. It was chaining a camera, a microphone, a language model and a video service — each of which fails in its own way — into one flow an ordinary person can finish.

What I did

  • Built live preview and capture on Picamera2, with a seven-page PySide6 desktop UI: home, capture, story, prompt, video, history and settings.
  • Designed three creative modes: shoot reality (photo + story), recover a dream (text or voice), and black-box finale (reconstruction from narrative fragments).
  • Used OpenAI Chat Completions as an "AI director" to turn photo and story into a cinematic treatment, then compressed and translated it into short English Runway prompts via a dedicated optimizer.
  • Added real-time speech-to-text over the OpenAI Realtime WebSocket, with a volume gate, auto-submit on silence, a maximum segment length and retained recordings — fixing unstable transcript segmentation.
  • Drove the Runway API for generation across Gen-4 / Gen-4.5 / Veo / Seedance model configurations, running jobs on a background thread so a multi-minute wait never froze the UI.
  • Unified photos, text, audio, prompts and videos into session directories (metadata.json plus photos/story/prompts/videos/audio) so every piece stays traceable.
  • Modelled and 3D-printed a three-part enclosure: a front window for the camera module, side cutouts for power and Ethernet, designed around ribbon-cable bend radius and serviceability.

Course project for Innovative Experiments in Electronic Science III. Below: the assembled device, and a screen recording of the interface in use.

The three-part enclosure assembled. A front window exposes the Raspberry Pi Camera module, the ribbon cable folds back inside to the board, and the sides leave power and Ethernet accessible. The window position and cable bend radius were measured off the real board rather than modelled first and forced to fit.
The three-part enclosure assembled. A front window exposes the Raspberry Pi Camera module, the ribbon cable folds back inside to the board, and the sides leave power and Ethernet accessible. The window position and cable bend radius were measured off the real board rather than modelled first and forced to fit.
Another angle.
Another angle.
界面演示

Outcome

  • All three creative modes verified end to end, with test work saved under the sessions directory.
  • Representative runs: 3 photos plus a story produced 3 video versions; a spoken dream produced a storm-liner short; an escape-room narrative produced a sci-fi finale (Gen-4.5, 5–8 s, 1280×720).
  • The enclosure was printed and assembled, taking the project from a program that runs to a device that can be shown.

Technologies

  • Raspberry Pi
  • Picamera2
  • PySide6 / Qt
  • OpenAI API
  • Runway API
  • Realtime Speech over WebSocket
  • Multithreading
  • 3D Modelling & Printing