Terminal-based video streaming client
About this project
A CLI tool to stream videos directly in the terminal, using yt-dlp to fetch video data from thousands of supported websites, a demultiplexer made from scratch, and rendering frames directly in the terminal with the Kitty graphics protocol
Key Features
- Video streaming: Videos are played as soon as the tool start, without downloading data in temporary or permanent files
- Demultiplexing MP4 videos (extracting frame and audio data): Video frames and audio data is pulled from a single download stream, to avoid overhead cost of starting separate video and audio download streams and improve performance
- Rendering frames directly in the terminal: Frames are displayed in the terminal with the Kitty graphics protocol, without any external video player
Challenges
The three main challenges tackled in this project were demultiplexing, synchronization across threads and encoding frames according to the Kitty graphics protocol specification. Demultiplexing (extracting audio and video data from a MP4 container) as the video is being downloading required a deep dive into the structure of MP4 containers to parse data and metadata. The output audio and video streams need to be handled in parallel, using multiple threads to build queues of content ready to be displayed. In the case of video frames, extra processing was needed to format the data so it could be sent to the terminal.
Learnings
This project helped me improve my knowledge of multi-threaded programming and memory management in Rust, and taught me the details of video and audio format. It was also an occasion to work with data streaming over the Internet, and learn how to handle the associated uncertainty (incomplete data, irregular streaming speeds...).