In this episode, Lex Fridman welcomes two of the most important figures in the multimedia world: Jean-Baptiste Kempf, president of VideoLAN and a central figure behind VLC and FFmpeg, and Kieran Kunhya, a codec engineer, FFmpeg contributor and the person behind FFmpeg’s famous account on X. Fridman describes FFmpeg as the invisible backbone behind YouTube, Netflix, Chrome, VLC, Discord and almost every platform that handles video or audio on the internet, with an estimate that more than 90% of video processing workflows rely on it. VLC has been downloaded more than six billion times, with no advertising and no tracking.
The two guests explain what happens when we press play. First, the program retrieves a data stream from an address, then the demuxer separates the audio, video and subtitle tracks, and each track is subsequently decoded. Kempf stresses that video compression is on the order of 100x to as much as 200x, and that everything is guided by human perception: we work in YUV rather than RGB, mimicking how the eye distinguishes brightness from colour, and use frequency transforms and prediction to remove redundant information.
Kempf explains the difference between a container and a codec: MP4 is not the same as H.264, although the industry has made things difficult with names such as MPEG-4 Part 10. VLC, he points out, does not trust the file extension — it treats it only as a hint and examines the contents. This philosophy grew out of the program’s roots in streaming over UDP networks, where packets are lost and files are often corrupted. Kunhya adds that an .MP4 file can actually be anything, and this tolerance of broken files made VLC popular during the piracy era.
The open-source community is described as fundamentally meritocratic. Kempf says that all that matters is code quality: “You could be a dog, I don’t care — I don’t care where you come from, I look at your code.” He mentions that teenagers have written thousands of lines of assembly in FFmpeg, including 16-year-old Ruikai Peng, who found and fixed security issues without creating drama. Kunhya notes that blunt communication, such as that of Linus Torvalds, is usually “on topic” and is often made worse by the fact that many contributors are not native English speakers.
Kempf tells the story of VLC, which began at École Centrale Paris in the 1990s with the Network 2000 project: students streamed MPEG-2 satellite video over a local network, a decade before YouTube. VideoLAN Client became VLC, and in 2003 Kempf took action to move it beyond the university and make it sustainable. In 2005, the project nearly died, with only two active developers, but he decided to keep it going, believing the technology was significant and useful.
Kempf became famous for repeatedly turning down tens of millions of dollars to keep VLC free of advertising. He explains that the offers came from shady advertising companies that wanted to bundle toolbars or spyware, or change the user’s search engine. “You have to make money ethically,” he says, adding that he wanted to be able to sleep soundly at night. The last offer was, as he describes it, “obscene”, with the argument that he could use the money to build something new and open source — but he refused.
Much of the discussion concerns the incident involving Google’s security engineers, who used artificial intelligence to generate vulnerability reports for FFmpeg. Kunhya explains that Google told the media how good its AI was before the issues had been fixed, applying a 90-day deadline that does not suit volunteer development, and that the vulnerability involved an obscure game codec from the 1990s. He also mentions the XZ affair and the incident in which Microsoft requested free, urgent support from volunteers, offering a one-off payment of a few thousand dollars instead of a contract. Donations rose significantly after the debate, although they do not cover even one full-time salary.
The two engineers explain why assembly language remains irreplaceable. Kunhya says SIMD allows a single instruction to process sixteen numbers at once, delivering speed improvements of 10x to 50x over C — with one example showing a 62x speedup. VideoLAN’s AV1 decoder, dav1d, has 240,000 lines of handwritten assembly and uses its own calling conventions to save cycles. Kempf names Henrik Gramner and Martin Storsjö as assembly wizards, with the latter writing code on his phone at the playground. Kunhya has created assembly lessons (asm-lessons) to pass on this art, which is at risk of being lost.
The story of H.264 and x264 stands out as an example of innovation by amateurs. Kunhya explains that hobbyists, many from the anime community, challenged the industry’s obsession with mathematical metrics such as PSNR and introduced psychovisual distortion and adaptive quantisation, making images actually look better. The ParkJoy sample from Swedish television became the standard test that separated the men from the boys. Kempf adds that x264, which originated at the same school as VLC, remains a benchmark for every new codec.
Kempf explains the multimedia patent war: H.265/HEVC became almost impossible to license because of multiple patent pools, leading companies such as Google and Netflix to create the Alliance for Open Media and the royalty-free AV1 codec. Kunhya notes that in the MPEG world, patents are not even discussed during standardisation. Kempf says that software patents are rejected in France, and that if he had to pay all the fees for VLC, they would amount to more than €200 per user.
Reverse engineering is described as a form of archaeology. Kempf recounts how Kostya Shishkov, a Ukrainian “borderline genius”, reverse-engineered GoToMeeting’s proprietary codec in two months, treating the world as a “binary specification”. Kunhya describes the process: taking memory dumps from virtual machines, stepping through execution in a debugger, and seeking samples from the community. Bit exactness is crucial: every implementation of a decoder must produce exactly the same bits.
Burnout among open-source maintainers is a topic approached candidly. Kempf says he received death threats with powder in an envelope when he decided to end support for PowerPC, and that he now maintains libraries whose previous maintainers abandoned them because of psychological pressure. He says that libXML is no longer maintained by anyone, and that the time zone database is maintained by one person. Daniel Stenberg, the maintainer of curl, is fighting “AI slop” — fake reports generated by models that burden volunteers.
The archiving community is presented as a model of ethical conduct. Led by Dave Rice from the City University of New York, it funded the development of FFV1, a lossless codec designed for fast software encoding and resilience to bit loss. Kunhya cites the example of the New Domesday Book in the United Kingdom, which was archived on BBC microcomputers and required reverse engineering after 20 years — a warning about what could happen in a thousand years. C, he says, will be like Latin: it will still be learned and remain usable, making FFmpeg a Rosetta Stone for the future.
Looking ahead, the two engineers see multimedia expanding into new senses: point clouds, volumetric video, haptic data, even smells. Kempf says that anything timed that represents something we can sense will find its way into FFmpeg — “if it’s brainwaves, it will be brainwaves”. He also discusses Kyber, his new startup for ultra-low latency in the remote control of robots, drones and vehicles, targeting four milliseconds “glass-to-glass”, having already achieved seven. The project is open source with dual licensing.
The episode closes with gratitude and a philosophical note. Kempf says regrets are “a tax on your mind”: you learn from mistakes but do not regret them. Fridman thanks the two guests for the software that has brought joy to millions of people, and the episode ends with the words of Linus Torvalds: “Most good programmers do programming not because they expect to get paid or get recognition by the public, but because it is fun to program.”
Comments