Show HN: I turned my face rec system into a video codec Before the pandemic, my tiny startup was doing quite well selling Edge AI systems, based on our own lightweight AI inference engine, with object detection and face recognition for smart city and smart retail & food service applications. When the real world shut down, there was suddenly nothing to monitor on streets and in restaurants, so I set out to try and evolve our real time face recognition system into a video codec for high quality face-to-face online interactions, as I was not satisfied with the quality of Zoom and friends. I got it to work, and the first release for IOS was just approved on Apple's app store, link: https://ift.tt/jbEU0s5 The way it works is that you create a meeting URL, which you can share out-of-band, for instance via slack or text message. You can also share as a QR code which the app can scan to join a call. You then place your device on a surface in front of you so that the front camera can see you, and it will recognize you face and assign you to your own session, which is broadcast to the meeting channel. If more than one person is in view, both of you will be broadcast but with separate session ids, like if you were on separate cameras. Other meeting participants will show up on your screen and you can start talking. It is optimized for eye contact, meaning that the eyes will actually make it through to the other side as more than just dark pixel clouds, so thinks should feel a bit more personal than the standard Zoom/Teams/or Google Meet call. Because it uses face rec, you can ONLY show your face, and if you disappear from view your audio will stop after a while, to avoid situations like when you need to go the the restroom but forget to mute. This also solves dick-pics etc. The CODEC is not based on H26[45], but is pure AI that runs on the GPU. There is a neural network that compresses the video in real time, and another one decompressing on the receiving end. Finding a tight network architecture that would do this in real time with acceptable quality was a major part of the effort. There are several quality settings possible, but right now it is set fairly high and for 20FPS maxes out around 700kbit/s, though typically uses about half. I've demonstrated good results down to around 200kbit/s, so in theory it should work over satellite links or even Bluetooth. The protocol is UDP with no congestion control but with (Wirehair) FEC to protect against mild packet loss, future versions will detect packet loss and adapt to available bandwidth. The audio just uses OPUS and may click a little bit, I blame AudioEngine or the fact that the last time I wrote audio code was for the game I published for the Amiga in 1994. If you don't have a friend around or multiple devices to play with, there is an "echo test" server mode that allows you to be in a meeting with yourself. Traffic will be peer-to-peer if possible, but otherwise you will be relaying through my tiny Raspberry PI server, so YMMV. I plan to try to switch to something like fly.io soon to improve scalability. There is also a MacOS version coming very soon, and the underlying AI engine also runs on Windows & Linux. Android support is planned. Please take a look and let me know what you think. https://ift.tt/7RDSFxy May 26, 2022 at 02:45PM
Women Pioneers at Muni: Adeline Svendsen and Muni’s First Newsletter By Jeremy Menzies To close out Women’s History Month, here’s a look back at one woman whose work to bring Muni staff together in the late 1940s created a legacy that lives on to this day. Adeline “Addy” Svendsen was founding editor of Muni’s first internal newsletter, “ Trolley Topics .” Adeline Svendsen sits at her desk in the Geneva Carhouse office building in this 1949 shot. Trolley Topics was a new venture when it started in February 1946. As Svendsen wrote in the first issue it was created, “to bring a little fun, a little news, and a lot of good will to all our fellow employees in the Railway.” Just two years prior in 1944, Muni merged with the Market Street Railway Company, expanding the small municipal operation into the largest transit provider in the city with hundreds of employees, vehicles of every shape and size, and dozens of facilities scattered across town. The newsletter was meant to help unite ...
Comments