Show HN: The Future of Data APIs Hello everyone, I'm Matteo, and, over the last few months, I have been working with my co-founder and other folks from Goldman Sachs, Netflix, Palantir, and DBS Bank to simplify building data APIs. I have personally faced this problem myself multiple times, but, the inspiration to create a company out of it really came from this Netflix article (https://ift.tt/pYQyHs9). You know the story: you have tons of data locked in your data platform and RDBMS and suddenly, a PM asks to integrate this data with your customer-facing app. Obviously, all in real-time. And the pain begins! You have to set up infrastructure to move and process the data in real-time (Kafka, Spark, Flink), provision a solid caching/serving layer, build APIs on top and, only at the end of all this, you can start integrating data with your mobile or web app! As if all this is not enough, because you are now serving data to customers, you have to put in place all the monitoring and recovery tools, just in case something goes wrong. There must be an easier way !!!!! That is what drove us to build Dozer. Dozer is a simple open-source Data APIs backend that allows you to source data in real-time from databases, data warehouses, files, etc., process it using SQL, store all the results in a caching layer, and automatically provide gRPC and REST APIs. Everything with just a bunch of SQL and YAML files. In Dozer everything happens in real-time: we subscribe to CDC sources (i.e. Postgres CDC, Snowflake table streams, etc.), process all events using our Reactive SQL engine, and store the results in the cache. The advantage is that data in the serving layer is always pre-aggregated, and fresh, which helps us to guarantee constant low latency. We are at a very early stage, but Dozer can already be downloaded from our GitHub repo (https://ift.tt/Pz2wEmC). We have taken the decision to build it entirely in Rust, which gives us the ridiculous performance and the beauty of a self-contained binary. We are now working on several features like cloud deployment, blue/green deployment of caches, data actions (aka real-time triggers in Typescript/Python), a nice UI, and many others. Please try it out and let us know your feedback. We have set up a samples-repository (https://ift.tt/G2TrLJx) for testing it out and a Discord channel (https://ift.tt/WbvSs2B) in case you need help or would like to contribute ideas! Thanks Matteo April 6, 2023 at 12:16PM
Women Pioneers at Muni: Adeline Svendsen and Muni’s First Newsletter By Jeremy Menzies To close out Women’s History Month, here’s a look back at one woman whose work to bring Muni staff together in the late 1940s created a legacy that lives on to this day. Adeline “Addy” Svendsen was founding editor of Muni’s first internal newsletter, “ Trolley Topics .” Adeline Svendsen sits at her desk in the Geneva Carhouse office building in this 1949 shot. Trolley Topics was a new venture when it started in February 1946. As Svendsen wrote in the first issue it was created, “to bring a little fun, a little news, and a lot of good will to all our fellow employees in the Railway.” Just two years prior in 1944, Muni merged with the Market Street Railway Company, expanding the small municipal operation into the largest transit provider in the city with hundreds of employees, vehicles of every shape and size, and dozens of facilities scattered across town. The newsletter was meant to help unite ...
Comments