Skip to main content

Show HN: Garnet – Distributed Python on Kubernetes https://ift.tt/3a0ibzw

Show HN: Garnet – Distributed Python on Kubernetes Garnet (https://garnet.ai) is a framework to run distributed and parallel Python on Kubernetes. We make it easy to scale from laptop to cluster, without requiring any devops work. Garnet plugs into any existing Python environment (notebook, IDE, CI/CD pipelines, custom apps) using a lightweight client library, and enables users to scale popular data/ml libraries (including pandas, NumPy, scikit-learn, XGBoost, etc.) and custom code to a remote cluster for execution. Under the hood, we natively wire up Dask with Kubernetes for scheduling, which brings advantages such as dynamic resource allocation, ephemeral clusters and autoscaling, while abstracting the devops complexity for developers and data scientists. Our vision is to build the orchestration layer for distributed computing in Python. We’re starting off with Dask, but have goals of supporting other frameworks such as Ray and Modin in the future. Some of the cool use cases we’ve seen from our users are: Bursting to the cloud from local dev environments for data and memory intensive computations Standing up managed Dask and Kubernetes clusters programmatically in 3-4 lines of Python, without any devops knowledge (Docker, k8s etc.) Parallelize existing codebases in pandas, NumPy, scikit-learn, XGBoost etc. with minimal code changes, without opting for a more complex system such as Spark Currently, users can download our client library and run Dask workloads on our fully managed cluster (see docs). We’re working on adding the capability to run Garnet on your own cloud (Kubernetes) in an upcoming release. We’re excited to hear your feedback, and what you’d like to see. Please drop your email on our website so we can get in touch directly. January 23, 2021 at 06:49AM

Comments

Popular posts from this blog

Show HN: Tape It, iOS recording app for musicians https://ift.tt/3udBTSi

Show HN: Tape It, iOS recording app for musicians Hello HN, Over the last 15 months, two friends and I developed the music recording app we felt we wanted based on our own needs as musicians. It's called Tape It [1] and has just recently hit the Apple App Store [2]. We put a lot of effort into a good UX to help musicians really focus on playing their instrument instead of pretending to be a recording engineer. The app records in stereo on newer iPhones (although that's a premium feature; the free version only records in standard mono audio quality). I would be really grateful for advice from this community on how to best approach marketing. We had a great TechCrunch article covering our launch [3], and we posted it on various music websites. Turns out advertising on Google or Apple Search is a dark art, though. We have some good ideas for developing a good social media presence, but they will take time. Please hit us with feedback, opinions and advice that you think a young ind...

Show HN: Moderator,lightweight peer4peer anon forum https://ift.tt/3fZSDGl

Show HN: Moderator,lightweight peer4peer anon forum hello all! here's a link to my little pinteresting like forum that stores no data on the server and uses IPFS for image storage. The design aesthetic is that everything would in 64kb of memory so we're going for a collapse-proof low bandwidth experience. this makes moderator really fast. https://moderator.rocks is the web preview, a flutter client is in the works at https://ift.tt/32wqdRb take a look, post something fun, ask questions. I'm also on twitter @moderatorium in case interested. Have fun! January 26, 2022 at 12:23AM

Show HN: Comment on live websites just like you comment on Google Docs/Figma https://ift.tt/GRhrjX0

Show HN: Comment on live websites just like you comment on Google Docs/Figma I'd love your feedback on this new JS plugin we launched. With this, you can comment on live websites just like you comment on Google Docs or Figma. You can use is to get Copy or UI feedback right on the website you are building. Feedback can be provided in rich formats like audio and video. You can get started by installing a JS tag in the footer of the website. You can then turn the review mode on or off on demand by adding “?review=true” to the URL. Demo video (43s): https://www.youtube.com/watch?v=cdnfBEw8TfI Demo video: https://www.youtube.com/watch?v=h6vxzXJuh8o https://ift.tt/ocLpdEu October 26, 2022 at 02:18AM