Skip to main content

Show HN: Example POC to run the Dolly V2 12B model in 8bit mode https://ift.tt/e4rAYI9

Show HN: Example POC to run the Dolly V2 12B model in 8bit mode I banged this out today because nobody seems to have figured this out yet (and I wanted to play with this model). I was inspired by the fact that Simon Willison was struggling to get this running[0], so I wanted to take a crack at it! This is by no means "clean", but it is a baseline for others to hack from. Feel free to send over PRs and I'll land them! (I'd love it somebody used this to glue this into this webui[1].) 0: https://ift.tt/LHd6KCA 1: https://ift.tt/vZc7rqx https://ift.tt/KRUDoLh April 15, 2023 at 04:46AM
Labels:

Comments

Post a Comment

Popular posts from this blog

Show HN: KittenDNS – easy to setup, rule engine, LetsEncrypt compatible https://ift.tt/JEbrc0i

Show HN: KittenDNS – easy to setup, rule engine, LetsEncrypt compatible My goal was to create a DNS server as simple as the LDAP server I've been contributing to (glauth) and here it is: https://ift.tt/i6S4IHQ A few things to know about it: - It is easy to configure using a Toml file - I am using it to bail my coredns/etcd instances when under attack - (yes, it also works as a service location server) - It comes with a simple, plain English, rule engine that doesn't do much at this point - It can be used with LetsEncrypt. I use it to retrieve certificates for my home servers. - RFC: all the nice ones :) Anyway, feedback is welcome. May 16, 2022 at 03:34AM
Post a Comment
Read more https//reachsingh.blogspot.com»