Desktop QA Assistant With Llama3 in Rust

ChatGPT is a marvel of research and engineering. But it’s much much more than just that. ChatGPT has had a far more significant impact than just being an exceptional feat of technology. It has, probably for the first time, made AI accessible to everyone, not just AI/ML practitioners but everyone. LLMs are a household name now. You sit in a coffee-shop, and you hear ChatGPT can do it. 3 years ago it would most surely be a bunch of techies talking about their next project....

In the spirit of AI for everyone we started building quick desktop application to interface with the very capable LLaMA3 8B model by Meta AI in our previous post. The series: Part I: The Setup Part II: The model (this post) Part III: Putting it all together Goals We completed the basic setup in our previous post. In this part of the series, we’ll load the model and run our first AI inference....

In this series of posts, we have been building a desktop application to interface with LLaMA3. We did our setup in Part I and built our app’s backend in Part II. In part III and last part of this series we are going to build a simple client interface with SvelteKit and run our desktop application. Recap If you have landed strait into this post, I’d highly recommend you take a quick peek into the last 2 parts (links below) to help you get up to speed with the progress....