Series: Voice Assistant Desktop App with LLaMA3 and Whisper in Rust

In my previous series Desktop QA Assistant With Llama3 in Rust we built a desktop app capable of interfacing with LLaMA3. In this post, we’ll extend the modality of the app to accept audio instructions instead of just text. In this part of the series, we’ll deal with the setup, loading the models and get our text inference up using the rust ML framework called Candle. The series: Part I: The Setup (this post) Part II: Audio instructions and the App Who is this for?...

In this series we are on a mission to build our very own desktop native app that can interface with LLM LLaMA3-8B over text and audio instructions. In our pervious installment of the series we did our setup with Tauri 2.0 Beta and loaded our LLaMA3 and Whisper models using Huggingface Candle framework, then we ran our text generation pipeline. In this post we complete this journey with voice instructions and get our app to respond to our audio instructions....