Series: Desktop App for Document QA with RAG

Retrieval Augmented Generation is an approach in natural language processing that combines the power of large language models with traditional knowledge retrieval. This technique enhances the capabilities of AI systems by allowing them to access and incorporate relevant information from databases or documents during the generation process. RAG addresses some limitations of traditional language models by providing up-to-date, authoritative information and reducing hallucinations. In this tutorial, we’ll explore how to build a local RAG system - our own Document QA....

In this series we building our own local Retrieval Augmented Generation (RAG) desktop application for Document QA. We got our Embedding flow working along with some smart text-splitting techniques in our previous post. In this post we are building a toy (but working) Vector Database to store, search and retrieve data based on the document embeddings. Series Snapshot Part 1: we implement Embedding generation from text data. We used Stella_en_1.5B_v5 and it’s Candle Transformers’ implementation as the embedding model and used the crate text-splitter to split our text into meaningful chunks....

In our previous posts of this series, we set up embedding generation and storage using the Stella_en_1.5B_v5 model and created a mini Vector Store inspired by Spotify’s ANNOY. Now, we’ll look at extracting text from PDFs and using LLaMA3 for text generation. Series Snapshot Part 1: we implement Embedding generation from text data. We used Stella_en_1.5B_v5 and it’s Candle Transformers’ implementation as the embedding model and used the crate text-splitter to split our text into meaningful chunks....

In this series we have been working on building our own Local Document QA application. So far we have implemented stella_en_1.5B_v5 as our Embedding model, wrote our own mini vector store and in our previous post we integrated a model for document layout analysis and text extraction from .pdf files. Then we set the groundwork for our text generation with a LLaMA 3.x model. In this post we glue them together and build our query-to-answer flow....

In this blog series on building a Desktop Document QA app, we’ve tackled several key challenges: crafting document embeddings, building a Spotify’s ANNOY-inspired vector store, and developing a document Layout analysis and extraction pipeline powered by a Detection2 model. With indexing and basic QA flows now complete, we’re ready for the next phase of our journey - evaluating and implementing some cutting-edge techniques for a better RAG. TL; DR Github...