"Retrieval Augmented Generation Chatbot uses a knowledge base to answer questions about information the model has no clue about. The knowledge base usually is an organization's internal documentation. The model has never seen the documentation during the training process. The documentation is given to the model as a context inside the prompt, and therefore, the model is able to answer questions correctly."
"The model's prompt length is limited, so documentation has to go through the selection process before being appended as a context to the prompt. The quality of generated responses highly depends on the selection process. In this article, I ran experiments on the Tapir documentation. I created the test dataset of the query (question) and documentation paragraph pairs. Each pair relates to the different parts of Tapir's documentation. I evaluated the model based on the top-3 accuracy."
Retrieval-augmented systems use external knowledge bases, typically internal documentation, provided to the model at query time because the model lacks that information from training. Prompt length limits require a selection process to choose which document passages become context, and selection quality strongly affects response quality. The retriever architecture uses a bi-encoder to convert queries into feature vectors and compare them to document vectors in a vector database; a cross-encoder can be applied for reranking. Experiments on Tapir documentation employed a dataset of query–paragraph pairs drawn from different documentation sections and measured top-3 retrieval accuracy. An example covered decoding failures and error handling.
Read at Medium
Unable to calculate read time
Collection
[
|
...
]