Vector databases are commonly used to store vector embeddings for tasks like similarity search to build recommendation and question-answering systems. Milvus is one of the open-source databases that stores embeddings in the form of vector data, it is well suited because it has indexing features like Approximate Nearest Neighbours (ANN) enabling fast and accurate results.
In this article, we'll demonstrate the steps of how to use a HuggingFace dataset, create embeddings from the dataset, and divide the dataset into two halves (testing and training). You'll also learn how to store all the created embeddings into the deployed Milvus database by creating a collection, then perform a search operation by giving a question prompt and generating the most similar answers.
[
Collection
]
[
|
...
]