Unlocking Book Knowledge with GPT-3

Table of Contents

1. Introduction
2. The Problem of Searching Books
3. The Solution: Using GPT-3 and Universal Sentence Encoder
4. Implementing the Semantic Search
5. Generating Answers with GPT-3
6. Improving the Search Process
7. Using the Web UI for Book GPT
8. Conclusion

Introduction

Recently, while working on a project, I encountered a problem that I couldn't seem to solve. I realized that the answer might be in a book I had read a few months back. Eagerly flipping through its pages, scanning every chapter, I couldn't find what I was looking for. It was like trying to find a drop of water in the ocean. That's when it hit me - if we have Google to search the web, why don't we have something similar for searching through books?

The Problem of Searching Books

The idea is simple: if I ask a question to an AI, it should scan the entire book and generate the answer to my question. To make this more reliable, it should also include page numbers from where it is referring to the information. However, one of the biggest challenges standing in our way is that we cannot give an entire book to GPT-3 because the maximum number of tokens it can accept is 4000, which is roughly 3000 words. A typical 100-page book has 30 thousand words, which is 10 times more than the GPT-3's limit.

The Solution: Using GPT-3 and Universal Sentence Encoder

To overcome this challenge, we can break down the book into smaller chunks. For example, we can divide a 100-page book into 3600 chunks of approximately 100 words each. Next, we can use Universal Sentence Encoder, a deep learning model that can take any text and output a high-dimensional dense vector (embedding) that captures semantic meaning.

Implementing the Semantic Search

To implement the semantic search, we start with loading the book and extracting its text. Then, we break it down into smaller chunks, generate embeddings for each chunk using the Universal Sentence Encoder, and store these embeddings. When a user asks a question, we generate embeddings for the question and use a simple K-nearest neighbor algorithm to find the top-n chunks that are most similar to the user's question.

Generating Answers with GPT-3

Now that we have narrowed down the search to the most relevant chunks, we can generate answers using GPT-3. We create a prompt based on the user's question and pass it to the GPT-3 model. The model will generate an answer tailored to our needs. We can set the maximum number of tokens to 512, which should be sufficient for most answers.

Improving the Search Process

While the current approach is effective, there is still room for improvement. Some chunks may not relate to the user's question, and we may want a straightforward answer rather than a reference to a chunk. Prompt engineering can help address these issues and improve the accuracy of the search results.

Using the Web UI for Book GPT

To make the search process more user-friendly, a web UI for Book GPT has been created and hosted on the Hugging Face platform. Users can input their questions and receive answers from a variety of PDFs, not just the "Dive into Deep Learning" book. The system can handle any PDF and provide accurate answers.

Conclusion

With the combination of GPT-3 and the Universal Sentence Encoder, we have created a powerful tool for searching books and generating answers to specific questions. This approach breaks down large texts into smaller chunks, uses semantic embeddings to find relevant chunks, and generates tailored answers using GPT-3. While there is room for improvement, this method shows promise in simplifying the search process and retrieving precise information from books.