Experience the Magic of AI-Generated Old English!

Experience the Magic of AI-Generated Old English!

Table of Contents:

1. Introduction
2. Background 2.1 Old English Poetry Data Set 2.2 Why GPT Neo? 2.3 Training Time 2.4 Pre-processing the Data 2.5 Model Improvements
3. The Journey of NLP 3.1 Finding the Data Set 3.2 Product Specification 3.3 Tech Stack 3.4 Hosting the Model
4. Challenges and Future Improvements 4.1 Bugs and Random Responses 4.2 Adding a Jokes Data Set 4.3 Joining Data Sets
5. Demo 5.1 Front End 5.2 Back End 5.3 Web Page Demonstration
6. Conclusion

Introduction Old English Generator: Exploring Old English Poetry with GPT Neo

Background Old English poetry has always been a fascinating subject, and we wanted to delve deeper into this rich literary tradition. This article focuses on our journey of using GPT Neo, a powerful natural language processing model, to generate text in Old English. We will discuss the Old English poetry data set, why we chose GPT Neo, our training process, pre-processing challenges, and model improvements.

2.1 Old English Poetry Data Set Our data set consists of numerous old poems from various writers of the old age. We gathered these poems from poetryfoundation.com, which covers subjects like love, nature, mythology, and folklore. These poems are specifically from two periods, namely the Renaissance and modern.

2.2 Why GPT Neo? We selected GPT Neo for several reasons. Firstly, it excels in natural language understanding and generation tests, making it ideal for our project. Secondly, GPT Neo is a pre-trained model, leveraging a large corpus of text from the internet to learn grammar, syntax, semantics, and a broad understanding of language. Lastly, its large-scale language model with a substantial number of parameters enables it to capture complex patterns and dependencies in data.

2.3 Training Time Initially, we trained the model on the CPU, which took approximately 10 minutes. However, after switching to the GPU and expanding the data set, the training time decreased to around 5 minutes. We optimized our training settings to achieve this short training time.

2.4 Pre-processing the Data Pre-processing the data posed a significant challenge due to the excessive length of the text, which exceeded the model's token limits. To address this, we reduced the events of the data and noticed a decrease in pad tokens being generated, positively impacting the model's performance. We also applied better pre-processing techniques to minimize pad tokens and improve results.

2.5 Model Improvements To achieve better results, we modified various parameters during the training process. These modifications, as shown in the screenshot below, enhanced the model's performance and reduced issues such as repeated conjunctions at the beginning of generated text.

3. The Journey of NLP Our journey of exploring Old English poetry with NLP involved various key steps. In this section, we will discuss finding the data set, creating a product specification, determining the tech stack, and hosting the model.

3.1 Finding the Data Set Finding a suitable data set was our initial challenge. Eventually, we discovered a data set comprising numerous poems written in Old English, hosted on howyouface.org. This data set served as the foundation for our project.

3.2 Product Specification To streamline our efforts, we created a product specification outlining the basic requirements for our minimum viable product (MVP). This provided a clear outline of the system's functionalities and guided our development process.

3.3 Tech Stack In terms of technology, we leveraged various tools and frameworks to build our Old English Generator. The tech stack included Python as the primary programming language, Flask for web application development, JavaScript for interactive elements, and HTML/CSS for the user interface.

3.4 Hosting the Model To make our Old English Generator accessible to users, we hosted the GPT Neo model on Hugging Face, a platform that allows easy deployment and usage of transformer models. By utilizing Flask, we created a web application that integrated the model and presented the generated output to users.

4. Challenges and Future Improvements While our Old English Generator has shown promising results, there are still areas for improvement. This section discusses the challenges we encountered and potential future enhancements.

4.1 Bugs and Random Responses Although our model performs well overall, it occasionally generates random responses and repeats certain phrases. These occurrences are more likely when the user inputs incoherent text. Addressing this issue to ensure consistent and coherent outputs remains a priority for future development.

4.2 Adding a Jokes Data Set Initially, we aimed to include a jokes data set to generate funny responses. However, finding a usable and clean data set proved challenging. Incorporating a suitable jokes data set would enhance the variety and amusement value of the model's output.

4.3 Joining Data Sets We also recognized potential challenges in combining data from different data sets simultaneously. As our Old English Generator evolves, finding effective ways to seamlessly integrate multiple data sets will be essential for expanding the model's capabilities.

5. Demo To showcase the Old English Generator, we developed a web page that allows users to interact with the model. This section provides an overview of the front end, back end, and demonstrates the capabilities of the generated output.

5.1 Front End The front end of our web page was built using a bootstrap template, which we customized to include only the necessary elements. It features a title area, a short description, and a UI section with input and output text areas, as well as a "Generate" button.

5.2 Back End The back end functionality of our Old English Generator is powered by Python. The main Python file serves as the brain of the project, importing necessary libraries, initializing the Hugging Face inference API, and setting up the Flask app. The generator function reads the user's input, applies relevant adjustments to parameters, and generates the output text.

5.3 Web Page Demonstration During the demonstration, users are prompted to enter an Old English text in the input text area. After clicking the "Generate" button, the output text, generated by the GPT Neo model, appears in the output text area.

6. Conclusion In conclusion, our journey of exploring Old English poetry with GPT Neo has been an exciting and challenging endeavor. Through the utilization of a suitable data set, proper pre-processing techniques, and the power of GPT Neo, we successfully developed an Old English Generator. While there is room for improvement, the generated outputs demonstrate the potential of harnessing NLP models for exploring and understanding historical literary works.