Unleash Your Imagination with NLP Story Generator

Table of Contents

1. Introduction
2. Background
3. Data Collection
4. Data Pre-processing
5. Word Embeddings
6. NLP Models
   1. N-Gram Models
   2. Recurrent Neural Network (RNN) Models
   3. Transformer Models
7. Experiments and Training
8. Evaluation Metrics
9. Human Evaluation
10. Results and Analysis
11. Conclusion
12. Future Directions
13. GitHub Repository

Introduction

Language models that can generate coherent and creative stories have always been an interesting problem in the field of Natural Language Processing (NLP). The goal of this project was to explore the effectiveness of various deep learning models in generating high-quality stories. By developing a language model that could not only generate coherent stories but also exhibit creativity in its generations, we aimed to bridge the gap between human creativity and technology. This article explores the journey of developing an NLP story generator and discusses the challenges, opportunities, and results of the project.

Background

The project focused on the intersection of human creativity and technology. One of the applications of this NLP story generator is to provide artists and creators with a starting point or new directions for a story, helping them overcome writer's block. From a student perspective, it also allowed us to examine the challenges and opportunities in the domain of text generation.

Data Collection

To train our models, we collected a dataset containing writing prompts and their corresponding stories. This dataset was sourced from Reddit's "WritingPrompts" subreddit, which provided a rich source of diverse text in terms of genres such as science fiction, fantasy, romance, mystery, and more. The dataset consisted of approximately 300K writing prompts and stories, ensuring a diverse and extensive training set.

Data Pre-processing

Before training the models, we performed basic text cleaning operations such as removing punctuations, converting all data to lowercase, and removing any non-alphanumeric characters. For tokenization, we used libraries like Spacey and Hugging Face Transformers, depending on the specific model being trained.

Word Embeddings

To understand the semantic relationships between words in the dataset, we created word embeddings using word2vec models from the GenSim library. These embeddings allowed us to capture the contextual meaning of words and explore their relationships. We then created 2D visualizations using t-SNE plots to visualize the similarity between words.

NLP Models

We explored different NLP models for this project, including n-gram models, recurrent neural network (RNN) models, and transformer models.

N-Gram Models

N-gram models generate repetitive tokens and do not capture the context of the input prompt. As a result, they are not sufficient to generate coherent passages.

Recurrent Neural Network (RNN) Models

RNN models, especially those using LSTM (Long Short-Term Memory) cells, are effective in modeling long-term sequential dependencies. This is a key requirement for generating coherent and engaging stories. In this project, we used two instances of AWD-LSTM, a pre-trained model. We trained it from scratch and then fine-tuned it for our specific task.

Transformer Models

Transformer models, such as GPT (Generative Pre-trained Transformer), were also explored. These models use self-attention mechanisms to capture global dependencies and have achieved state-of-the-art performance in various NLP tasks. We trained GPT from scratch and fine-tuned GPT-2 and GPT-3 models.

Experiments and Training

We trained multiple models for our project, keeping the batch size, text size, training set, and test set consistent to maintain consistency and fairness in evaluation. We experimented with different architectures and hyperparameters to optimize the performance of the models.

Evaluation Metrics

In evaluating the quality of the generated stories, we considered several parameters such as coherence, grammatical structure, and novelty. While perplexity scores provide insights into how well a model is trained, metrics like BLEU (BiLingual Evaluation Understudy) and ROUGE (Recall-Oriented Understudy for Gisting Evaluation) are not sufficient to evaluate the quality of a story. Therefore, human evaluation was deemed necessary.

Human Evaluation

To gather human evaluations, we created a Jupyter notebook that allowed human evaluators to provide prompts and rate the generated stories from different models. We collected ratings ranging from one (poor) to five (excellent) for each model. The average ratings were calculated and analyzed to determine the best-performing model.

Results and Analysis

The results of our evaluation showed that the GPT-2 fine-tuned model named "ed" had the highest average rating among all models. This indicates that the GPT-2 model, after fine-tuning, performed exceptionally well in generating high-quality stories. However, it is important to note that training large language models like GPT-2 is expensive and time-consuming.

Conclusion

In this project, we successfully developed an NLP story generator that bridged the gap between human creativity and technology. The GPT-2 fine-tuned model emerged as the best-performing model in generating high-quality and coherent stories. Although the aim is not to replace human creativity, the NLP story generator can augment and inspire creative processes. The project also highlighted the challenges and opportunities in the domain of text generation.

Future Directions

There are several avenues for future research in this area. Further exploration can be conducted to improve the training process of language models, making it more efficient and cost-effective. Additionally, exploring techniques to enhance the creativity and diversity of generated stories would be valuable. The NLP story generator can be expanded to other domains, such as generating poems or scripts, to explore the broader possibilities of AI-assisted content creation.

GitHub Repository

To access the code and implementation details of this project, please visit our GitHub repository. Feel free to explore the code and reach out to us if you have any questions.

Highlights

• Developed an NLP story generator that generates coherent and creative stories.
• Explored the effectiveness of various deep learning models in text generation.
• Collected a diverse dataset of writing prompts and stories for training.
• Pre-processed the data by cleaning and tokenizing it.
• Created word embeddings to capture semantic relationships between words.
• Explored n-gram models, RNN models, and transformer models for text generation.
• Conducted experiments and training to optimize model performance.
• Evaluated models using both automated metrics and human evaluations.
• Identified the best-performing model through human evaluation.
• Highlighted the challenges and opportunities in the field of text generation.

FAQ

Q: How long did it take to train the language models? A: Training large language models like GPT-2 can be time-consuming, often taking several days or even weeks depending on the available computing resources.

Q: Can this NLP story generator replace human creativity? A: The aim of this project is not to replace human creativity but to augment and inspire creative processes. The generated stories serve as a starting point or new directions for artists and creators, helping them overcome writer's block.

Q: What metrics were used for evaluating the quality of the stories? A: We considered parameters such as coherence, grammatical structure, and novelty. However, automated metrics like perplexity, BLEU, and ROUGE were not sufficient. Human evaluation played a crucial role in determining the quality of the stories.

Q: Are there any plans to expand the NLP story generator to other domains? A: Yes, there are potential future directions to explore. The NLP story generator can be extended to other domains such as generating poems or scripts, providing AI-assisted content creation in various creative fields.

Q: Where can I access the code and implementation details of this project? A: You can find the code and implementation details in the GitHub repository associated with this project. Please visit the repository link provided above for more information.