StoryTelling

KO | EN

gitlite — search

#python #java #python3 #arduino #golang #machine-learning #rust #html #flask #javascript #seismology #nodejs

★ 13 Open GitHub ↗

A neural network based StoryTeller that outputs a short story from an input image

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# StoryTelling

StoryTelling is a recurrent neural network that generates short stories about images. This is a recreation of an original project by Jamie "Ryan" Kiros (@ https://github.com/ryankiros).

Original code repository can be found here : https://github.com/ryankiros/neural-storyteller

This repository contains code for generating stories with your own images, as well as instructions for training new models.

### Updates to the original code:
1) Updating the code to run on Python3+
2) Original code uses COCO to generate captions from images which are then fed into the model to generate stories. We have moved to Microsoft's Vision API instead to get captions & labels. Hence you will see that COCO related code commented out in this repository. But we strongly recommend to learn more about COCO and use the captions generated by it to feed into this project's RNN model. COCO is Microsoft's open source image caption genration project.
3) Training a new model on children's bedtime stories. About 45k sentences of bedtime stories were used to train the model. Original project was trained on 14 million passages from romance novels. Trained model can be found in storyteller folder. To run this project on romance genre, refer the original project to download the necessary model files.
   
### Things to Know:
1) The project has lot of other dependencies like Theano, Tensorflow, Keras etc. To run this project, you will need all these dependencies resolved.
2) For Theano the default float type is set to 'float64'. This may create conflicts while working with Tensorflow & Keras. As per the general suggestion for deep learning models go, you must convert the default type for Theano to float32 to avoid such cross compatibility issues. 
    Use ----->   theano.config.floatX = 'float32' 
3) The training of the model on children's stories was done on a Google Cloud Instance (set up with a deep learning instance image) with 1 GPU.
4) To collect the training data, stories were downloaded from multiple open source websites, or even scraped if publicly available. Collecting data from multiple sources meant lot of data cleaning, and bringing sentences to same encoding that should match the original project.
5) Make sure your data is clean and is free from empty lines. If your training data has empty lines, weird & nerve-wracking errors will be thrown while training your own model - which are hard to debug.

This project was done as a submission for our final project for Business Data Science course  (in MS in IT & Management) @ McCombs School of Business - University of Texas as Austin.
It was a team effort with other 2 members - Mouhamed Ndoye (@ https://github.com/mhmdndoye) & Anuja Srivastava (@ https://github.com/anuja1011)

More details on how this project was implemented etc is written as a blog post available here :

StoryTelling

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README