NeuralTalk and Walk

This video is captioned in the upper left completely by machine. It was generated in real time by a neural network programmed to describe images. Stanford Ph.D. student Andrej Karpathy developed the NeuralTalk system and Kyle McDonald adapted it to describe this laptop of Amsterdam on a laptop.

Yes, it changes pretty fast, so you might want to keep your finger ready to pause. The original program generated four captions per second! McDonald reduced that to only one caption per second so we have a chance of following it. The program isn’t completely accurate, but it’s close enough to give you an “uncanny valley” feeling.

If you want a real laugh, a different neural network was used for an earlier experiment in captioning the intro to Star Trek: The Next Generation. That program definitely needs a bigger vocabulary. (via Metafilter)