Becoming a Pokémon trainer takes on a new meaning in this project for generating new pocket monsters using a machine learning algorithm.
Matthew Rayfield, a programmer who makes mobile and web-based toys, created 3,000 new Pokémon using open-source AI models.
Videos by VICE
To do this, he used a combination of his own code, and OpenAI’s “Generative Pretrained Transformer 2,” or GPT-2, a system for generating new text. GPT-2 was rumored to be so powerful—and potentially manipulative—that OpenAI would only tease details about it to the public before releasing the full model. It’s since been released along with simpler models, and its developers are even exploring the model’s image generation capabilities themselves.
But Rayfield didn’t know about the image generation research released in the middle of developing his own project, so he ended up doing the whole thing the long way.
In a video explaining his process, Rayfield said he collected 800 images of pixel-sprites from three different Pokémon games, and wrote a script that would translate each sprite into text, pixel-by-pixel. Each character is assigned a color, with ~ being transparent, and other characters like !, b, a, etc. represent colors.
The result was 100,000 lines worth of sprites. He used that text to re-train GPT-2, which output a random text-based sprite, and then reverse-engineer the line version into a colored-in image. What comes out are garbled little pixel creatures that, if you use your imagination, are pretty close to Pokémon. And let’s be honest, even most canonical Pokémon make no natural sense anyway, so the leap isn’t far.
Some of the best AI-generated sprites, however, do look close.
“I wouldn’t say it looks like a Pokémon, but it looks Pokémon-like,” Rayfield said of the results. “If I saw that I’d say there’s something going on there, they’re trying for something… I’m not sure they did it, but it’s certainly not just random pixels. It’s got a substance to it, a body there.”
Rayfield asked illustrator Rachel Briggs to draw some of the best-looking sprites, and they’re believable as something Nintendo might cook up. I’d catch these.
You can browse the full set of results here, or make your own using the Google Colab notebooks linked in Rayfield’s Github page for the project.
