• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

Google's Deep Mind has learned how to play Montezuma's Revenge

The AI system was able to solve the complex game in just four tries

https://www.youtube.com/watch?v=0yI2wJ6F8r0

Google's Deep Mind has learned how to play yet another game - this time because it had been 'incentivised' to want to win.

"Intrinsic rewards" meant the AI obtained "significantly improved exploration in a number of hard games
, including the infamously difficult Montezuma's Revenge", wrote Google researchers in a paper.

Intrinsic motivation (IM) algorithms typically use signals to make the AI more 'curious' and are inspired by classic, human-based psychological ideas.

Montezuma's Revenge was a 1984 platform game for the Atari 2600 in which a character navigates a series of complex rooms in an underground Aztec temple.

The model, which had inbuilt rewards, explored 15 rooms out of a potential 24 – the old model, which was not incentivised, explored only two.

Deep Mind has already been trained to play Atari games, learning how to play 49 games by itself, and earlier this year beat Go champion of the world 4-1.

But games like Montezuma's Revenge are more challenging for the AI, said the researchers, because they require forward planning rather than just simple reaction.

The researchers now plan to develop a model to tackle even harder games like Starcraft, and to compete against humans in gaming contests.
http://www.wired.co.uk/article/google-ai-montezuma-revenge

Screen_Shot_2016-06-09_at_11.44.21_AM.0.png

http://www.theverge.com/2016/6/9/11893002/google-ai-deepmind-atari-montezumas-revenge
 

TechJunk

Member
I thought Montezuma's Revenge was when you go to Mexico and get the shits.

You're right, it is. The game title is a joke in that sense. I remember the first time hearing an adult talk about Montezuma's revenge when I was a kid, wondering why they were talking about the PC game I loved to play...
 

Calabi

Member
It's fascinating how games could help the birth of AI. Its a simple environment seemingly designed for it. Almost a recursive loop.
 

jiggles

Banned
Despite being trained to pick up the key and climb back to the spawn point, it immediately decided that it is more efficient to kill the human than cater to its needs to get it safely back to the door.
 
However, that human intervention changes the environment the robot is operating in, and can lead to two problems: the robot could begin to learn that the scientists want it to stay indoors, meaning it might ignore the more important task. Or, potentially worse, it could still favor the harmful action, and just view the interruption as an obstacle it should try to avoid. In resisting human intervention, the AI could even learn to disable the red button. It's a disturbing thought.
Oh dear...
 
Wow it's even more frightening now !

I've never understood why this intimidates people. Even if an AI learned to adjust it's own code to break the established ruleset, can you imagine what would happen? BSOD. Changing code is never simple.

This is also incredible. I'm very interested in what exactly entails this reward system. I need to read this through properly when I get the time.
 
I've never understood why this intimidates people. Even if an AI learned to adjust it's own code to break the established ruleset, can you imagine what would happen? BSOD. Changing code is never simple.

This is also incredible. I'm very interested in what exactly entails this reward system. I need to read this through properly when I get the time.

I need to read through as well. How does one "reward" an AI?
 
I've never understood why this intimidates people. Even if an AI learned to adjust it's own code to break the established ruleset, can you imagine what would happen? BSOD. Changing code is never simple.

A powerful tool is always intimidating. Of course in theory you can make sure to have all the mechanisms in place for this IA to not go out of her boundaries. There is still the risk of human mistakes or even more probable human malicious use, hacks etc.

Aztechnology said:
I need to read through as well. How does one "reward" an AI?
With numbers. You tell your IA to (or you write your IA in a way such that it wants to) maximize some numbers in your game for instance the score. What is difficult for the IA is to trade-off between 'Hey I am doing cool stuffs that give me high scores I should keep doing this' and 'Well, what if there is something else completely different to what I am doing now and that leads to much higher rewards'. This is called the exploration-exploitation trade-off and is one of the hardest part of such problems.
 

clintar

Member
Not impressive until it gets to the point it would rather stop playing the games and instead just read and post about games on Neogaf.
 
Top Bottom