Interactive Reinforcement Learning

We can use tensorflow.js to create complex agents directly in the browser. For example, we can create a simple neural network that we can use to predict actions given observations using the following code. This code uses the example of the FrozenLake environment.

let inputSize = 16;   // the frozenlake 4x4 env has 16 different observations
let outputSize = 4;   // the action can be up, down, left or right
let hiddenSize = 50;  // choose a number of "hidden" units for the network
let agent = tf.Sequential();
agent.add(tf.layers.dense({inputShape: [inputSize], units: hiddenSize,
                                  useBias: true, activation: "relu"}));
agent.add(tf.layers.dense({units: hiddenSize, useBias: true, activation: "relu"}));
agent.add(tf.layers.dense({units: outputSize}));

Each environment has the same API. When opening an environment, the environment object is made available globally under the env variable. To inspect variables and their contents, simply log them to the console.

console.log(env);

Initially you should call env.reset(), to start the environment. This method returns an initial observation.

let obs = env.reset();

Once you've made an observation, your agent should select an action. A list of possible actions is shown in the documentation for each environment. To apply the action to the environment, call env.step(action)and pass your action as an argument to the step method. This method returns a tuple containing the next observation, the reward from the step, a boolean showing if the environment has reached the end, and an object possibly containing some debugging information about the game.

let action = env.action_space.sample();   // Sample a random action 
let [obs, rew, done, info] = env.step(action);