Challenge Framework

The Air Hockey Challenge is built upon MushroomRL, a Reinforcement Learning Library. The general framework of challenge consists of two key components, i.e., Agent and AirHockeyChallengeWrapper.

You should modify the air_hockey_agent/agent_builder.py to develop [YOUR AGENT] that interacts with the AirHockeyChallengeWrapper.

Example: DummyAgent

We build up a DummyAgent as a first example. The agent will hold static at its initial position.

import numpy as np

from air_hockey_challenge.framework import AgentBase


def build_agent(env_info, **kwargs):
    """
    Function where an Agent that controls the environments should be returned.
    The Agent should inherit from the mushroom_rl Agent base env.

    Args:
        env_info (dict): The environment information
        kwargs (any): Additionally setting from agent_config.yml
    Returns:
         (AgentBase) An instance of the Agent
    """

    return DummyAgent(env_info, **kwargs)


class DummyAgent(AgentBase):
    def __init__(self, env_info, **kwargs):
        super().__init__(env_info, **kwargs)
        self.new_start = True
        self.hold_position = None

    def reset(self):
        self.new_start = True
        self.hold_position = None

    def draw_action(self, observation):
        if self.new_start:
            self.new_start = False
            self.hold_position = self.get_joint_pos(observation)

        velocity = np.zeros_like(self.hold_position)
        action = np.vstack([self.hold_position, velocity])
        return action

AgentBase is a base class that provides several util functions to get specific observation. Further details can be found in Agent.

env_info is a dictionary contains all of the necessary information about the environment. Further details about can be found in env_info.

Two functions are necessary in the ChallengeCore reset and draw_action. reset is called at the beginning of each episode. At each step, draw_action should return an ndarray of shape [2, number_of_joints] consisting of the [Desired Position] and [Desire Velocity].

You also need to implement the function build_agent which returns an object of your agent.

Run

You can check available arguments of run.py by

$ python run.py -h

To start the evaluation locally

$ python run.py -r -e 3dof-hit --n_cores 1 --n_episodes 1

Note that in the remote evaluation, you need to put agent related arguments in air_hockey_agent/agent_config.yml. The evaluation related arguments will be overwritten.