Skip to main content

​
Overview

The Agent class provides a foundation for implementing various types of agents, such as reinforcement learning agents or autonomous agents.

​
Class Definition

class Agent:
    def __init__(self, name: Optional[str] = None)

​
Parameters

​
name
Optional[str]
The name of the agent. If None, a random name is generated.

​
Properties

​
name

@property
def name(self) -> str
Get the name of the agent.
​
return
str
The agent’s name.

​
Core Methods

​
act

def act(self, observation: Any) -> Any
Choose an action based on the current observation. This method should be overridden by all subclasses.
​
observation
Any
required
The current observation of the environment.
​
return
Any
The action to take.

​
learn

def learn(self, experience: Any) -> None
Learn from experience. This method should be overridden by all subclasses that support learning.
​
experience
Any
required
The experience to learn from.

​
reset

def reset(self) -> None
Reset the agent’s state. This is typically called at the beginning of a new episode.

​
Persistence

​
save

def save(self, path: str) -> None
Save the agent’s state to a file.
​
path
str
required
The path to save the agent to.

​
load

def load(self, path: str) -> None
Load the agent’s state from a file.
​
path
str
required
The path to load the agent from.

​
Example Usage

import neurenix as nx
from neurenix.agent import Agent

# Define a custom agent
class MyAgent(Agent):
    def __init__(self, name=None):
        super().__init__(name)
        self.policy = None  # Your policy network
        
    def act(self, observation):
        # Implement action selection logic
        if self.policy is None:
            return random.choice([0, 1, 2, 3])  # Random action
        
        # Use policy network
        with nx.Tensor.no_grad():
            action = self.policy(observation)
        return action
    
    def learn(self, experience):
        # Implement learning logic
        state, action, reward, next_state, done = experience
        
        # Update policy based on experience
        loss = self.compute_loss(state, action, reward, next_state, done)
        loss.backward()
        self.optimizer.step()
    
    def reset(self):
        super().reset()
        # Reset any episode-specific state
        self._episode_rewards = []

# Create an agent
agent = MyAgent(name="MyCustomAgent")

# Use the agent in an environment
observation = env.reset()

for step in range(1000):
    # Agent selects an action
    action = agent.act(observation)
    
    # Environment steps
    next_observation, reward, done, info = env.step(action)
    
    # Agent learns from experience
    experience = (observation, action, reward, next_observation, done)
    agent.learn(experience)
    
    observation = next_observation
    
    if done:
        agent.reset()
        observation = env.reset()

# Save the trained agent
agent.save("my_agent.pth")

# Load the agent later
new_agent = MyAgent()
new_agent.load("my_agent.pth")

​
Built-in Agent Types

Neurenix provides several built-in agent implementations:

DQN Agent

Deep Q-Network agent for discrete action spaces

A2C Agent

Advantage Actor-Critic agent

PPO Agent

Proximal Policy Optimization agent

SAC Agent

Soft Actor-Critic agent for continuous control
See the RL Algorithms documentation for details on these implementations.