rlightning.policy.simple_ppo_policy

class rlightning.policy.simple_ppo_policy.SimplePPOPolicy(config: PolicyConfig, role_type: PolicyRole)

Bases: BasePolicy

A simple PPO Policy with a CNN encoder for testing purposes. it takes the actor-critic architecture.

construct_network(env_meta, *args, **kwargs)

Construct the neural network architecture.

Parameters:

• env_meta – Environment metadata for network configuration.

• *args – Variable positional arguments.

• **kwargs – Variable keyword arguments.

evaluate(obs, action)

get_action(obs: MockModule('torch.Tensor'))

get_action_value(obs)

get_trainable_parameters()

Return a dict of module state dicts.

get_value(obs: MockModule('torch.Tensor'))

Compute state value

Parameters:

obs (torch.Tensor) – Batched observation array

Returns:

Batched state value

Return type:

torch.Tensor

load_state_dict(state_dict, strict=True, assign=False)

Load trainable parameters from state_dict.

postprocess(data)

rollout_step(env_ret: EnvRet) → PolicyResponse

save_weights(save_dir: str, epoch: int)

setup_optimizer(optim_cfg)

Set up the optimizer for training.

Parameters:

optim_cfg – Optimizer configuration.

train()

Run a training step for the policy.

update_dataset(data)

Update the dataset in the policy by getting a batch from the buffer.

Submodules

• rlightning.policy.simple_ppo_policy.ppo_policy