rlightning.engine.sync_rl_engine

Synchronous reinforcement learning engine module.

This module implements the SyncRLEngine for synchronous training where rollout and training happen sequentially in a single thread.

class rlightning.engine.sync_rl_engine.SyncRLEngine(config: MainConfig, env_group: EnvGroup = None, policy_group: PolicyGroup | None = None, buffer: DataBuffer | None = None)

Bases: BaseEngine

Synchronous reinforcement learning engine.

This engine implements synchronous training where rollout and training happen sequentially. Supports both on-policy and off-policy algorithms.

evaluate(obj_set: str, prefix: str = '') → None

Perform evaluation to collect experience from environments.

Uses eval_env_list inside env_group if available, otherwise falls back to the training env_list.

Parameters:

• obj_set – Object set identifier for environment reset options.

• prefix – Prefix for logging metrics.

rollout(obj_set: str, prefix: str = '') → None

Perform rollout to collect experience from environments.

Parameters:

• obj_set – Object set identifier for environment reset options.

• prefix – Prefix for logging metrics.

run() → None

Run the main training loop.

Executes the training loop for the configured number of epochs, performing rollout, training, and periodic evaluation.

train() → None

Perform training on collected experience.

Samples data from the buffer, updates the dataset, and trains the policy. Logs training info metrics.

Raises:

ValueError – If buffer size is smaller than batch size when sampling without replacement.

update_dataset() → None

Update the dataset in the policy group from the buffer.

warm_up()

init and dummy run for constructing RL dataflow