rlightning.engine.async_rl_engine

Asynchronous reinforcement learning engine module.

This module implements the AsyncRLEngine for asynchronous training where rollout, training, and weight updates run in separate threads.

class rlightning.engine.async_rl_engine.AsyncCoordinator

Bases: object

A coordinator to manage asynchronous tasks.

is_running() → bool

Check if the coordinator is still running.

notify_dataset_ready() → None

Notify that dataset is ready for training.

notify_train_step_done() → None

Notify that the training step is done.

notify_weight_update_step_done() → None

Notify that the weight update is done.

start() → None

Start the coordinator.

stop() → None

Stop the coordinator.

wait_for_dataset_ready() → None

Wait until dataset is ready for training.

wait_for_sync_weights() → None

Wait for signal to sync weights.

wait_for_update_dataset() → None

Wait for signal to update dataset.

wait_for_weights_updated() → None

Wait until weights are updated for evaluation.

class rlightning.engine.async_rl_engine.AsyncRLEngine(config: MainConfig, env_group: EnvGroup, policy_group: PolicyGroup, buffer: DataBuffer)

Bases: BaseEngine

Asynchronous reinforcement learning engine.

This engine implements asynchronous training where: - Rollout thread: Collects experience from environments. - Training thread: Updates policy using collected experience. - Weight update thread: Broadcasts updated weights to eval policies.

evaluate() → None

Evaluate is not implemented for async engine.

rollout() → None

Perform rollout to collect experience from environments.

Runs the rollout loop, collecting experience by stepping through environments and storing transitions in the buffer. Runs until the done_flag is set.

run() → None

Launch asynchronous training threads.

Starts three worker threads (rollout, train, weight update) and waits for them to complete. Prints timing summary when finished.

train() → None

Perform training on collected experience.

Runs the training loop for the configured number of epochs, sampling from the buffer and updating the policy. Sets the new_weights_ready flag after each training step and done_flag when training completes.

update_dataset() → None

Update dataset from buffer to train policy.

warm_up()

init and dummy run for constructing RL dataflow Performs a dummy rollout and training iteration to ensure proper dataflow construction.