rlightning.utils.builders

Builder utilities for constructing RL components.

This module provides factory functions for building the main components of a reinforcement learning system: data buffers, environment groups, policy groups, and training engines.

rlightning.utils.builders.build_data_buffer(buffer_cls: str, buffer_cfg: BufferConfig, obs_preprocessor: Preprocessor | None = <function default_obs_preprocessor>, reward_preprocessor: Preprocessor | None = <function default_reward_preprocessor>, env_ret_preprocess_fn: Callable | None = <function default_env_ret_preprocess_fn>, policy_resp_preprocess_fn: Callable | None = <function default_policy_resp_preprocess_fn>, preprocess_fn: Callable | None = <function default_preprocess_fn>, postprocess_fn: Callable | None = <function default_postprocess_fn>) → DataBuffer

Build a data buffer instance.

Parameters:

• buffer_cls – Name of the buffer class to instantiate.

• buffer_cfg – Buffer configuration object.

• obs_preprocessor – Observation preprocessor function.

• reward_preprocessor – Reward preprocessor function.

• env_ret_preprocess_fn – Preprocessing function for environment returns.

• policy_resp_preprocess_fn – Preprocessing function for policy responses.

• preprocess_fn – General preprocessing function for each timestep.

• postprocess_fn – Post-processing function for completed episodes.

Returns:

Configured DataBuffer instance.

rlightning.utils.builders.build_engine(config: MainConfig, env_group: EnvGroup, policy_group: PolicyGroup, buffer: DataBuffer) → BaseEngine

Build engine instance

Parameters:

• config – main config

• env_group – env group

• policy_group – policy group

• buffer – data buffer

Returns:

engine instance

Return type:

BaseEngine

rlightning.utils.builders.build_env_group(env_cfgs: List[EnvConfig] | EnvConfig, preprocess_fn: Callable | List[Callable] | None = <function default_env_preprocess_fn>) → EnvGroup

Build an environment group from configuration.

Parameters:

• env_cfgs (Union[List[EnvConfig], EnvConfig]) – Env config(s) to build from.

• preprocess_fn (Optional[Union[Callable, List[Callable]]]) – Preprocess function(s) for observations, either a single callable or one per env config.

Returns:

EnvGroup instance managing the configured environments.

rlightning.utils.builders.build_policy_group(policy_cls: str, policy_cfg: PolicyConfig, cluster_cfg: ClusterConfig, backend: str = 'nccl', is_colocated: bool = False) → PolicyGroup

Build a policy group with train and eval workers.

Creates and configures policy workers for training and evaluation, supporting distributed execution with Ray.

Parameters:

• policy_cls – Name of the policy class to instantiate.

• policy_cfg – Policy configuration object.

• cluster_cfg – Cluster configuration object.

• backend – Communication backend for distributed training.

• is_colocated – If True, only initialize training policies for comm groups.

Returns:

PolicyGroup containing configured train and eval workers.

rlightning.utils.builders.create_distributed_policy_class(policy_cls: type, role_type: PolicyRole) → type

Create a distributed policy class by mixing policy_cls with DistributedMixin.

rlightning.utils.builders.define_env_instance_cfgs(env_cfg: EnvConfig, num_workers: int = 1) → List[EnvConfig]

Factory function to define duplicated env instance configs.

The number of replicas is determined by num_workers.

Parameters:

• env_cfg – Base environment configuration.

• num_workers – Number of environment instances to create.

Returns:

List of environment configurations, one per worker.