rlightning.models.model_utils

rlightning.models.model_utils.compute_entropy_from_logits(logits, epsilon=1e-10)

Compute entropy by logits.

Parameters:

• logits – [B, vocab-size, seq-len]

• epsilon – Small constant for numerical stability.

Returns:

[B, seq-len]

Return type:

entropy

rlightning.models.model_utils.compute_logprobs_from_logits(logits, target)