Binary reward calculation overrides continuous task_score

[gjx0422]

Hi authors, thanks for the great work and releasing the benchmark!

I noticed an inconsistency in the reward calculation on the webshop. While task_score is computed continuously (as described in the paper), the final reward is forcefully binarized before being multiplied by the scale factor.

This turns a fine-grained feedback signal into a sparse binary reward (0 or 1), which seems contradictory to the continuous evaluation metric and might negatively impact RL training.

Could you clarify if this binarization is intentional, or a bug in the published code?

Thanks for your time!

[Melmaphother]

Hi authors, thanks for the great work and releasing the benchmark!

I noticed an inconsistency in the reward calculation on the webshop. While task_score is computed continuously (as described in the paper), the final reward is forcefully binarized before being multiplied by the scale factor.

This turns a fine-grained feedback signal into a sparse binary reward (0 or 1), which seems contradictory to the continuous evaluation metric and might negatively impact RL training.

Could you clarify if this binarization is intentional, or a bug in the published code?

Thanks for your time!

Sorry for the delayed response. In practice, either the task score or the success signal can be used as the reward. The current open-source implementation uses the success signal, scaled by a factor of 10, which is less prone to reward hacking. We also experimented with using the task score directly and found that the difference was not significant. Therefore, both choices are reasonable in practice.