pkl files in reorder.py

[lhmin0614]

Hi, It seems there are several pkl files that contain beam search information in reorder.py
ex) "experiments/wikitext-103-lrsched/wikitext-103-203456-samples/bs_results_height_n_freqknown_{args.freqknown}.pkl"

However, I can't find the code that produce that pkl file.
The only file I can get from beamsearch_rust is a file doesn't match with that form.
(There's no "original single", "candidate single", and "bleu". The File only contains "original" and "results" keys.)

[elazzouzi1080]

Hi @lhmin0614,

I noticed your comment regarding the .pkl files in reorder.py, specifically about how the files from beamsearch_rust only contain "original" and "results" keys, and are missing fields like "original single," "candidate single," and "bleu".

I was wondering if you ever managed to figure out how to generate those .pkl files with the missing fields or if you found any relevant code or approach to achieve this? Any insights or updates would be really helpful!

Thanks in advance!