Fix BPE training across consecutive line endings

[Alphaxiaoteng]

Summary

Fixes #1534.

train_from_files previously fed each physical line to the BPE trainer independently. Consecutive newline-only lines therefore never appeared in one training sequence, so byte-level BPE could not learn merges such as ĊĊ. Keep ordinary lines unchanged while coalescing consecutive control-only lines in a private training iterator. The public LinesWithEnding iterator keeps its existing behavior.

Tests

• cargo test training_lines --lib
• cargo test --test training bpe_training_from_files_keeps_consecutive_line_endings_together -- --exact
• cargo test --lib
• make lint
• git diff --check

AI assistance was used for repository analysis and implementation; the change and tests were reviewed manually.