求一个RedPajama-Data-1T-Sample中的book_sample.jsonl数据集处理的脚本，做微调任务 #56

Open

opened

on Jun 24, 2024

类似与tools/alpaca_tokenizer.py的脚本，处理book_sample.jsonl的数据

to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

Labels

No labels

No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests