NER for document fields (attr_extraction) and OCR correction, mostly names (error_correction).
pip install -r requirements.txt
No data in git. Where to put the CSVs: attr_extraction/data/ and error_correction/data/.
NER:
cd attr_extraction
python run_experiments.py --device cuda --label-scheme bio --models conv conv_crf lstm lstm_crf 2_gram 2_gram_crf 3_gram 3_gram_crf bert
python run_experiments.py --config experiments.yaml
After checkpoints, speed:
python scripts/run_benchmark_suite.py --scheme bio --output results_fin/benchmark_full_bio.csv --skip-missing
Models: conv, conv_crf, lstm, lstm_crf, 2_gram, 2_gram_crf, 3_gram, 3_gram_crf, bert.
Label schemes: bio, bioes, marker.
OCR (from repo root):
python error_correction/run_fio_experiments.py --config error_correction/experiments_fio.yaml
python error_correction/run_fio_experiments.py --models conv lstm --encodings del_ins --device cuda
python -m error_correction.infer --model error_correction/checkpoints_fio/suite_paper/conv_del_ins.pt --text "IVANOF"
CSV columns: system = OCR, markup = gold.
Encodings: raw, del, del_ins, seq2seq (seq2seq is rut5 only).
Checkpoints / outputs go to checkpoints_* and results_*.