This project provides examples on Knowledge Distillation (rationale-based method - incorporates step-by-step reasoning) and Parameter-efficient Finetuning (using QLoRA).
References for KD are thanks to Anaumghori and Google Research. Reference for QLoRA is thanks to Schaeffer23.