HarpIA Survey is a tool that implements the execution and evaluation of large language models (LLMs) by human evaluators using Moodle.
HarpIA Survey is comprised of two Moodle plugins:
- HarpIA Ajax, a local plugin that implements an API which allows a Moodle user to send requests to a language model as part of an activity;
- HarpIA Interaction, a plugin for the Database activity that provides an interactive field, using the first plugin to communicate with a language model.
In order to install the plugins on a Moodle instance, follow the documentation in the repositories linked above.
Alternatively, the Docker images in this repository contain a full Moodle installation:
Following Moodle's license, HarpIA Survey is provided freely as open source software, under version 3 of the GNU General Public License.
The copyright of the described software is jointly-owned by the University of São Paulo and the IBM Corporation, and the University of São Paulo acknowledges that the IBM Corporation has a nonexclusive, irrevocable, worldwide, royalty-free and fully-paid-up Commercial license to use, test, execute, perform, display, evaluate, distribute, prepare derivatives of, and commercialize the software.