chore: migrate gift-eval module - #369
Conversation
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
There was a problem hiding this comment.
Pull request overview
Migrates the in-repo timecopilot.gift_eval implementation to the standalone PyPI package timecopilot-gift-eval, updating dependencies and all internal references so users can access the functionality via timecopilot_gift_eval / the timecopilot[gift-eval] extra.
Changes:
- Add
timecopilot-gift-eval>=0.3.0as an optional extra (gift-eval) and include it in dev/docs dependencies. - Remove the in-repo
timecopilot/gift_evalimplementation files and update tests/experiments/docs to import fromtimecopilot_gift_eval. - Update lockfile(s) and documentation to reflect the new distribution/import paths.
Reviewed changes
Copilot reviewed 15 out of 18 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
uv.lock |
Adds the gift-eval extra and locks timecopilot-gift-eval==0.3.0 into dev/docs and optional dependency metadata. |
pyproject.toml |
Introduces gift-eval optional dependency and installs timecopilot-gift-eval for dev/docs environments. |
timecopilot/gift_eval/utils.py |
Removed (module migrated out of repo). |
timecopilot/gift_eval/gluonts_predictor.py |
Removed (module migrated out of repo). |
timecopilot/gift_eval/eval.py |
Removed (module migrated out of repo). |
timecopilot/gift_eval/data.py |
Removed (module migrated out of repo). |
tests/gift_eval/test_gift_eval.py |
Switches imports to timecopilot_gift_eval and cleans up test output. |
tests/gift_eval/test_evaluation.py |
Switches imports to timecopilot_gift_eval and adjusts which columns are compared to the reference results. |
tests/gift_eval/conftest.py |
Uses GIFTEval from timecopilot_gift_eval for fixtures (data download + reference CSV caching). |
mkdocs.yml |
Preloads timecopilot_gift_eval for API docs generation. |
experiments/gift-eval/src/run_timecopilot.py |
Updates imports to the external timecopilot_gift_eval package. |
experiments/gift-eval/src/run_modal.py |
Updates dataset-term import to timecopilot_gift_eval.utils. |
experiments/gift-eval/src/download_results.py |
Updates dataset-term import to timecopilot_gift_eval.utils. |
experiments/gift-eval/pyproject.toml |
Renames the experiments project and depends on timecopilot-gift-eval>=0.3.0. |
docs/examples/gift-eval.ipynb |
Updates installation instructions and imports/links to the new timecopilot_gift_eval path. |
docs/api/gift-eval/gift-eval.md |
Documents the new standalone package and updates mkdocstrings targets to timecopilot_gift_eval.*. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 18 changed files in this pull request and generated 1 comment.
Suppressed comments (2)
docs/examples/gift-eval.ipynb:791
- Spelling: “mantainers” should be “maintainers”.
"The TimeCopilot's [GIFT-Eval integration](https://timecopilot.dev/api/gift-eval/gift-eval/#timecopilot_gift_eval.eval.GIFTEval) was designed considering reproducibility as one of its main features. The library can replicate the official results provided by the mantainers of the benchmark for the [`SeasonalNaive`](https://huggingface.co/spaces/Salesforce/GIFT-Eval/tree/main/results/seasonal_naive) method. The following code replicates the Seasonal Naive performance for the datasets evaluated in this notebook. The reproducibility of the results for the rest of the datasets are tested continuously in the [library's repo](https://github.com/TimeCopilot/timecopilot/blob/main/tests/gift_eval/test_evaluation.py)."
docs/api/gift-eval/gift-eval.md:3
- This page introduces the new
timecopilot_gift_evalimport path, but it doesn’t explicitly mention that earlier docs/examples usedtimecopilot.gift_eval. Adding a brief migration note here would reduce confusion for existing users following older snippets (e.g., changelogs still referencetimecopilot.gift_eval).
GIFT-Eval integration lives in the standalone [`timecopilot-gift-eval`](https://pypi.org/project/timecopilot-gift-eval/) package on PyPI. Install it with TimeCopilot via:
| gift-eval = [ | ||
| "timecopilot-gift-eval>=0.3.0", | ||
| ] |
There was a problem hiding this comment.
should be fixed in the latest commit
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 17 out of 20 changed files in this pull request and generated no new comments.
Suppressed comments (3)
docs/examples/gift-eval.ipynb:56
- The notebook installs
timecopilot[gift-eval]>=0.0.22, but thegift-evalextra is introduced in this PR (and the repo currently reports version 0.0.30). This command will fail on PyPI until the first release that includes thegift-evalextra; update the minimum version accordingly (or drop the version pin).
"%pip install \"timecopilot[gift-eval]>=0.0.22\""
docs/examples/gift-eval.ipynb:791
- Typo in user-facing docs text: “mantainers” should be “maintainers”.
"The TimeCopilot's [GIFT-Eval integration](https://timecopilot.dev/api/gift-eval/gift-eval/#timecopilot_gift_eval.eval.GIFTEval) was designed considering reproducibility as one of its main features. The library can replicate the official results provided by the mantainers of the benchmark for the [`SeasonalNaive`](https://huggingface.co/spaces/Salesforce/GIFT-Eval/tree/main/results/seasonal_naive) method. The following code replicates the Seasonal Naive performance for the datasets evaluated in this notebook. The reproducibility of the results for the rest of the datasets are tested continuously in the [library's repo](https://github.com/TimeCopilot/timecopilot/blob/main/tests/gift_eval/test_evaluation.py)."
docs/changelogs/index.md:5
- Docs nav now lists
v0.0.31, butpyproject.tomlstill declaresversion = "0.0.30". Unless a0.0.31release/version bump is included as part of this change, the docs site will advertise a version that doesn't match the package metadata. Consider bumping the project version (and any related release metadata) or holding off on adding thev0.0.31changelog/nav entry until the release PR.
- [v0.0.31](v0.0.31.md)
this pr migrates the gift-eval module that lived in the repo. now users can directly access it using
timecopilot-gift-eval