Replace stale Hugging Face spaCy adapter - #2201
Open
SharonHart wants to merge 2 commits into
Open
Conversation
Remove spacy-huggingface-pipelines so the analyzer can use secure Transformers 5 releases. Add an internal scored-span component with spaCy plugin discovery, preserve failure behavior, and cover it with focused regression tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2e86159d-80a9-4926-9378-c5dabe07a39f
Contributor
Coverage report (presidio-anonymizer)
Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||||||||||||||||||||
Contributor
Coverage report (presidio-structured)
Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||
Contributor
Coverage report (presidio-image-redactor)
Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||||||||||||||
Contributor
Coverage report (presidio-cli)
Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||
Contributor
There was a problem hiding this comment.
Pull request overview
This PR replaces the external spacy-huggingface-pipelines adapter with an internal spaCy token-classification component to unblock upgrading Transformers (addressing security advisories) while preserving existing Transformers NLP engine behavior (Doc.spans with scores, batching/fallback behavior, and spaCy serialization/plugin discovery).
Changes:
- Introduce an internal spaCy factory/component (
presidio_hf_token_pipe) backed bytransformers.pipeline(task="token-classification")and wire it intoTransformersNlpEngine. - Raise the Transformers minimum to
>=5.5.0(pinning5.6.2in the lockfile) and removespacy-huggingface-pipelines; keepglinerwithin<5.7. - Add targeted tests and update documentation references from the old adapter to the Transformers pipeline docs.
Reviewed changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| presidio-analyzer/uv.lock | Removes spacy-huggingface-pipelines, updates/pins Transformers and related transitive dependencies/markers. |
| presidio-analyzer/pyproject.toml | Adds spacy_factories entry point for the internal pipe; updates optional dependency constraints for Transformers/GLiNER. |
| presidio-analyzer/presidio_analyzer/nlp_engine/huggingface_token_pipe.py | New internal spaCy component implementing token-classification spans + scores with batch fallback and safe logging. |
| presidio-analyzer/presidio_analyzer/nlp_engine/transformers_nlp_engine.py | Switches engine wiring from the external adapter to the internal pipe factory/config. |
| presidio-analyzer/presidio_analyzer/predefined_recognizers/nlp_engine_recognizers/transformers_recognizer.py | Updates recognizer docs to reflect the internal engine/score flow rather than the removed adapter. |
| presidio-analyzer/tests/test_transformers_nlp_engine.py | Adds coverage ensuring engine load registers the internal pipe and passes the expected model name. |
| presidio-analyzer/tests/test_huggingface_token_pipe.py | New unit tests for device selection, factory behavior, batch fallback, PII-safe logging, validation, and serialization no-ops. |
| docs/analyzer/nlp_engines/transformers.md | Updates docs link from the removed adapter to official Transformers token-classification pipeline documentation. |
Comment on lines
+199
to
+201
| spans.append(span) | ||
| spans.attrs["scores"].append(score) | ||
| previous_end = end |
Contributor
Coverage report (presidio-analyzer)
Click to see where and how coverage changed
The report is truncated to 25 files out of 79. To see the full report, please visit the workflow summary page. This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
spacy-huggingface-pipelineswith a decoupled internal spaCy token-classification componentDoc.spans, batch fallback behavior, CPU/CUDA selection, and spaCy serialization/plugin discovery<5.7compatibility rangeThis removes the stale adapter's
transformers<5constraint, which blocked fixes for the six Dependabot alerts associated with GHSA-69w3-r845-3855, GHSA-29pf-2h5f-8g72, and GHSA-fgcw-684q-jj6r.Validation
3006 passed, 12 skippedin the analyzer all-extras test suiteuv lock --check, and wheel build passedspacy_factoriesentry pointhf-internal-testing/tiny-bert-for-token-classificationat revisionf89ef50d84f2959688279d4b2c09faf823da2069produced identical labels, offsets, span text, and scores under Transformers 4.57.6 and 5.6.2