Skip to content

[feat] support external custom models - #635

Draft
tiankongdeguiji wants to merge 1 commit into
alibaba:masterfrom
tiankongdeguiji:codex/custom-model-package
Draft

[feat] support external custom models#635
tiankongdeguiji wants to merge 1 commit into
alibaba:masterfrom
tiankongdeguiji:codex/custom-model-package

Conversation

@tiankongdeguiji

Copy link
Copy Markdown
Collaborator

Summary

  • add a generic CustomModel entry backed by google.protobuf.Any
  • load all models from an optional tzrec_custom.models package, with TZREC_CUSTOM_PACKAGE support
  • unpack strongly typed user protobuf configuration and pass it to BaseModel
  • generate optional custom-package protobuf bindings in scripts/gen_proto.sh
  • update custom-model documentation and add focused coverage

Motivation

The previous custom-model workflow required each user model to edit shared model proto files. Those edits regularly conflict with upstream changes during merge or rebase. This change keeps user model code and proto definitions in an independent package while preserving reuse of shared messages such as MLP.

User impact

A custom package can contain multiple models under <package>.models and their configs under <package>.protos. Pipeline configs select a model by full class_path and embed its strongly typed config through Any. The default package is tzrec_custom; other package roots can be selected before startup with TZREC_CUSTOM_PACKAGE.

Test Plan

  • PYTHONPATH=. conda run -n tzrec130 python -m unittest tzrec.utils.load_class_test tzrec.utils.config_util_test tzrec.main_test
  • PYTHONPATH=. conda run -n tzrec130 pre-commit run -a
  • generated a temporary custom proto/package through scripts/gen_proto.sh, then verified package auto-import, Any parsing, and custom model creation in a fresh Python process
  • attempted PYTHONPATH=. conda run -n tzrec130 python scripts/pyre_check.py; Pyre could not start because the existing upstream .pyre_configuration contains invalid JSON (a trailing comma)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant