feat: production-level reliability, observability, and OpenAI compatibility improvements - #90

Merged

chottokun merged 21 commits into

mainfrom

feature/production-level-improvements

Sep 12, 2026

Commits on Sep 12, 2026

style: fix ruff lint errors across scratch and scripts
chottokun
committed
56cbe4d
ci: enforce uv audit, gitleaks detection, and repository-wide linting
chottokun
committed
6402926
feat: add readiness check endpoint (/ready) and tests
chottokun
committed
19fd7db
docs: document /ready endpoint, CI security checks, and code health fixes
chottokun
committed
33ed8a5
feat: implement Prometheus metrics (/metrics) and middleware instrumentation
chottokun
committed
553d2c3
perf: batch multimodal image and text tensor inference in VisualizedBGEEmbeddingModel
chottokun
committed
2d0439a
refactor: make TEI proxy completely asynchronous with httpx.AsyncClient
chottokun
committed
2f856a3
docs: document metrics, multimodal batching, and async TEI proxy
chottokun
committed
2b9a63b
ci: update Docker and Compose container healthchecks to use /healthz probe
chottokun
committed
34338ba
feat: add structured JSON logging with request ID tracking
chottokun
committed
8405171
docs: document structured JSON logging with request ID tracking
chottokun
committed
ac28e89
fix(security): mitigate DNS Rebinding and TOCTOU in image downloads via IP pinning
chottokun
committed
3bb4973
docs: document DNS Rebinding and TOCTOU mitigation in changelog and log
chottokun
committed
de6e353
feat: add GET /v1/models, payload limit middleware, and POST /v1/models/unload
chottokun
committed
d276380
docs: document /v1/models, payload limit, and model unload in changelog and log
chottokun
committed
0fb8171
feat: support dimensions, encoding_format base64, and rate limiting middleware
chottokun
committed
8ea1af2
feat: implement graceful shutdown request drain and TORCH_DTYPE mixed precision inference
chottokun
committed
64b44ba
feat: add token consumption and batch size metrics, and model preloading on startup
chottokun
committed
069638b
feat: add Kubernetes deployment manifests and enrich OpenAPI documentation
chottokun
committed
2b3099f
feat: implement MAX_CONCURRENT_INFERENCES semaphore and multi-key client authentication
chottokun
committed
de0b758
refactor: integrate AsyncThreadSemaphore and bind ErrorResponse to OpenAPI documentation
chottokun
committed
a0852d7

Navigation Menu

feat: production-level reliability, observability, and OpenAI compatibility improvements - #90

feat: production-level reliability, observability, and OpenAI compatibility improvements#90
chottokun merged 21 commits into
mainfrom
feature/production-level-improvements

Commits on Sep 12, 2026

style: fix ruff lint errors across scratch and scripts

ci: enforce uv audit, gitleaks detection, and repository-wide linting

feat: add readiness check endpoint (/ready) and tests

docs: document /ready endpoint, CI security checks, and code health fixes

feat: implement Prometheus metrics (/metrics) and middleware instrumentation

perf: batch multimodal image and text tensor inference in VisualizedBGEEmbeddingModel

refactor: make TEI proxy completely asynchronous with httpx.AsyncClient

docs: document metrics, multimodal batching, and async TEI proxy

ci: update Docker and Compose container healthchecks to use /healthz probe

feat: add structured JSON logging with request ID tracking

docs: document structured JSON logging with request ID tracking

fix(security): mitigate DNS Rebinding and TOCTOU in image downloads via IP pinning

docs: document DNS Rebinding and TOCTOU mitigation in changelog and log

feat: add GET /v1/models, payload limit middleware, and POST /v1/models/unload

docs: document /v1/models, payload limit, and model unload in changelog and log

feat: support dimensions, encoding_format base64, and rate limiting middleware

feat: implement graceful shutdown request drain and TORCH_DTYPE mixed precision inference

feat: add token consumption and batch size metrics, and model preloading on startup

feat: add Kubernetes deployment manifests and enrich OpenAPI documentation

feat: implement MAX_CONCURRENT_INFERENCES semaphore and multi-key client authentication

refactor: integrate AsyncThreadSemaphore and bind ErrorResponse to OpenAPI documentation