Skip to content

the moondream local rung has never done real inference — needs someone with CUDA or Apple Silicon #653

Description

@LetsGetToWorkBro

What happened

#650 fixed the moondream adapter to construct vl(local=True) only (the bare call was a cloud client that transmitted frames before auth failed — read the PR), and #651 moved to moondream 2.0.1. The local path is kestrel's Photon runtime, and it hard-requires an accelerator:

# moondream/photon_vl.py (2.0.1)
raise RuntimeError(
    "Photon local inference needs a supported accelerator, but neither "
    "CUDA nor Apple Silicon MPS is available in this Python environment.")

Every CI runner here is CPU-only, so test_moondream_stays_local.py proves the adapter's construction through a fake factory — but nobody in this repo has ever watched MoondreamClassifier return a real answer from the real runtime. Per CLAUDE.md #1, that's a gap worth being honest about: the guard tests are green and the actual capability is unverified.

Ask

You have an NVIDIA GPU or an Apple Silicon Mac:

cd host-python
pip install -e ".[vision]"
python - <<'PY'
from PIL import Image
from dreamlayer.object_lens.classify_backends import MoondreamClassifier
clf = MoondreamClassifier()
print(clf(Image.open("any-photo-of-an-object.jpg")))
PY

Report back: hardware, what it printed, wall time for first call (weights download) and second call (warm). If it fails, the traceback is exactly what this issue wants — the except path in _ensure will have swallowed it, so run with logging at DEBUG.

Stretch goal: a real_model-marked test for it. Note the tension before you write one — real-models.yml fails on any skip by design, and its runner is CPU-only, so the test cannot live there until a GPU runner exists. A manually-invoked script under tools/ (or a mark that only a self-hosted runner selects) is the honest shape.

Where

host-python/src/dreamlayer/object_lens/classify_backends.py (MoondreamClassifier), host-python/src/dreamlayer/tests/test_moondream_stays_local.py

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedExtra attention is neededpythonPull requests that update python code

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions