Discovery of providers with delegated-ipfs.dev

[agmap]

Discovery of providers
For many CIDs that I want to fetch, no providers are found because they are not announced on ipni-endpoint cid.contact but on the delegated routing endpoint delegated-ipfs.dev.

It would be really helpful if the provider discovery would use the delegated routing endpoint
https://delegated-ipfs.dev/routing/v1/providers//routing/v1/providers/{cid}

How can I set up this endpoint for provider discovery in Lassie fetch?

https://specs.ipfs.tech/routing/http-routing-v1/#get-routing-v1-providers-cid

[willscott]

I guess this is because the cid.contact endpoint isn't running cassette to query the DHT currently, right @masih ?

Agree that these two endpoints should ideally be unified.

Lassie currently uses the IPNI query endpoint, which came out before the IPFS delegated routing interface, and has a couple small differences in semantics that mean you can't 'just switch'.

[masih]

I guess this is because the cid.contact endpoint isn't running cassette to query the DHT currently

In terms of root cause this is due to the fact that delegated routing provide functionality is built around the way DHT provide works. IPNI is built for mass publication, and advertising the diff of what is no longer provided, instead of periodic one cid at a time publishing of CIDs onto the DHT.

This fundamental difference makes it difficult to implement the backing for delegated routing provide in IPNI.

A while back (2 years ago?) i proposed the notion of ephemeral advertisements for IPNI which would have been equivalent to the way DHT provide works where a content provider would need to periodically re-advertise content. That idea has not been implemented.

[masih]

About DHT lookup fallback: Cascade is not currently running on cid.contact. Operationally speaking it is expensive to run. If there is sufficient demand we can reconsider?

[agmap]

If the DHT cannot be integrated into cid.contact, it would be nice if I could configure the delegated-ipfs.dev endpoint for the provider search in Lassie:
https://delegated-ipfs.dev/routing/v1/providers//routing/v1/providers/{cid}

[parkan]

I'm considering trying to implement this in my fork since the legacy API on cid.contact appears to be broken:

https://cid.contact/routing/v1/providers/bafybeihucxcdihbimpydhn35u4velf64w5uxgaztyb5s6uyqtms3uxogoq <--- good, has the real transport-ipfs-gateway-http peer
https://cid.contact/multihash/QmemTSN6sgk4PsRyGDNni24niDMpuoYjqUWJ1813juxRis <--- only has the storacha claim peer and doesn't respect protocol filters

trying to pull this with lassie results in barfing on decoding the storacha metadata and retrieval failure with no candidates

EDIT: @masih has confirmed that this issue is due to caching on the legacy endpoint, however I suspect that I may still want to support delegated routing since we run a custom node for that