New and crazy camera types

[henrypinkard]

There are many new types of cameras coming online, like event-based cameras, lensless cameras, multi-color cameras (i.e. more than just RGB), point scanning systems that don't have physical sensors, and compound cameras that combine multiple physical sensors. It might be hard to design an API that accounts for all this in a rapidly changing field, but it is at least worth considering all the stuff that currently exists.

I'm hoping here that people with knowledge/experience with different camera types can share some thoughts:

• What nontraditional cameras have you worked with?
• How do they differ from a 2D monochrome or RGB sensor?
• How it breaks the assumptions of and doesn't conform to the current camera API?
• What unique challenges would be posed in creating a generic API for their use (e.g. how they are controlled, data format)?
• Anything else worth considering?

[rwb27]

This is not really a novel camera, but possibly a camera system that doesn't fit the current API: I work a lot with Raspberry Pi cameras, usually controlled over the network. That means that the quickest way to capture an image ends up with the image on a remote machine and I retrieve it later. So far so good, it works nicely with snapImage and getImage, but it doesn't play nicely with a local ring buffer. I'm not a heavy MMCore user (yet!) so my apologies if this is spurious, but my reading of the docs is that there's a bit of assumption that there is a ring buffer, and that it is local, I think? That's definitely something that could be generalised to good effect.

Related to this, I not infrequently get into a situation where I want to call snapImage several times before calling getImage. Is there a way to return a reference from snapImage that you then pass to getImage so that you know you're retrieving the right thing? This makes my network implementation way easier (and is the way I currently do it from non-MMCore control scripts).