k4A doesn't reach 30fps

[paolo-scoppola]

Hi prisonerjohn,

I am testing example-streams on a pc with Intel i7 GTX1060 running both Windows 10 + Visual Studio 2017 and Ubuntu 18.04 + Qt Creator. The Kinect grabbing frame rate doesn't reach 30 fps as it does in the Microsft SDK example, that is in the specs of the camera.

Windows: app 50 fps k4a: 15 fps.

the console gives first a list of "[warning] ofGLProgrammableRenderer: draw(): texture is not allocated"
the it gives "[warning] ofxAzureKinect::Stream::updatePixels: No Color capture found (358)!" once per second.

Ubuntu: app 60fps k4a: 19 fps.

Do you have any suggestion? It's a strong limitation...

Thanks a lot,

Paolo

[prisonerjohn]

Are you using the body tracker? If so, there's a potential fix here you can try: https://github.com/prisonerjohn/ofxAzureKinect/commits/hotfix/body-latency

If you want, you can also share your project and I can test on my end.

[paolo-scoppola]

Hi prisonerjohn. I know about the potential fix, but I am using just your stream example with only RGB/DEPTH/IR frames. On my PC your example runs with a grabbing frame rate between 14 and 16 fps in place of 30, while the rendering frame rate is 47 in place of 60.

The Azure Kinect Viewer from Microsoft get 30 fps on grabbing, so it looks like there is bottle neck somewhere else, maybe in the addon itself. That is, my pc is able to make the Kinect work has it is expected.

This is my pc configuration:

i7-7700HQ CPU 2.80 GHz / 16 GB Ram / NVidia GTX 1060 - Windows 10 - OF 10.0.0 - Visual Studio 2017

I send you also your addon as I downloaded from the dev branch with the stream example compiled. So you can run and see your performances.

https://www.dropbox.com/s/l7td6s7szz8vz9x/ofxAzureKinect.zip?dl=0

Thanks a lot!!

Paolo

[prisonerjohn]

How can you tell what the actual framerate is in the Azure Kinect Viewer?
If I use the same configuration, the video feedback and latency looks about the same to me... but it's hard to tell.

If you don't need full 4K resolution for color, lowering that can help improve performance.

[paolo-scoppola]

Hi prisonerjohn,

I am really sorry for this late reply, something didn't work and I didn't received the notification of your answer.

Yes, you're right, the Microsoft app don't tell you the grabbing frame rate, but on the three pc I used for the test, it's quite easy to understand that on this app it runs sensibly faster then your addon, arguably 30 fps, that is the maximum from the kinect 3. Moreover my OF application can use also the Kinect 1 and 2 and these two cameras reach a grabbing frame rate that looks much higher then the Kinect 3. I've been developing OF application for kinect since ten years and my eyes reconginze immediatly if thing are running smooth or not ... ;)

I know Theo Watson used your addon in a project and he says it worked very well. I am attending to speak with him to have his feedback. I noticed that the grabbing frame rate is not the same for all of the three PC. It's higher on that one with Ubuntu and on the other two Windows PC it's higher on that one with a better CPU.

Maybe both Theo and you have a PC with high speed CPU that don't slow down the grabbing frame rate.

Thanks a lot!

[prisonerjohn]

Thanks for the details @paolo-scoppola
Please let me know if you figure something out or if there's any additional testing I can do on my end.

[paolo-scoppola]

Yes,

One thing it could be useful is investigating the point in your code when you take pixel data from the Microsoft API and put in the OF data structure. This is the point where we could lose fps. Maybe the data conversion takes too much time and so OF buffer isn't ready to receive a new frame when it's provided by the Kinect. I try to read your code, but I couldn't find an answer.

[paolo-scoppola]

I found it. It caused by OF 10.0.

On OF 11.0 and 11.2 it reaches 30fps

[paolo-scoppola]

Review: on 0.11.2 I get 30fps, on 0.11.0 it looks I still get 16 fps

[prisonerjohn]

Wow weird, but interesting find. Any idea what the change in OF that causes it is?

[paolo-scoppola]

Not at all... I read the release notes, but nothing made me think about. One thing: I cannot set 1024x1024 resolution on the depth camera. I just changed the camera setting in your stream example and this feature still not work even on 0.11.2