Hi @octo,
500ms/frame is a hard timeout and unfortunately can’t be changed.
We will still discuss in case it can be relaxed further, and let you know. 
You can potentially try batching during inference in case your codebase isn’t using resources fully to fasten it up.