I did some more digging today and it seems that this issue does indeed only show up on certain platforms; I was not able to reproduce it on x84_64.
I tried swapping out hubs, a variety of cables, different ports (all of the before as USB3 and USB2) and attaching directly.
I cloned HEAD and played around a bit on my armv7l device that I mainly use for capturing.
Out of the box I see freezes of the cam after a few captures.
Moving backwards in time, swapping out libASICamera2.bin freezes still appeared with
seems to work fine and I was able to take a few hundred captures without any issue.
So I think it looks like a bug was introduced in the SDK.