Sounds similar to the problem I was having with my ASI120mm-mini guide camera. Everything worked fine indoors, would run for hours but when I plate-solved it wall came tumbling down.
The solution was to change the camera for 16bit to 8bit in the INDI control panel. Once that was done, it worked as expected.