This explains my crashes as I have been doing the same thing, inspecting the subs by zooming in and often it would stall for up to one minute and recover or Kstars simply crashes.

I couldn't locate the source of the leak. So you're sure it's KStars and not the INDI driver?

This is not a consistent occurrence. A few nights ago, Kstars crashed after ~100 frames (32MB each) and I was greeted with the Desktop telling me that there was not enough memory to send a report.

Last night I had a totally uneventful unattended session, I collected ~15GB of images until the early morning using the scheduler without memory use having increased beyond 2 GB.

I have no idea what triggers these irregular memory leaks/overruns that result in Kstars crashing or how to provoke them reliably.

I am running Indi and Kstars build of 02-13-2020 on Ubuntu MATE 18.04.

This is very good point!
The leak is directly related to image files and their size. So there are two components to blame: CCD driver or EKOS.
I used indi_atik_ccd but the test should be run with other CCD drivers to narrow the search.

Please copy this simple script to /usr/local/bin/test_memleak
Then set the access rights:
Then add the script to EKOS CCD module when defining your session. Capture around 30 images and check the content of /tmp/memstat file , which contains memory available after each capture. Calculate average and compare it to a single FITS file (bin1 and bin2)

I have just tested it with ZWO ASI 120MM (file sizes: 2465 kB for BIN1 and 622 kB for BIN2)

30 x 1s BIN1
average memory drop: 2796 kB
standard deviation: 1101 kB

30 x 1s BIN2
average memory drop: 858 kB
standard deviation: 1126 kB

Apparently the memory leak is still related to file size and does not depend on CCD driver used.

Why do these leaks happen randomly with the same installation - no changes made?

Here goes an update on testing this nasty bug. When you set Upload to Local (instead of default Client), there is no memory leak. Even if you enable Summary Screen Preview and Use FITS Viewer options.
Make sure to set directory where your files should be written to. For whatever reason the parameter is called Remote (it is on the right side of Upload drop-down control) in Ekos CCD tab.
Stay tuned

Got a screen shot?

Look the screenshot above and set the options just below Directory: Upload to Local, Remote to directory where you want the files to be saved.
This way files will be saved only on the machine where INDI server is running.

Interesting; you might be onto something as I have mine set to "client" as well and have been experiencing the crash, as you know....



Jim

I think I nailed it! So the actual memory leak is related to FITS Viewer.
1. If you set Upload to Local, FITS Viewer does not render preview images (this is a feature) - no memory leak
2. If you set Upload to Client, you can enable/disable image preview in FITS Viewer in separate window (KStars / Settings / Configure KStars / FITS / Use FITS Viewer )
a) FITS Viewer disabled - confirmed memory leak at image file size for every captured image
b) FITS Viewer enabled - possible minor memory leak BUT memory is released right after you close FITS Viewer

I will dive into the code after this finding so a persistent solution is developped.
In the meantime... If you face a memory leak bug and you use Upload to Client option in Ekos, you should ENABLE FITS Viewer (KStars / Settings / Configure KStars / FITS / Use FITS Viewer ) and close the viewer window from time to time.
Otherwise use Upload to Local, which will make you immune to this bug.

Thanks! I actually never ran it without FITS viewer enabled... I'll get right to it.