×

INDI Library v1.9.7 Released (29 Jul 2022)

Bi-monthly INDI Library released with new drivers and bug fixes.

KStars memory leak and crash

  • Posts: 67
  • Thank you received: 5
I am trying to use the sanitizers on kstars to see what gets thrown out. Threw out a number of complaints but not yet sure whether they are actual leaks or not. Still investigating.
2 months 6 days ago #83487
The topic has been locked.
  • Posts: 202
  • Thank you received: 41

Replied by Gene N on topic KStars memory leak and crash

For whatever it is worth, on a build from source not on an Astoberry distribution (last version I loaded) Kstars 3.5.7, Ian's method to show issue
Quote
"1. backup and then rm -r ~/.local/share/kstars ~/.indi ~/.config/kstarsrc for a fresh start
2. launch KStars
3. launch Ekos
4. wizard local devices -> create profile -> add simulator telescope and CCD
5. start Ekos
6. go to guider tab -> loop
7. Check RES memory usage via htop, check again every few minutes and notice a steady increase
"

I see no issues whatsoever after running for hours. I tried changing the main sim ccd resolution and using that as the guide capture also and still no growth.

This is on Pi with - 1gig- memory and buster
Linux pi-4-1 5.10.63-v7l+
2 months 5 days ago #83513
The topic has been locked.
  • Posts: 202
  • Thank you received: 41

Replied by Gene N on topic KStars memory leak and crash

Disable all Pulseaudio and notifications?
I ran the same on an Astroberry with Kstars 3.5.5 and it gobbled up memory.

Disabled the service as above, still growing
Stopped NGIX service and still growing.

Disabled pulsaudio and all Notifications and -STOPPED- growing.

After an hour:

====================================
top - 12:35:52 up 2:01, 3 users, load average: 0.51, 0.41, 0.54
Tasks: 198 total, 3 running, 195 sleeping, 0 stopped, 0 zombie
%Cpu(s): 9.3 us, 4.4 sy, 0.0 ni, 85.3 id, 0.1 wa, 0.0 hi, 0.9 si, 0.0 st
MiB Mem : 459.7 total, 49.0 free, 213.4 used, 197.3 buff/cache
MiB Swap: 512.0 total, 277.2 free, 234.8 used. 181.3 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27888 root 20 0 638280 184280 90568 R 34.7 39.1 0:55.56 kstars
27947 root 20 0 70380 12016 8544 R 6.9 2.6 0:01.98 indi_simu+
27614 root 20 0 0 0 0 I 5.3 0.0 0:06.31 kworker/u+
570 root 20 0 229664 33880 17792 S 4.3 7.2 4:12.68 Xorg

top - 13:40:28 up 3:06, 3 users, load average: 0.71, 0.63, 0.64
Tasks: 195 total, 1 running, 194 sleeping, 0 stopped, 0 zombie
%Cpu(s): 8.5 us, 3.3 sy, 0.0 ni, 88.1 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st
MiB Mem : 459.7 total, 38.6 free, 211.2 used, 209.9 buff/cache
MiB Swap: 512.0 total, 280.2 free, 231.8 used. 179.2 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27888 root 20 0 638280 184296 90568 S 34.7 39.2 18:46.90 kstars
27947 root 20 0 70380 12128 8608 S 8.3 2.6 4:48.05 indi_simu+

With Pulseaudio/Notifications running:
top - 11:16:12 up 41 min, 3 users, load average: 1.78, 1.30, 0.86
Tasks: 204 total, 3 running, 201 sleeping, 0 stopped, 0 zombie
%Cpu(s): 12.4 us, 6.6 sy, 0.0 ni, 80.2 id, 0.0 wa, 0.0 hi, 0.8 si, 0.0 st
MiB Mem : 459.7 total, 42.7 free, 249.7 used, 167.3 buff/cache
MiB Swap: 512.0 total, 317.0 free, 195.0 used. 132.6 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1929 root 20 0 766644 171056 59476 S 34.1 36.3 5:14.50 kstars
1985 root 20 0 70248 11584 8196 S 9.3 2.5 1:02.27 indi_simu+

astroberry@astroberry:~ $ top
top - 12:13:20 up 1:38, 3 users, load average: 0.29, 0.53, 0.71
Tasks: 198 total, 2 running, 196 sleeping, 0 stopped, 0 zombie
%Cpu(s): 8.6 us, 4.1 sy, 0.0 ni, 87.1 id, 0.0 wa, 0.0 hi, 0.3 si, 0.0 st
MiB Mem : 459.7 total, 31.4 free, 294.2 used, 134.1 buff/cache
MiB Swap: 512.0 total, 296.0 free, 216.0 used. 90.5 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1929 root 20 0 923868 216068 43748 S 31.1 45.9 24:20.27 kstars
1985 root 20 0 70248 7532 4844 S 7.9 1.6 5:15.49 indi_simu
2 months 4 days ago #83542
The topic has been locked.
This is actually a known issue and you can see in various places that notifications on non-KDE systems should be disabled due to some memory leak. I filed a bug report a couple of years ago with KDE (it's an old issue) but I don't think it was ever resolved.
Jasem Mutlaq
Support INDI & Ekos; Get StellarMate Astrophotography Gadget.
How to Submit Logs when you have problems?
Add your observatory info
2 months 3 days ago #83554
The topic has been locked.
  • Posts: 19
  • Thank you received: 1
Ooooo!
So in my alternative post, Nou had commented about a message I'd seen in the log file (and dismissed!) about a phonon error or warning . I'd seen it in the SGL post and as mentioned, ignored it for now.
Seeing Nou's comment, I'd disabled the SOUNDS only but instead wrote warnings to a text file so I'd be able to see warnings or track them back.
Just started another simulation with ALL notifications OFF, and also disabled the "low resources" mode - it looks like my memory usage is more stable, although that's only after a short period. I'll leave it running and come back with another report later.
There was also another update released recently for lots of Indi updates, so if the issue is fixed, it could have been either - I'll test again later with notifications back on and see what happens.
2 months 3 days ago #83562
The topic has been locked.
  • Posts: 207
  • Thank you received: 39
So, now that the problem is found and fixed.... Could you put a summary about how to stop sound and notifications on Astroberry ;)
2 months 2 days ago #83576
The topic has been locked.
  • Posts: 19
  • Thank you received: 1
Nope, still seeing memory usage increase with notifications on or off.
Sorry folks.
I'm not sure how to run heaptrack. It's installed, but not a great linux user to do have to either ask for a how-to crash course, sorry folks.
The following user(s) said Thank You: Rafa Barberá
2 months 2 days ago #83582
The topic has been locked.
  • Posts: 176
  • Thank you received: 38

Replied by nou on topic KStars memory leak and crash

What kind of usage memory increase? Because increase of cache/buffers is no indication of memory leak.
2 months 2 days ago #83584
The topic has been locked.
  • Posts: 18
  • Thank you received: 2

Replied by MH on topic KStars memory leak and crash


This was my experience as well - I delved waaaay into the rabbit hole when I was having this problem and found the "turn notifications and sounds off" potential solution (there are a number of threads on this, with this proposed solution), and it didn't seem to matter.

I recall actively being excited like "oh nice an easy solution" and of course it couldn't be that simple haha
The following user(s) said Thank You: Rafa Barberá
Last edit: 2 months 2 days ago by MH. Reason: Additional thought, quotes still sad
2 months 2 days ago #83585
The topic has been locked.
  • Posts: 19
  • Thank you received: 1
@Nou, sorry if my terminology is incorrect. Just assumed memory leak is when memory is used up continuously, until the system has used all resources and causes things to crash. Let's not get bogged down with terminology!
As in my original post , I've had several sessions crash shortly after going to bed, and believe them to be from a lack of memory on the RPi.
I say this as I've documented the memory usage fill up, then the kstars logfile stops writing (kstars crashes) and the available memory increases and remains steady until I return to offload the imaging data several hours later.
Since really delving deeper (all the different memories, and purging the caches, etc), I've been unable to make it crash again, and as the nights are so short imaging isn't worth it just at present.
Again, I have a 'fix' that should work for me, but I'm disappointed that I've not been able to find the reason and solve the problem, but it does seem like there are others still seeing the same, perhaps without crashes...
Last edit: 2 months 2 days ago by Pete Simpson.
2 months 2 days ago #83586
The topic has been locked.
  • Posts: 18
  • Thank you received: 2

Replied by MH on topic KStars memory leak and crash

Something you may be able to do to recreate (and what I used previously) was to just take a lot of dark frames over a long period of time, especially if you need to build a darks library for specific temperatures and haven't had time yet.

Otherwise, can just take a lot of covered photos even if they aren't needed for calibration frames; the bug tended to reproduce itself when doing something like that for me.

There are also some ways you can log RAM usage over time.

Could use something like one of the answers to log RAM, CPU usage of Kstars over time to a local file and cross-reference, such as a one-liner I quickly cobbled together based on that answer:
while true; do ps -p <kstars_PID> -o %cpu,%mem,cmd; sleep 5; done | while read memStats; do echo "$(date): $memStats"; done | tee -a memtest.logs

This gives you timestamp outputs per line, which you can likely cross-reference with the Kstars logs; I strongly suspect that the memory increase will creep up after each iteration of the memory leak, or around the same time.

Edit: Actually, it looks like pmap may be better suited for this task based on this thread ; so to modify the above, it would look like:
while true; do pmap -x <kstars_PID>; sleep 5; done | while read memStats; do echo "$(date): $memStats"; done | tee -a memtest.logs
Last edit: 2 months 2 days ago by MH. Reason: Likely better diagnostic one-liner
2 months 2 days ago #83587
The topic has been locked.
  • Posts: 176
  • Thank you received: 38

Replied by nou on topic KStars memory leak and crash

Ok just tested and running heaptrack is quite easy. Just install "sudo apt install heaptrack heaptrack-gui" and then run "heaptrack kstars" after you close down kstars it will even tell you next command to analyze results. Try run one short session with few frames exposed and then longer one. Then post screenshot of summary tab so we can investigate further.
2 months 1 day ago #83596
The topic has been locked.
Time to create page: 1.102 seconds