[Solved] Fits viewer freezes (crashes kstars) with auto stretch

Jerry Black
Topic Author
Offline
Elite Member

Posts: 460
Thank you received: 69

Replied by Jerry Black on topic Fits viewer freezes

No problem... here it is www.dropbox.com/s/jmt6cas06gj5qh7/NGC%20...-32-43_100.fits?dl=0

4 years 4 months ago #46064

Please Log in or Create an account to join the conversation.

Jerry Black
Topic Author
Offline
Elite Member

Posts: 460
Thank you received: 69

Replied by Jerry Black on topic Fits viewer freezes

Yes, that size is probably correct. The actual fits file is 90M.

4 years 4 months ago #46065

Please Log in or Create an account to join the conversation.

Jerry Black
Topic Author
Offline
Elite Member

Posts: 460
Thank you received: 69

Replied by Jerry Black on topic Fits viewer freezes

turns out: auto stretch was enabled as well as auto WCS and 3D cube. Turning off all of these allows the image to be displayed.
However as before, now as soon as I click on "Show toggle Autostretch" it dies.
gdb indicates it's still from ./kstars/fitsviewer/stretch.cpp:35 in the call to samples(downsample_size);

// Returns the median of the sample values.
// The values are not modified.
template <typename T>
T median(T *values, int size, int sampleBy)
{
  const int downsampled_size = size / sampleBy;
  std::vector<T> samples(downsampled_size);
  for (int index = 0, i = 0; index < size; ++i, index += sampleBy)
    samples[i] = values[index];
  return median(samples);
}

Thanks

4 years 4 months ago #46068

Please Log in or Create an account to join the conversation.

Hy Murveit
Offline
Administrator

Posts: 1220
Thank you received: 565

Replied by Hy Murveit on topic Fits viewer freezes

Jerry,

Thanks for sending the file.

FWIW, it's a 8256 x 5504 RGB file.
I just ran kstars on a 4Gb Raspberry Pi 4 (and also ran it on my Macbook). I ran File->Open->YOUR_FILE, and it opened and displayed in ~4s on my RPi.
I looked at the process size while I was doing this on my RPi.
Before loading the file, the virtual memory process size was 525Mb and it was taking 145.5Mb of physical memory.
I loaded the file and VM jumps around a bit, peaking at about 1050Mb and winding up at 966Mb, using 560Mb of physical memory.
Toggling the stretch, it stays pretty stable at ~966Mb and then goes back to ~530Mb VM with 150Mb Physical.

Is it possible that you are running out of memory (e.g. it seems possible that you have a 1Gb limit for your machine)?
This is a pretty large file--e.g. simply representing it in memory would be 8256 x 5504 x 3 channels x 2 bytes/channel = 273Mb .
KStars is growing by 525Mb, so perhaps there is some waste in KStars that we could remove.
(and yes, it may or may not be necessary to keep it all in memory to display a smaller version of it, but currently it's all in memory, and then some).

Have you opened files this large on your same machine before?
How much memory do you have, how much virtual memory (you can get those by running the linux top command, see below).
Were there other jobs occupying lots of memory?
Can you run top as I did and see how much virtual and physical memory you're using before and while you open this file?

Thanks,
Hy

PS in case you're not familiar with it, here's how I'd use top.
I'd type just the word top in a unix shell with kstars already started, but no fits file loaded yet.
The 4th and 5th line will tell you how much virtual and physical memory you have (e.g. in my case it says 3900 Mb).
Look for the PID for the kstars line and note that number.
Then kill top (with control-c) and restart it as : top -p THAT_PID, e.g. top -p 8927
Now it will just show the kstars process to make it easier.
Type: s 1 and it will refresh every second.
Type: e a few times, it cycles through how it displays the memory, I like to see numbers in megabytes (with an m after the VIRT and RES numbers, where res means physical memory).
Then you can load your file and note how the VIRT and RES columns change.

4 years 4 months ago #46076

Please Log in or Create an account to join the conversation.

Jerry Black
Topic Author
Offline
Elite Member

Posts: 460
Thank you received: 69

Replied by Jerry Black on topic Fits viewer freezes

Thanks Hy for taking an interest, your help is greatly appreciated.

The Odroid-N2 is 4G and I was able to open these previously and I <em>think</em> with auto screen stretch enabled.
It opens in about 2 sec on the Odroid-N2 (the eMMC storage is faster than SD). The image opens with Kstars on my Mac and can be auto-stretched without a problem.

top gives:

KiB Mem : 3801320 total, 1780620 free, 1072236 used
KiB Swap: 1900632 total, 1900632 free,   0 used
 
              VIRT     RES         SHR     S  %CPU  %MEM
kstars      2088.8m   240.4m     90.5m     S    9    6.5
kstars/fits 2988.8m   687.1m     95.7m     S    9   18.5

the gdb bt doesn't appear to have changed... This is an AstroPi3 install on the Odroid-N2, maybe it's a compiler optimization issue (a wild guess based on the optimized out messages in the bt)?

(gdb) bt
#0  0x0000007fb52804d8 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x0000007fb52818b4 in __GI_abort () at abort.c:79
#2  0x0000007fb52ba65c in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7fb537bb88 "%s\n") at ../sysdeps/posix/libc_fatal.c:181
#3  0x0000007fb52c09c4 in malloc_printerr (str=str@entry=0x7fb5377878 "double free or corruption (!prev)") at malloc.c:5350
#4  0x0000007fb52c25e8 in _int_free (av=0x7fb53a1a70 <main_arena>, p=0x555b4bf4b0, have_lock=<optimized out>) at malloc.c:4281
#5  0x00000055558fc73c in __gnu_cxx::new_allocator<unsigned short>::deallocate(unsigned short*, unsigned long) (this=<synthetic pointer>, __p=<optimized out>)
    at /usr/include/c++/7/ext/new_allocator.h:125
#6  0x00000055558fc73c in std::allocator_traits<std::allocator<unsigned short> >::deallocate(std::allocator<unsigned short>&, unsigned short*, unsigned long) (__a=<synthetic pointer>..., __n=<optimized out>, __p=<optimized out>) at /usr/include/c++/7/bits/alloc_traits.h:462
#7  0x00000055558fc73c in std::_Vector_base<unsigned short, std::allocator<unsigned short> >::_M_deallocate(unsigned short*, unsigned long) (this=<synthetic pointer>, __n=<optimized out>, __p=<optimized out>) at /usr/include/c++/7/bits/stl_vector.h:180
#8  0x00000055558fc73c in std::_Vector_base<unsigned short, std::allocator<unsigned short> >::~_Vector_base() (this=<synthetic pointer>, __in_chrg=<optimized out>)
    at /usr/include/c++/7/bits/stl_vector.h:162
#9  0x00000055558fc73c in std::vector<unsigned short, std::allocator<unsigned short> >::~vector() (this=<synthetic pointer>, __in_chrg=<optimized out>)
    at /usr/include/c++/7/bits/stl_vector.h:435
#10 0x00000055558fc73c in (anonymous namespace)::median<unsigned short> (sampleBy=<optimized out>, size=45441024, values=0x3f80000000000000, this=<optimized out>, this=<optimized out>) at ./kstars/fitsviewer/stretch.cpp:35
#11 0x00000055558fc73c in (anonymous namespace)::computeParamsOneChannel<unsigned short> (width=<optimized out>, height=<optimized out>, inputRange=127, params=0x7fffffd9e8, buffer=0x3f80000000000000, this=<optimized out>, this=<optimized out>) at ./kstars/fitsviewer/stretch.cpp:219
#12 0x00000055558fc73c in Stretch::computeParams(unsigned char*) (this=this@entry=0x7fffffda68, input=0x7f06149010 "X\004_\004_\004\243\004\243\004\353\003\353\003~\004~\004\243\005\243\005\031\005\031\005D\005D\005,\006,\006\027\004\027\004\206\004\206\004\060\005\060\005\261\004\261\004\356\003\356\003\311\005\311\005_\005_\005\314\004\314\004\270\003\270\003\314\004\314\004U\005U\005'\005'\005\210\004\210\004X\004X\004n\004n\004\033\005\033\005\353\004\353\004\355\005\355\005%\005%\005z\004z\004f\004f\004k\005k\005\367\004\367\004\206\004\206\004k\005k\005\206\004\206\004\006\005\006\005]\005]\005\264\004\264\004\b\005\b\005\356\004\356\004I\004I\004\070\004\070\004d\004d\004\017\005\017\005Q\005Q\005\337\005\337\005\203\004\203\004f\005f\005\r\005\r\005\001\004\001\004f\004"...) at ./kstars/fitsviewer/stretch.cpp:378
#13 0x00000055557b1768 in (anonymous namespace)::doStretch(FITSData*, QImage*, bool) (data=0x55586bc000, outputImage=outputImage@entry=0x55586b6a08, stretchOn=<optimized out>)
    at ./kstars/fitsviewer/fitsview.cpp:56
#14 0x00000055557b99e4 in FITSView::rescale<unsigned short>(FITSZoom) (this=this@entry=0x55586b6920, type=type@entry=ZOOM_KEEP_LEVEL) at ./kstars/fitsviewer/fitsview.cpp:423
#15 0x00000055557bb3f0 in FITSView::rescale(FITSZoom) (this=this@entry=0x55586b6920, type=type@entry=ZOOM_KEEP_LEVEL) at ./kstars/fitsviewer/fitsview.cpp:340
#16 0x00000055557bb518 in FITSView::toggleStretch() (this=0x55586b6920) at ./kstars/fitsviewer/fitsview.cpp:1218
#17 0x00000055558f3ad4 in FITSViewer::toggleStretch() (this=0x555996b1c0) at ./kstars/fitsviewer/fitsviewer.cpp:1003
#18 0x00000055558457e4 in FITSViewer::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (_o=<optimized out>, _c=<optimized out>, _id=<optimized out>, _a=<optimized out>)
    at ./obj-aarch64-linux-gnu/kstars/KStarsLib_autogen/GB6ZSSQLTO/moc_fitsviewer.cpp:251
#19 0x0000007fb5f5117c in QMetaObject::activate(QObject*, int, int, void**) () at /usr/lib/aarch64-linux-gnu/libQt5Core.so.5
#20 0x0000007fb68f6ec8 in QAction::triggered(bool) () at /usr/lib/aarch64-linux-gnu/libQt5Widgets.so.5
#21 0x0000007fb68f990c in QAction::activate(QAction::ActionEvent) () at /usr/lib/aarch64-linux-gnu/libQt5Widgets.so.5

Last edit: 4 years 4 months ago by Jerry Black.

4 years 4 months ago #46083

Please Log in or Create an account to join the conversation.

Jerry Black
Topic Author
Offline
Elite Member

Posts: 460
Thank you received: 69

Replied by Jerry Black on topic Fits viewer freezes (crashes kstars) with auto stretch

Hi,

Strangely for me, I get the same error (as I see when running on the AstroPi3 Odroid-N2) when I tested this on my RPi4 (4G) using Stellarmate - as soon as I do an auto-stretch

Cheers

stellarmate@stellarmate:~$ gdb kstars
GNU gdb (Ubuntu 8.1-0ubuntu3.2) 8.1.0.20180409-git
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "aarch64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from kstars...Reading symbols from /usr/lib/debug/.build-id/05/4c30d4c3320dea2f9858f5402d028749695ab0.debug...done.
done.
(gdb) run
Starting program: /usr/bin/kstars 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ff0a29080 (LWP 16156)]
[New Thread 0x7febbd5080 (LWP 16157)]
org.kde.kstars: Welcome to KStars 3.3.8
org.kde.kstars: Build: 2019-11-20T17:32:01Z
org.kde.kstars: OS: "neon"
org.kde.kstars: API: "arm64-little_endian-lp64"
org.kde.kstars: Arch: "arm64"
org.kde.kstars: Kernel Type: "linux"
org.kde.kstars: Kernel Version: "4.19.81-v8-20"
org.kde.kstars: Qt Version: 5.9.5
[New Thread 0x7fe8971080 (LWP 16159)]
[New Thread 0x7fe0567080 (LWP 16160)]
[New Thread 0x7fdfd66080 (LWP 16161)]
org.kde.kstars: Processing  "unnamedstars.dat" , HTMesh Level 3
org.kde.kstars:   Sky Mesh Size:  512
org.kde.kstars: Loaded DSO catalog file:  "unnamedstars.dat"
org.kde.kstars: "Star HD20,794 not found."
org.kde.kstars: "Star HD98,230 not found."
File opened:  "/usr/share/kstars/ngcic.dat"
org.kde.kstars: Loading NGC/IC objects
File opened:  "/home/stellarmate/.local/share/kstars/comets.dat"
org.kde.kstars: "Object named NGC 6050A not found"
[New Thread 0x7fdf565080 (LWP 16163)]
glibc >= 2.1 detected.  Using GNU extension sincos()
[New Thread 0x7fde29a080 (LWP 16165)]
[New Thread 0x7fdda99080 (LWP 16166)]
[New Thread 0x7fdc9a1080 (LWP 16167)]
[New Thread 0x7fbffff080 (LWP 16168)]
[New Thread 0x7fbf7fe080 (LWP 16169)]
[New Thread 0x7fbeffd080 (LWP 16172)]
File opened:  "/home/stellarmate/.local/share/kstars/comets.dat"
[Thread 0x7fbeffd080 (LWP 16172) exited]
File opened:  "/home/stellarmate/.local/share/kstars/asteroids.dat"
[Thread 0x7fde29a080 (LWP 16165) exited]
[Thread 0x7fdda99080 (LWP 16166) exited]
[Thread 0x7fbffff080 (LWP 16168) exited]
org.kde.kstars.fits: Loading FITS file  "/home/stellarmate/Pictures/NGC 7635_Light_120_secs_2019-10-29T22-03-07_114.fits"
double free or corruption (!prev)
 
Thread 1 "kstars" received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x0000007ff50b38b4 in __GI_abort () at abort.c:79
#2  0x0000007ff50ec65c in __libc_message (action=action@entry=do_abort, 
    fmt=fmt@entry=0x7ff51adb88 "%s\n") at ../sysdeps/posix/libc_fatal.c:181
#3  0x0000007ff50f29c4 in malloc_printerr (
    str=str@entry=0x7ff51a9878 "double free or corruption (!prev)") at malloc.c:5350
#4  0x0000007ff50f45e8 in _int_free (av=0x7ff51d3a70 <main_arena>, p=0x555b70cad0, 
    have_lock=<optimized out>) at malloc.c:4281
#5  0x00000055558fc73c in __gnu_cxx::new_allocator<unsigned short>::deallocate (
    this=<synthetic pointer>, __p=<optimized out>)
    at /usr/include/c++/7/ext/new_allocator.h:125
#6  std::allocator_traits<std::allocator<unsigned short> >::deallocate (
    __a=<synthetic pointer>..., __n=<optimized out>, __p=<optimized out>)
    at /usr/include/c++/7/bits/alloc_traits.h:462
#7  std::_Vector_base<unsigned short, std::allocator<unsigned short> >::_M_deallocate (
    this=<synthetic pointer>, __n=<optimized out>, __p=<optimized out>)
    at /usr/include/c++/7/bits/stl_vector.h:180
#8  std::_Vector_base<unsigned short, std::allocator<unsigned short> >::~_Vector_base (
    this=<synthetic pointer>, __in_chrg=<optimized out>)
    at /usr/include/c++/7/bits/stl_vector.h:162
#9  std::vector<unsigned short, std::allocator<unsigned short> >::~vector (
    this=<synthetic pointer>, __in_chrg=<optimized out>)
    at /usr/include/c++/7/bits/stl_vector.h:435
#10 (anonymous namespace)::median<unsigned short> (sampleBy=<optimized out>, 
    size=45441024, values=0x3f80000000000000, this=<optimized out>, 
    this=<optimized out>) at ./kstars/fitsviewer/stretch.cpp:35
#11 (anonymous namespace)::computeParamsOneChannel<unsigned short> (
    width=<optimized out>, height=<optimized out>, inputRange=127, 
---Type <return> to continue, or q <return> to quit--- 
    buffer=0x3f80000000000000, this=<optimized out>, this=<optimized out>) at ./kstars/fitsviewer/stretch.cpp:219
#12 Stretch::computeParams (this=this@entry=0x7fffffd888, 
    input=0x7f8a149010 "#\004E\004E\004,\004,\004o\005o\005\307\004\307\004\206\004\206\004f\004f\004i\005i\005\206\004\206\004\264\004\264\004f\004f\004R\005R\005m\005m\005u\004u\004\302\004\302\004\212\005\212\005\376\003\376\003\024\004\024\004I\004I\004\066\004\066\004\353\003\353\003\003\004\003\004Q\005Q\005r\005r\005r\005r\005\360\004\360\004\337\003\337\003\222\004\222\004\374\005\374\005\264\005\264\005\275\004\275\004\b\004\b\004\243\004\243\004\374\003\374\003\344\004\344\004\330\004\330\004\351\004\351\004\316\004\316\004\215\004\215\004\311\004\311\004\344\003\344\003\r\005\r\005G\006G\006I\004I\004\240\005\240\005p\004p\004\376\003\376\003\206\005\206\005\325\004\325\004\240\004\240\004;\004"...) at ./kstars/fitsviewer/stretch.cpp:378
#13 0x00000055557b1768 in (anonymous namespace)::doStretch (data=0x5558536a20, outputImage=outputImage@entry=0x55585f1958, 
    stretchOn=<optimized out>) at ./kstars/fitsviewer/fitsview.cpp:56
#14 0x00000055557b99e4 in FITSView::rescale<unsigned short> (this=this@entry=0x55585f1870, type=type@entry=ZOOM_KEEP_LEVEL)
    at ./kstars/fitsviewer/fitsview.cpp:423
#15 0x00000055557bb3f0 in FITSView::rescale (this=this@entry=0x55585f1870, type=type@entry=ZOOM_KEEP_LEVEL)
    at ./kstars/fitsviewer/fitsview.cpp:340
#16 0x00000055557bb518 in FITSView::toggleStretch (this=0x55585f1870) at ./kstars/fitsviewer/fitsview.cpp:1218
#17 0x00000055558f3ad4 in FITSViewer::toggleStretch (this=0x55585f90f0) at ./kstars/fitsviewer/fitsviewer.cpp:1003
#18 0x00000055558457e4 in FITSViewer::qt_static_metacall (_o=<optimized out>, _c=<optimized out>, _id=<optimized out>, 
    _a=<optimized out>) at ./obj-aarch64-linux-gnu/kstars/KStarsLib_autogen/GB6ZSSQLTO/moc_fitsviewer.cpp:251
#19 0x0000007ff5d89c64 in QMetaObject::activate(QObject*, int, int, void**) () from /usr/lib/aarch64-linux-gnu/libQt5Core.so.5
#20 0x0000007ff67b54e0 in QAction::triggered(bool) () from /usr/lib/aarch64-linux-gnu/libQt5Widgets.so.5
#21 0x0000007ff67b7e9c in QAction::activate(QAction::ActionEvent) () from /usr/lib/aarch64-linux-gnu/libQt5Widgets.so.5
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb)

4 years 4 months ago #46089

Please Log in or Create an account to join the conversation.

Jerry Black
Topic Author
Offline
Elite Member

Posts: 460
Thank you received: 69

Replied by Jerry Black on topic Fits viewer freezes (crashes kstars) with auto stretch

So I compiled Kstars from source code on the Odroid-N2 and debugged it using QTCreator.

in stretch.cpp
if I halve the size of maxSamples e.g. 5000000 / 2; shown below (line 216)
This code then works for my large fits files.

I'm not sure exactly what the memory problem was, but that seems to get around the problem.

Is there a better way to report this. than here? How do I contact @murveit

// Find the median sample.
constexpr int maxSamples = 500000 / 2;
const int sampleBy = width * height < maxSamples ? 1 : width * height / maxSamples;
const int size = width * height;
T medianSample = median(buffer, width * height, sampleBy);

The following user(s) said Thank You: Hy Murveit

4 years 4 months ago #46111

Please Log in or Create an account to join the conversation.

Hy Murveit
Offline
Administrator

Posts: 1220
Thank you received: 565

Replied by Hy Murveit on topic Fits viewer freezes (crashes kstars) with auto stretch

Jerry,

Thanks for your reports. I do appreciate them. I am looking into it. I doubt that the maxSamples change is a general fix, though it may fix this specific crash on your specific machine. There must be some memory issue somewhere in the code, but I was having difficulty finding it. I'll continue to look and have asked for help too. I've also messaged you on this forum. Please check your messages.

Hy

4 years 4 months ago #46112

Please Log in or Create an account to join the conversation.

Hy Murveit
Offline
Administrator

Posts: 1220
Thank you received: 565

Replied by Hy Murveit on topic Fits viewer freezes (crashes kstars) with auto stretch

BTW, if you've gone as far as compiling on your odroid and running with QtCreator, you'd do me a big favor if you could also undo you change, run valgrind in qtdreator and watch the crash, and send me all the valgrind output. See doc.qt.io/qtcreator/creator-valgrind-overview.html Would want to make sure valgrind is running its memory checker, but I think it does that by default.

Thanks,
Hy

4 years 4 months ago #46113

Please Log in or Create an account to join the conversation.

Andrew
Offline
Supernova Explorer

Posts: 1309
Thank you received: 226

Replied by Andrew on topic Fits viewer freezes

This is a tangent, but you appear to have some issues in your optical train to look into. There's coma and plenty of vignetting. Maybe too much focal reduction? And there are prominent diffraction spikes that shouldn't be present with an SCT. That can happen with cabling for hyperstar, or worst case a scratched corrector plate, filter, focal reducer...

4 years 4 months ago #46118

Please Log in or Create an account to join the conversation.

Jerry Black
Topic Author
Offline
Elite Member

Posts: 460
Thank you received: 69

Replied by Jerry Black on topic Fits viewer freezes

Thanks for the feedback, yes I'm not sure what's going on with this, but it's likely all my doing... I'd appreciate any adivce.

The SCT is an OLD hand-me-down 10" Meade which I have attached an 2" extension tube from a slightly newer 8" Meade LX200 with no focal reducer between to a Nikon Z7. I'm not sure why I get that vignetting. The coma I presume is a normal reflection of this SCT's quality.
For the diffraction spikes, I don't really know either, I assumed it was a side effect of a 120 sec. exposure at ISO 12800

I have had advice that that ISO was crazy, I was just seeing what a larger number of shorter high ISO exposures might do when combined. Here is the result (for better or worse)
www.dropbox.com/s/o2pwgnsgwhsk6vd/light_..._sub_div_ST.jpg?dl=0
not quite as clean or sharp as those refractors the imagers around here have.

4 years 4 months ago #46120

Please Log in or Create an account to join the conversation.

Andrew
Offline
Supernova Explorer

Posts: 1309
Thank you received: 226

Replied by Andrew on topic Fits viewer freezes

Ok. I'll just quickly say that from a little sleuthing I found that the older Meade SCTs require their F/6.3 focal reducer/field flattener to correct for these aberrations. Their newer models feature 'advanced coma-free optics' and do not require one.
As for the diffraction spike, inspect all optical surfaces carefully. Also check your secondary mirror collimation from time to time, especially if you are transporting the telescope.

Clear skies.

4 years 4 months ago #46152

Please Log in or Create an account to join the conversation.

INDI Library v2.0.7 is Released (01 Apr 2024)