×

INDI Library v2.0.6 is Released (02 Feb 2024)

Bi-monthly release with minor bug fixes and improvements

New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed

  • Posts: 1309
  • Thank you received: 226
The new solver really can solve fast! But I'm not exactly sure how fast as I did not see a time to solve stat in the output. That would be useful for determining optimal profile or custom settings.
I have managed to crash it on occasion with the simulator, but it's not very repeatable.
3 years 5 months ago #61987

Please Log in or Create an account to join the conversation.

  • Posts: 1009
  • Thank you received: 133
When I try to compile the current stellarsolver my compiler (GCC 10.0) complains about missing includes in stellarsolver/sep/extract.cpp (#include <cstdio>) and stellarsolver/sep/lutz.cpp (#include <cstdlib>), else sprintf and malloc are not declared....

As for kstars, should I use 'master' or 'extractor' for testing?
3 years 5 months ago #61991

Please Log in or Create an account to join the conversation.

  • Posts: 106
  • Thank you received: 4

I am using an Intel NUC with 4 physical cores (8 logical threads) and 16 GB of RAM.


Already added by the kstars team, thanks!
Powered by

GNU / Linux
Git
KDE neon
KStars | EKOS | INDI

and some cheap hardware
3 years 5 months ago #62004

Please Log in or Create an account to join the conversation.

  • Posts: 106
  • Thank you received: 4
Meant as feedback for the developers:
Internal SEP | Internal Solver | 4-ParallelSmallScale (no downscale)

M57 solves in 4.44 seconds
Iris Nebula solves in 3.50 seconds
M65 solves in 2.84 seconds

Good work!
Powered by

GNU / Linux
Git
KDE neon
KStars | EKOS | INDI

and some cheap hardware
3 years 5 months ago #62005

Please Log in or Create an account to join the conversation.

  • Posts: 152
  • Thank you received: 20
Hi Rob,

I finally had a chance to do some testing with StellarSolver. I'm running a build from kstars commit 15ba3c90a1ec4724a28b43cfa69fa1fb6693242a and SS commit b8ce7148a1bad6f9dd4f5d8b4c4ef95396db4a3c.

I had been using ASTAP. I switched the solving method to "Internal solver" and left all settings as-is. I then ran Load and Slew and it just worked out of the box - no problems, generally. However, I did find a failure case that I'll share here.

BTW - when testing with the simulators I left the Solver Action as "Nothing" but the slew still occurred. Also, after the unwanted slew, the Load and Slew button was now greyed out. To recover it I had to stop/start the indi services.

The following image solves with no problem, even though the image has some problems including mystery stars.

www.dropbox.com/s/20kh08zb0xhc1y5/ic63_L...-35-21_001.fits?dl=0

This one works as well, without the mystery stars:

www.dropbox.com/s/4lx6wqjxu40lll6/ic63_L...-52-38_002.fits?dl=0

However, this one of the same field does not solve. The main differences I can determine is the median signal of this frame is higher than the other two and the image is rotated 180deg from the others.

www.dropbox.com/s/cfxuw6xdln8qjc5/ic63_L...-10-04_040.fits?dl=0

I've attached the log file with Alignment logging turned on. Also, below is from the log window on the alignment tab. Interesting that it says it needs 19.7GB of RAM. Is that true?

The optics are a corrected Dall-Kirkham at focal length of ~1700mm. Pixel scale is 6um.

Greg
2020-10-22T09:40:09 Solver Failed
2020-10-22T09:40:09 internal pixel buffer full
2020-10-22T09:40:00 Starting Internal StellarSolver Sextractor. . .
2020-10-22T09:40:00 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2020-10-22T09:40:00 There should be enough RAM to load the indexes in parallel.
2020-10-22T09:40:00 Evaluating Installed RAM for inParallel Option.  Total Size of Index files: 19.7149 GB, Installed RAM: 31.2902 GB
2020-10-22T09:39:35 Solver Failed
2020-10-22T09:39:35 Solver was aborted, timed out, or failed, so no solution was found
2020-10-22T09:39:35 Starting Internal StellarSolver Astrometry.net based Engine. . .
2020-10-22T09:39:35 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2020-10-22T09:39:35 Image width 640 pixels; arcsec per pixel range 0.93949 1.76154
 
2020-10-22T09:39:35 Scale range: 10.0212 to 18.7898 arcmin wide
 
2020-10-22T09:39:35 Set odds ratio to solve to 1e+9 (log = 20.7233)
 
2020-10-22T09:39:34 Configuring StellarSolver
2020-10-22T09:39:34 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2020-10-22T09:39:34 Starting Internal StellarSolver Sextractor. . .
2020-10-22T09:39:34 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2020-10-22T09:39:34 There should be enough RAM to load the indexes in parallel.
2020-10-22T09:39:34 Evaluating Installed RAM for inParallel Option.  Total Size of Index files: 19.7149 GB, Installed RAM: 31.2902 GB
2020-10-22T09:39:34 Image received.
2020-10-22T09:39:29 Capturing image...
2020-10-22T09:39:23 Slewing to target coordinates: RA (00h 59m 30s) DEC ( 61° 05' 18").
2020-10-22T09:39:23 Solution coordinates: RA (00h 59m 30s) DEC ( 61° 05' 18") Telescope Coordinates: RA (12h 45m 39s) DEC ( 90° 00' 00")
2020-10-22T09:39:23 WCS information updated. Images captured from this point forward shall have valid WCS.
2020-10-22T09:39:23 Solver RA (14.55265) DEC (60.97575) Orientation (0.22974) Pixel Scale (0.73429)
2020-10-22T09:39:23 Field parity: pos
 
2020-10-22T09:39:23 Field rotation angle: up is 0.229738 degrees E of N
2020-10-22T09:39:23 Pixel Scale: 0.734292"
2020-10-22T09:39:23 Field size: 55.0635 x 44.0628 arcminutes
2020-10-22T09:39:23 Field is: (-196.752, 50.8996) deg from search coords.
2020-10-22T09:39:23 Field center: (RA H:M:S, Dec D:M:S) = (00:58:12.637, +60:58:32.700).
2020-10-22T09:39:23 Field center: (RA,Dec) = (14.5527, 60.9758) deg.
2020-10-22T09:39:23 Solved with index:  4109
2020-10-22T09:39:23 Number of Matches:  11
2020-10-22T09:39:23 Solve Log Odds:  72.9335
2020-10-22T09:39:23 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2020-10-22T09:39:22 Starting Internal StellarSolver Astrometry.net based Engine. . .
2020-10-22T09:39:22 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2020-10-22T09:39:22 Scale range: 0.582152 to 0.873228 arcsec/pixel
 
2020-10-22T09:39:22 Set odds ratio to solve to 1e+9 (log = 20.7233)
 
2020-10-22T09:39:22 Configuring StellarSolver
2020-10-22T09:39:22 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2020-10-22T09:39:22 Stars Found after Filtering: 50
2020-10-22T09:39:22 Keeping just the 50 brightest stars
2020-10-22T09:39:22 Removing the stars with a/b ratios greater than 1.5
2020-10-22T09:39:22 Stars Found before Filtering: 20397
2020-10-22T09:39:18 Starting Internal StellarSolver Sextractor. . .
2020-10-22T09:39:18 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2020-10-22T09:39:18 There should be enough RAM to load the indexes in parallel.
2020-10-22T09:39:18 Evaluating Installed RAM for inParallel Option.  Total Size of Index files: 19.7149 GB, Installed RAM: 31.2902 GB
2020-10-22T09:38:29 World Coordinate System (WCS) is enabled. CCD rotation must be set either manually in the CCD driver or by solving an image before proceeding to capture any further images, otherwise the WCS information may be invalid.
2020-10-22T09:38:29 Idle.
Last edit: 3 years 5 months ago by Greg.
3 years 5 months ago #62007
Attachments:

Please Log in or Create an account to join the conversation.

  • Posts: 1309
  • Thank you received: 226
I removed a couple of the largest unnecessary indexes from my system hoping to get the total low enough for parallel loading into memory. I found it will work once. But after the first solve more ram in consumed and it states there is not enough on the next solve.
Disabling the inParallel option.
2020-10-23T23:33:37 Not enough RAM is available on this system for loading the index files you have in parallel
2020-10-23T23:33:37 Evaluating Installed RAM for inParallel Option.  Total Size of Index files: 1.53425 GB, Installed RAM: 3.56387 GB, Free RAM: 1.51933 GB

The thing is, I have 4Gb Pi4. My Conky widget (AstroPi3 script setup, Thanks Rob) is showing only about 650Mb is being used. So I should have enough RAM. Furthermore running a memory check command returns this:
free
            total        used       free          shared    buff/cache  available
Mem:        3736984      610880     1571604       70892     1554500     2912904
Swap:       3736976           0     3736976

It appears the value under free matches closely what Stellarsolver finds. However the memory check states nearly 3Gb is available. Which is closer to what Conky reports.

I hope that is the case and the available RAM can be correctly detected and used.
Thank you.
3 years 5 months ago #62036

Please Log in or Create an account to join the conversation.

  • Posts: 2876
  • Thank you received: 809
So I'm trying to err on the side of caution with loading the indexes in parallel. In the documentation, Astrometry.net just warns you to make sure you have enough RAM to load all the indexes at the same time. Earlier I had StellarSolver detect the total installed RAM and just calculate whether it can load all your indexes based upon that. However, since a user might have many programs open at the same time, that really isn't good enough especially on a PI. If you run out of memory you could get crashes and we were getting reported crashes earlier on PI's, which I think were because of this option. So I changed it to be more conservative about using this option. We will certainly keep doing testing and if it becomes apparent that we can give a little more leeway on whether to load all the indexes and it doesn't cause crashes, we can certainly do that. It is also entirely possible that it was not this option that was causing crashes and it was something else that we since fixed, but I would still lean on the side of caution and aim for stability rather than speed.
3 years 5 months ago #62037

Please Log in or Create an account to join the conversation.

  • Posts: 2876
  • Thank you received: 809
Hi Greg,

Thank you for doing some testing!

I am confused by your first issue: "BTW - when testing with the simulators I left the Solver Action as "Nothing" but the slew still occurred". I think this is a miscommunication in the interface. If you clicked the "Load and Slew" button then it is always supposed to slew to the target. The selections for the Solver action are only applicable if you click "Capture and Solve"

" after the unwanted slew, the Load and Slew button was now greyed out." So was this after it arrived at its target after the slew you mentioned? The buttons should be disabled while it is slewing, but after it gets there they should re-enable. If not, that could be a KStars bug.

" image has some problems including mystery stars". Do you mean that it detected noise as stars? This is certainly possible depending on your selected options profile. We are trying to develop better options, but they aren't quite perfected yet. Did you only try the one profile or did you try any others?

"However, this one of the same field does not solve. The main differences I can determine is the median signal of this frame is higher than the other two and the image is rotated 180deg from the others." I downloaded this file and it solved quickly. Which profile are you using to solve?


" Interesting that it says it needs 19.7GB of RAM. Is that true?" So the way astrometry's "load all indexes in parallel" option is supposed to work is that it loads all your index files into memory simultaneously, at least according to their documentation. This of course will make it much much faster than loading each index file one after the next. But if you have bigger index files than available RAM, this could be serious problem. I am not sure that it always will use that much RAM and if you have SWAP files or ZRAM, that could certainly prevent any crashes that could occur from taking up all the RAM, but I don't want to risk crashes, so in the code right now I calculate how much RAM there is and how big the indexes are and report that and disable the option if it is too much. We can play around with that to see if it really does take that much, but for now, I'm being cautious.

Thanks again for testing and let me know if you have any further questions,

Rob
3 years 5 months ago #62038

Please Log in or Create an account to join the conversation.

  • Posts: 535
  • Thank you received: 109

Just a friendly bump, the latest builds are still failing. download.copr.fedorainfracloud.org/resul.../builder-live.log.gz
make[2]: Leaving directory '/builddir/build/BUILD/stellarsolver-1.4.git'
In file included from /builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/extract.h:26,
                 from /builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/lutz.h:23,
                 from /builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/lutz.cpp:25:
/builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/lutz.cpp: In member function 'int SEP::Lutz::lutzalloc(int, int)':
/builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/sepcore.h:69:23: error: 'malloc' was not declared in this scope
   69 |   {if (!(ptr = (typ *)malloc((size_t)(nel)*sizeof(typ))))  \
      |                       ^~~~~~
/builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/lutz.cpp:63:5: note: in expansion of macro 'QMALLOC'
   63 |     QMALLOC(info, infostruct, stacksize, status);
      |     ^~~~~~~
/builddir/build/BUILD/stellarsolver-1.4.git/stellarsolver/sep/lutz.cpp:32:1: note: 'malloc' is defined in header '<cstdlib>'; did you forget to '#include <cstdlib>'?
   31 | #include <cstring>
  +++ |+#include <cstdlib>
   32 |
3 years 5 months ago #62058

Please Log in or Create an account to join the conversation.

  • Posts: 61
  • Thank you received: 10

Looks like there are two problems which prevent compiling stellarsolver atm.

Adding this into stellarsolver/sep/lutz.cpp
#include <cstdlib>

and this into stellarsolver/sep/extract.cpp
#include <cstdio>

fix the problem for me.
3 years 5 months ago #62063

Please Log in or Create an account to join the conversation.

  • Posts: 459
  • Thank you received: 69
Not sure if this is related, but I think so. The nightly has been crashing for me: after a load & slew align (which works), during focusing. I am attaching the ekosdebugger results. I'm not sure if I was monitoring the proper modules. It's unclear to me what indi process I should have monitored.

File Attachment:

File Name: indi_logs_...6-01.zip
File Size:4 KB

File Attachment:

File Name: kstars_log...3-53.zip
File Size:4 KB
3 years 5 months ago #62066
Attachments:

Please Log in or Create an account to join the conversation.

  • Posts: 152
  • Thank you received: 20
You are very welcome. It's the least I can do to help out. If only I knew C++... Another year perhaps.
Agreed that it is probably an interface communication issue. It would probably be better to have the button as "Capture & Solve", "Load & Solve" with the Solver Action section giving options "Slew To Target", "Solve Only", and maybe a checkbox for "Sync" in cases where people want the slew and sync to happen. Probably outside of the scope of this topic though - I was sharing as an FYI.
Correct - The slew occurred and failed (because I'm using sims and don't have images set up I guess). The Load and Slew button stays greyed out - only the Capture button can be activated.
Nope - that first image has some "mystery stars" in them that are not present in other images of the same object. My guess is this is a residual bulk image problem that came with switching the camera's mode? Not sure though. I guess the point is that the solver performed well in spite of these mystery stars. I was surprised it didn't confuse the solver.
I'm using whatever default was selected when I built the software with SS included the first time. Looks to be 4-ParallelSmallScale. I find it interesting that some of the images of the same object with similar characteristics solve, but some do not. Maybe these images are right at the edge of some threshold.
Gotcha. Happens to be I have 32G in this PC and I had forgotten about that detail. It's working, and it's fast enough that by the time I switch back to the Ekos window after loading an image that it's already solved. I'll have to try this out on lower memory systems soon.

Thanks again for your work on this!

Greg
3 years 5 months ago #62068

Please Log in or Create an account to join the conversation.

Time to create page: 0.767 seconds