×
INDI Library v1.8.5 Released (19 Apr 2020)

April 2020 release of INDI Library v1.8.5 introduces new drivers while providing fixes and improvements to existing devices and core framework.

New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed

2 months 2 weeks ago
rlancaste
Platinum Boarder
Platinum Boarder
Posts: 2264
Karma: 22
More
Topic Author
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #52637
For the last couple of months I have been playing around with an idea. First my idea was to combine Sextractor and Astrometry.net to avoid using python. That worked very well and so I build it in as an option in KStars. Not only did it work without python, it was often faster than traditional astrometry solving. But there are many parameters. So I started trying to explore and play around with the parameters and found it easier to do in a separate testing program instead of in KStars so that I could really focus on the problem. Then after I got it up and running, I got a little bolder. I thought it might be a cool idea if we could make astrometry.net into a library and get it natively running inside an Internal Library in KStars on Windows Mac and Linux, instead of an external set of programs that only works on certain operating systems all kludged together. So over the last couple of months, I have done exactly that. I don't believe it is done yet, but I think we are ready for some rigorous testing to start evaluating how well it works, try to make it better, figure out what parameters are effective and which ones are useless, and see what parameters should be added. Right now, for some images, the SexySolver is faster than any of the other methods, but for others it is not. It should by definition be faster because it doesn't have to save any temporary files along the way, nor does it need to call any external programs. So I think if we can fix this up, it should be much more efficient.

Here is my project: github.com/rlancaste/sexysolver-tester



To try this out, you don't need any programming knowledge, you just need some astronomy photos, a computer on which astrometry.net index files have been downloaded, and if you want to compare to external astrometry.net, sextractor, astrometry.net, or ANSVR, they would need to be installed, but that is optional. You just need to download the solver tester program and start trying it out.

Here is the download link to the Windows and Mac installers for the latest release version:
github.com/rlancaste/sexysolver-tester/releases/tag/0.8

What kind of feedback do I need? Well several things:
- Which parameters seem to help solve images faster and more accurately?
- For sextracting, which parameters really seem to help make the source extraction better?
- Which parameters don't seem to help too much?
- Which parameters should not be exposed to the users because they could mess up the solve?
- What additional parameters would we need to add?

For any programmers out there:
- If you can take a look at my code and help optimize it in any ways that would help
- Any Sextractor, SEP, or astrometry.net experts could look at my SexySolver code to see if it is accurate or can be improved
- I had to make a bunch of changes to astrometry to get it to work on Windows. It mostly works now on windows, but if there is any way to make my changes better, that would help.

Once we get this refined, it should be able to greatly enhance Plate Solving, Guiding, and Focusing algorithms and could be used to support other cross platform QT based programs that could run on Astrometry.

Thanks for your help,

Rob
Attachments:
The following user(s) said Thank You knro, Kaczorek, wvreeven, sterne-jaeger, pug916, starman345, DerPit, lantictac

Please Log in or Create an account to join the conversation.

2 months 2 weeks ago
wvreeven
Platinum Boarder
Platinum Boarder
Posts: 1551
Karma: 7
More
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #52643
Very nice Rob! Thanks very much for doing this. I tried it out on an image of Proxima Centauri that I took tonight. I noticed that when using the internal astrometry.net solver and aborting it, the astrometry.net process is not stopped. I had to kill it by hand. And as I have noticed with plate solving in KStars before (both with sextractor and astrometry.net), NOT using the scale is MUCH, MUCH faster than when using the scale.

Wouter

ASI6200 and filter wheel with a SkyWatcher Esprit 80 ED on a SkyWatcher HEQ5-Pro
ASI1600MM-Pro Cooled and filter wheel with an 8" TS Ritchey-Chrétien on a SkyWatcher EQ6-R

Please Log in or Create an account to join the conversation.

2 months 2 weeks ago
rlancaste
Platinum Boarder
Platinum Boarder
Posts: 2264
Karma: 22
More
Topic Author
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #52666
Thanks for trying it, and please try it some more. The more info we get the better.

For cancelling the internal solver, astrometry.net has a method that monitors for the creation of a “cancel” file. That is what I implemented here instead of trying to just kill the thread. What I read is that this is a more graceful of stopping the search and can avoid some side effects that could occur if you just kill it. Note that it doesn’t stop immediately when you do it this way, it sometimes takes a few seconds. If that way doesn’t work well I can try another way, but it said this way was better.

For the scale, are you sure your scale numbers in the image are right? I think it makes it go faster when the scale is right but makes it take much longer if the scale is wrong. I have some images that I took where the scale is right and some where it was wrong that I took with KStars. I don’t know what I did differently when I took them, I haven’t figured it out yet.

Thanks,

Rob

Please Log in or Create an account to join the conversation.

2 months 2 weeks ago
wvreeven
Platinum Boarder
Platinum Boarder
Posts: 1551
Karma: 7
More
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #52673
Rob, the astrometry.net processes (two of them because I tried twice and aborted both) kept on running for minutes and didn't stop even when I quit SexySolver. I noticed because they both were using 99% CPU time and the fans of my macBook started to kick in. I had to kill the processes manually.

As for the scale, I noticed several months or possibly over a year ago that astrometry.net is a lot faster without using scale even if the scale is more or less right. More or less in this case meaning perhaps a few arc minutes off on a scale of 2.5 degrees. Ever since this is the first thing I disable when I (re)install KStars and Ekos. I'll try with a more or less scale again and will let you know the results.


Wouter

ASI6200 and filter wheel with a SkyWatcher Esprit 80 ED on a SkyWatcher HEQ5-Pro
ASI1600MM-Pro Cooled and filter wheel with an 8" TS Ritchey-Chrétien on a SkyWatcher EQ6-R

Please Log in or Create an account to join the conversation.

2 months 2 weeks ago
rlancaste
Platinum Boarder
Platinum Boarder
Posts: 2264
Karma: 22
More
Topic Author
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #52678
Oh I might have misunderstood what you mean when you said "internal astrometry.net solver". Do you mean the "Internal SexySolver" Setting in SexySolver or one of the other settings using the KStars "internal" copy of astrometry.net? I had thought you meant the internal SexySolver.

Because the method that uses the internal sexysolver is actually my library version of Astrometry.net built into SexySolver and if you quit the program it can't still be running because it is actually a separate thread in the program itself not a separate process, so if you quit the SexySolver program, it can't still be running. But if you run any of the other solvers in the list, they start an external process. That external process is called "astrometry-engine" On a Mac, typically they will use the KStars "internal" build of astrometry.net, since that is already on your computer, but it can also use homebrew if you have that. But if you see a process called astrometry-engine, that is the "external" solver in SexySolver.

Sorry about the confusion. If you mean that the external astrometry.net solver did not stop running when you hit abort, then that is a different issue and I will definitely look into that. If it doesn't stop, then it might not stop in KStars either since I used the same method to stop that one, a kill command. Maybe we should use both the kill command and and cancel file.

Please Log in or Create an account to join the conversation.

2 months 2 weeks ago
rlancaste
Platinum Boarder
Platinum Boarder
Posts: 2264
Karma: 22
More
Topic Author
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #52685
One trick you could try would be to solve it once, note the scale, and enter a range that includes that

Please Log in or Create an account to join the conversation.

2 months 2 weeks ago
wvreeven
Platinum Boarder
Platinum Boarder
Posts: 1551
Karma: 7
More
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #52686
Sorry, my messages have not been clear. What I meant is:
- the internal astrometry.net solver keeps running in the background when I cancel it in the SexySolver app, even when I quit SexySolver. This is not the case in Ekos.
- I do not use scale in Ekos nor in SexySolver independently of my using internal astrometry.net or sextractor. I never use the online astrometry.net

I have tried to reproduce astrometry.net still running but can't at the moment. I'll try later again.

ASI6200 and filter wheel with a SkyWatcher Esprit 80 ED on a SkyWatcher HEQ5-Pro
ASI1600MM-Pro Cooled and filter wheel with an 8" TS Ritchey-Chrétien on a SkyWatcher EQ6-R

Please Log in or Create an account to join the conversation.

2 months 2 weeks ago
RDBeck
Expert Boarder
Expert Boarder
Posts: 143
More
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #52702
This seems to be a lot faster than the internal solver for me. The slowest of 14 solutions took 3.5s and the quickest took 0.9s. The average time was 1.3s, and the median time was 1s. All times rounded to the nearest 1/10th second.

I have attached the results table for information.

File Attachment:

File Name: Sexy_Solve..._RDB.zip
File Size:1 KB

Hypertuned NEQ-6Pro, ZWO OAG, ASI 120 min guide camera, SV 80ST, ASI 1600MM Pro, ASI filters and filter wheel, DIY Moonlite-compatible focuser, DIY SQM
Attachments:
The following user(s) said Thank You pug916

Please Log in or Create an account to join the conversation.

2 months 2 weeks ago
rlancaste
Platinum Boarder
Platinum Boarder
Posts: 2264
Karma: 22
More
Topic Author
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #52704
Thank you very much for the results. Did you try tweaking any settings to see which ones would make it solve faster?

Please Log in or Create an account to join the conversation.

2 months 2 weeks ago
RDBeck
Expert Boarder
Expert Boarder
Posts: 143
More
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #52705
I just loaded and solved the images without any adjustments. The images were all fits files captured in EKOS. The test was essentially what I would use for a "Load and Slew".

Hypertuned NEQ-6Pro, ZWO OAG, ASI 120 min guide camera, SV 80ST, ASI 1600MM Pro, ASI filters and filter wheel, DIY Moonlite-compatible focuser, DIY SQM

Please Log in or Create an account to join the conversation.

2 months 2 weeks ago
rlancaste
Platinum Boarder
Platinum Boarder
Posts: 2264
Karma: 22
More
Topic Author
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #52707
So with this program you can simulate the exact same setup you use in Ekos with load and slew so that we can compare the times. I am assuming you do not have the "use sextractor" checkbox checked in KStars? If that's true, you can select the Classic Astrometry.net from the combo box next to the solve button so that you can compare the time. If you do have that box checked, then it is a different option in the combo box, the "Ext Sextractor, Ext Solver" option. That way we can see the difference.

I think due the slight variation that we get in solving times with the same settings, it would be good to have an option to run a certain number of trials and calculate the average time instead of having to do each one and then figure it out. I will add that option

Please Log in or Create an account to join the conversation.

2 months 2 weeks ago
RDBeck
Expert Boarder
Expert Boarder
Posts: 143
More
New Internal Solver for Mac, Windows, and Linux -- Testing/ Experiments needed #52710
To the best of my knowledge, I don't have the use sextractor box ticked. I re-ran the solver with Classic Astrometry.net for the same targets. I have both sets of information from SexySolver as csv files and an ods file with both in the attached zip file. The ODS file is sorted by RA and then DEC, and it becomes obvious that the internal solver was run on the same target twice.

File Attachment:

File Name: Sexy_Solve...0425.zip
File Size:22 KB


All solutions were performed on my "pierside" computer, an Intel NUC i5-8259U processor with 16GB RAM running Ubuntu 19.10 64-bit. The classic solver took 5-10x the time of the internal solver.

Hypertuned NEQ-6Pro, ZWO OAG, ASI 120 min guide camera, SV 80ST, ASI 1600MM Pro, ASI filters and filter wheel, DIY Moonlite-compatible focuser, DIY SQM
Attachments:

Please Log in or Create an account to join the conversation.

Time to create page: 0.707 seconds