So here are all of the columns that sextractor can export

sextractor.readthedocs.io/en/latest/Param.html

In my program, I ask for x image, y image, mag auto, flux auto, flux max, cxx image, cyy image, and cxy image. Also I optionally request hfr

The XYLS and AXY files don't contains enough information. Adding info height, width, CDELT2 and RA,DEC in the header would be easy. The extension in not standard.

Disadvantage all FITS viewers ( DS9, AstroImageJ AvisFV) except FV can't display FITS files with tables.

Adding info as command line parameters would be inefficient. It is also possible to export/import all the information as .csv file. Adding some extra information required like image height, width, CDELT2 and RA,DEC would be easy. This wil make the information accessible for any spreadsheet. The only disadvantage would be values as text and not binary.

An other option would be to save the X,Y table information as an FITS image. Dimensions 2 x 500+ pixels (floats) or if you add the flux and noise level 4 x 500+ pixels. That would be very very easy to implement in ASTAP. There is no reason the process the information as FITS table. A table or image is for me both an data array. The other advantage would that the data is stored binary, 4 bytes per pixel.

So I see three options:

1) Passing the X,Y and flux information as FITS image. Very easy to implement in ASTAP.
2) csv format easy to implement. Accessible by spreadsheets. Disadvantage it is text. Adding some extra information required like image height, width, CDELT2 and RA,DEC will be easy.
3) FITS table . Accessibility by other tools is restricted to FV and Astrometry.net. Image info: height, width, CDELT2 and RA,DEC keywords should be added for ASTAP.

I have a little preference for option 1. Much less coding required then the other options.

Han

x, y and mag_auto will do for solving

Looking to the topic again this morning, probably the best option would be to allow your program export both the image and SExtractor detected object info in a FITS file containing two HDU's (Header Data unit) . The first HDU containing the image and the second HDU containing a table with x image, y image, mag auto, flux auto, flux max, cxx image, cyy. So a purpose which goes beyond astrometric (plate) solving.

This exported FITS could be picked up by other programs (by ASTAP) to display detected galaxies, stars or simple plate solving . In ASTAP they could compared against HyperLeda galaxy catalog or Gaia catalog.

Alternatively the object detection's could be saved in the FITS header using the keyword "ANNOTATE". That's done in ASTAP done automatic for asteroids, comets or manually. Also AstroimageJ saves annotation using the keyword ANNOTATE but in a different undocumented format.

Exporting detected stars by ASTAP in XYLS format doesn't really bring much except for testing. Without finding a real purpose I probably remove it again.

Han

I have completed the FITS table reader in ASTAP. This version is not released yet but It can read any binary FITS table format. As said before, my assumption is that any SEextractor table would be best combined with an image in the primary HDU:

Han

Here a FITS test image with a X, Y, magnitude table attached. Created with FV:

ufile.io/pjcyg8lj


Han

So, I think I have resolved the issues I was having with my parallel solvers. I am very pleased with the results! I can now blind solve jpeg images which were practically unsolvable before in seconds. It works with the internal and external astrometry.net versions. But it does work best with the internal solver.

There could be a couple of reasons to want to export the XYLS from ASTAP.
1. To use ASTAP as a alternate sextractor
2. To see what stars in your image ASTAP used for solving
3. To get back the list of objects in the image so that you could load the RA and Dec of each object without having to use sextractor

Originally I wanted to use it for #3. However, I have thought about this and realized there is a mistake in my logic. I realized it yesterday when I was working on the parallel solvers and I was looking at the stars in the image after a solve. I realized that the list of stars used for solving is not the full list of stars in the image. Of course I knew this before, but what I didn't think about was that when you sextract for solving purposes you want to use different settings than you do when you want to load information about the objects in the image. So really, to get the information about all the objects in the image, what I should really do is sextract with options for solving, solve the image, then sextract again with options for the objects we actually want to detect, and use those stars and the already loaded WCS data to get load the RA and DEC of the objects in that list.

Now, about #1, I am not sure that we would want to use ASTAP as an alternative to sextractor because you would have to solve the image first and the options are much more limited.

So yes that leaves option 2, which basically means as you said, it's for diagnostic purposes. So yes, I understand where you are coming from.

In reference to your our discussion about taking results from sextractor into ASTAP,

So the output of sextractor is just a binary fits table (It can also do ASCII, but astrometry.net didn't want to read those files). The column headings are very specific just to let you know. For example, the X position of a star has the heading X_IMAGE. And likewise for the other headings that I mentioned to you before.

I can certainly combine the data returned by sextractor with the data currently in the image buffer and save that as a FITS file with the appropriate FITS headers as you mentioned. But that would mean that a user would have to use my program in order to produce a file acceptable for ASTAP, since this would be a brand new format basically.

Part of the appeal of reading in an XYLS file and the options the way it works in astrometry.net would be the flexibility. A user could use sextractor, SEP, or maybe even astrometry.net to get the star info from an image. Then they could send that information directly to ASTAP to solve. They wouldn't need my program to do this because the format would already be compatible.

Obviously, ultimately, whichever way you want to accept the information into ASTAP would be fine with me, since I'm just making an interface for all these solvers and I've already had to make a number of adaptations to accommodate the differences between them. I definitely do understand the appeal of having the image data come along with the table of information. So, I can do it your way if you like.

Good news that you have parallel processing working. I'm curious how good it will perform.

The possibility to read FITS tables will stay in ASTAP. It works very flexible and the setup will not slow down the other processes. I will present the data nicely in a grid style interface including copy paste facility. It will read AXY or XYLS or any other FITS table. i like to use it also to import SExtrator data. I'm impressed by the SExtrator performance and it will most likely find new ways to use it. In a FITS table each column has name so no standard is required for storing the data. Using the correct columns can be automated. So having the option of export SExtrator data as a second HDU in FITS image file is on my wish list.

The ASTAP export to XYLS I will keep for the moment. I will see if i can sort the list on SNR (or add it) and see how Astrometry.net can process it.

Han

Good news, I have just released version 1.1 with the ability to solve fields in parallel. Depending on your computer, this could make solving very fast. On my MacBook with 16 cores, I'm getting solving times on some images in just a couple of seconds, whereas before it took minutes to solve these images if they solved at all. Note that if the position and scale are pretty well known, single core solves are still better. 0.3 seconds compared to 0.6 on my computer, but the Parallel solve really shines with the blind solves.

github.com/rlancaste/sexysolver-tester/releases/tag/1.1

Please keep testing. I think pretty soon, the Library will be ready to be integrated into kstars.

First feedback Windows version:

1) This new version has problems to solve H-alpha long exposurse of 200 sec. (I think the previous version also) E.g this image. No solution after 500 seconds:

ufile.io/2mzia5e3

Maybe it is matter of settings. Selecting 11000 objects! Filtering keep 500 stars doesn't work. Filtering Cut dim doesn't work.

Solving short exposures works.


2) Selecting external extractor without having one doesn't give warning.


3) Aborting process is problematic. Takes long time