×

INDI Library v1.9.8 Released (29 Sep 2022)

Bi-monthly INDI Library released with new drivers and bug fixes.

Astro-TIFF support, better use of the TIFF format.

  • Posts: 329
  • Thank you received: 88
There is currently an initiative to make better use of the TIFF format for astronomy.

The idea is to save a FITS header in a standard way into a TIFF file. This makes conversion between TIFF and FITS possible without loosing header information.

The FITS header is stored in the description tag of the TIFF file.  The specification is here: www.hnsky.org/astro-tiff.htm  Benefits:
  • The TIFF compression algorithm makes the files smaller
  • The files are readable by almost any image viewer.
The standard way is intended to be used for light, dark, flat and flat-darks. These files are normally written in large quantities and compression and viewability makes sense. Already some application developers are working on support. See the list at end of the specification in above webpage. For APP the only requirement would be to read TIFF and decode the tag description containing the header information in a string and decode information like date, exposure, gain, sensor temperature.The specification is not set up as a replacement of FITS. It just propose better of use of the existing TIFF file format. 
Han
The following user(s) said Thank You: Jasem Mutlaq, lock, AstroMuni
7 months 2 weeks ago #80683

Please Log in or Create an account to join the conversation.

Thank you Han for sharing this. This could be implemented at INDI level, is there a sample code that shows how this is done?
Jasem Mutlaq
Support INDI & Ekos; Get StellarMate Astrophotography Gadget.
How to Submit Logs when you have problems?
Add your observatory info
7 months 2 weeks ago #80684

Please Log in or Create an account to join the conversation.

  • Posts: 344
  • Thank you received: 68
OK, this may sound cynical. But I have always avoided TIFF as a file format.

I don't know why, but I seem to come across many applications that are supposed to support TIFF format images, but more often than not, when I try to open a TIFF file in these applications I have found that I get a message such as "format not recognised", "invalid file" and similar.

I don't know why this happens, it is just a vague anecdotal experience of mine, which has led me to avoid using the format for as long as I can remember.

However, I guess FITS files are just as unwieldy, and any non-lossy compression in a file format would be a good thing, I just hope that any implementation of the format is portable to other softwares.
__
I blog my progress: www.coochey.net
7 months 2 weeks ago #80685

Please Log in or Create an account to join the conversation.

  • Posts: 52
  • Thank you received: 5
I upvote this feature.
We have integrated this in the coming 1.0.0 release of Siril.
7 months 2 weeks ago #80691

Please Log in or Create an account to join the conversation.

  • Posts: 329
  • Thank you received: 88
It is only the filling of a standard TIFF description tag in a structured way with date, gain, exposure time, sensor temperature, used filter.....

Standard libraries like www.libtiff.org support the tag Description. The Description is just a string you could fill with the FITS header except that each line is ends with a LF or CR+LF as you like. LibbTiff has only LZW compression which performs a little less then Deflate. (LZW is used in GIF, Deflate in PNG)

Personally I'm using a Pascal library. Code for importing was simple since the FITS header is a simple structure. Filling the Description tag was only a few lines of code. For Ekos mainly writing Astro-TIFF is interesting. With the compression method Deflate (same like ZIP) achieves almost the standard FITS compression factor called RICE from the CFITSIO utilities FPACK  and FUNPACK leaving about 60% of the file size.


TIFF is a super flexible format. I don't think there are any compatibility problems since it is already a few decennia’s available. Except for FITS it is the only generic format which allows 32 and 64 bit in floating point images. But the intended usage if for 16 bit lights, darks, flat and flat-darks. Astro imagers produce these files in the thousands. The file size reduces to about 60% (use grey level format) of the original size and you can use standard image managers to organise and view your images. So the intention is just to use the power of this existing and established TIFF format.

Siril version 1.0 to be released soon will accept Astro-TIFF for stacking. CCDCiel has fully implemented Astro-TIFF reading and writing in the code. The next ASTAP program to be released today or tomorrow will fully support it including batch conversion both ways.

Han
The following user(s) said Thank You: Alfred
Last edit: 7 months 2 weeks ago by han.
7 months 2 weeks ago #80692

Please Log in or Create an account to join the conversation.

  • Posts: 52
  • Thank you received: 5
Han, Libtiff is supporting Deflate. I'm using it with Siril.
7 months 2 weeks ago #80698

Please Log in or Create an account to join the conversation.

  • Posts: 329
  • Thank you received: 88
Good. I looked it briefly up for somebody and could only find: Compression = 32946 PKZIP-style Deflate encoding (experimental)

which accordiing Wikpedia is different the compression type=8 Deflate. But the documentations for the coverage hasn't been updated since 2002. :)

Update, reading through the documentation it is explained. So the libtiff online documentation is not updated:

Adobe Photoshop Technical Notes March 22, 2002

proprietary ZIP/Flate compression code (0x80b2) has been used by some software vendors. This code
should be considered obsolete. The compression used by the obsolete code is identical to that defined
above. We recommend that TIFF implementations recognize and read the obsolete code, but only write the
official compression code (8).
Last edit: 7 months 2 weeks ago by han.
7 months 2 weeks ago #80699

Please Log in or Create an account to join the conversation.

  • Posts: 52
  • Thank you received: 5
Indeed I use COMPRESSION_ADOBE_DEFLATE
7 months 2 weeks ago #80711

Please Log in or Create an account to join the conversation.

  • Posts: 329
  • Thank you received: 88
INDIGO has implemented Astro-TIFF in their code.
Last edit: 6 months 2 weeks ago by han.
6 months 2 weeks ago #81444

Please Log in or Create an account to join the conversation.

I finally managed to pave the way for this in INDI v1.9.5 by defining INDI_TRANSFER_FORMAT and INDI_CAPTURE_FORMAT. We should have support for this in the next release. Now if anyone willing to submit a PR for this, it would be awesome! :D
Jasem Mutlaq
Support INDI & Ekos; Get StellarMate Astrophotography Gadget.
How to Submit Logs when you have problems?
Add your observatory info
6 months 2 weeks ago #81477

Please Log in or Create an account to join the conversation.

  • Posts: 143
  • Thank you received: 13
In the 1.9.5 release documentation I see that this is indeed implemented. What does that mean for support for the Astro-TIFF format? Does that require a further update to INDI and/or Ekos?
Willem Jan Drijfhout
astroworldcreations.com
6 months 2 weeks ago #81551

Please Log in or Create an account to join the conversation.

  • Posts: 26
  • Thank you received: 12
I tried this last night because I use CCDCiel for capture and Siril for processing. I am impressed with the long-term storage savings, but something isn't right yet. I'm quite used to using the EXPTIME header variable within my scripts to name my stacked files. When processing in Siril 1.0.0 with Astro-TIFF files, the EXPTIME is not a sum of all exposures, instead it is the length of a single sub-exposure. This isn't right... I haven't looked to see what other values might also not be correct. I'm unsure if this problem is in Siril or a formatting problem in CCDCiel.

Within CCDCiel, it's worth noting that there's a significant processing hit for adding compression. This isn't a problem for long exposures because you can configure CCDCiel to start the next exposure before the previous exposure is done being saved. When shooting something quick like bias or flat frames, however, it will take extra time between images. I use a Raspberry Pi... so this time can add up.

It would be nice to continue to capture regular FITS files and then use a command line tool to convert and compress the files to Astro-TIFF. Does such a tool exist yet?
The following user(s) said Thank You: Alfred, David Swinnard
6 months 1 week ago #81797

Please Log in or Create an account to join the conversation.

Time to create page: 1.014 seconds