×

INDI Library v2.0.6 is Released (02 Feb 2024)

Bi-monthly release with minor bug fixes and improvements

Issues with high acquisition rates

  • Posts: 9
  • Thank you received: 2
Thanks Jasem, your changes seem to have been effective!
I just ran tests with kstars-bleeding-2.9.2-7.2 and libindi-bleeding 1.6.2-10.2 on my Fedora 27 system:
- I can now generate sequences with thousands of exposures :-)
- The delay between exposures with my ASI290MM has been reduced to 300ms (was 500ms), which is inline with wulfblat's measurements. It does not depend much on the acquired frame size. Now and then the acquisition seem to pause for 1 or 2s (the countdown is restarted without exposures being acquired), but no big deal.
- With the simulator and tiny frames the delay is now about 100ms and very steady.
- It would be great if one would be able to specify a Delay < 1s in the sequence queue dialog (for some reason I am able to insert a comma in the widget, but not a decimal point, although my Desktop is localized with the English notation).

Now the bugs:
- Thanks to the increased throughput I was able to reproduce the annoying bug I was mentioning in my previous post, using short exposure times. So far it has only shown up while in the "Upload: Client" mode, even though I am not acquiring from a remote computer. From time to time, Ekos decides to abort a sequence after a few hundred to a few thousand exposures with the following type of error message:

2018-02-20T11:42:57 Failed to save file to /acquisition/test/Light/Red

I checked that there was still plenty of disk space left in all partitions. The bug happens with the CCD simulator too, so it is pretty easy to reproduce in a matter of 10-30min, e.g., by requesting a sequence of 10000 0.01s exposures.

- Another issue which may be more related to ASI driver is the fact that after completing (or aborting) a large sequence I often experience the following problems:
-- The KStars and Ekos windows freeze after pressing the Device "Disconnect" button, while consuming 100% CPU. The only thing I can do (after waiting a few minutes) is to kill KStars. When restarting KStars, Ekos and the INDI server, I get the message that the INDI server is already running, which is not an issue, as I am able to restart it.
- When I manage to disconnect and stop INDI, while trying to restart it, I get the following error message: "INDI server failed to start: Too many open files.", in which case the only thing it seems I can do is to restart KStars.

Thanks again for your help and sorry for the long post!
The following user(s) said Thank You: Jasem Mutlaq
Last edit: 6 years 1 week ago by Emmanuel Bertin.
6 years 1 week ago #23470

Please Log in or Create an account to join the conversation.

I've seen the failed to save file error before, I'll investigate. Btw, did you turn on "Rapid Looping" in the ASI driver or CCD Simulator? It's under Options.
6 years 1 week ago #23475

Please Log in or Create an account to join the conversation.

  • Posts: 9
  • Thank you received: 2
No I did not. I just tried it: when it is activated, in the sequence queue one single exposure is completed and then nothing more happens for both ASI and the CCD simulator. The spinning widget still turns, and I can abort the sequence, but that's it. The first time I tried it with the simulator the whole KStars crashed immediately when I started the sequence.
6 years 1 week ago #23476

Please Log in or Create an account to join the conversation.

  • Posts: 9
  • Thank you received: 16
I have started a thread on the ZWO ASI forum to address this issue: ASIGetExpStatus 250+ ms delay on Linux asking for assistance in diagnosing the delays.
The following user(s) said Thank You: Emmanuel Bertin
6 years 1 week ago #23481

Please Log in or Create an account to join the conversation.


Are you using KStars from GIT master?
6 years 1 week ago #23482

Please Log in or Create an account to join the conversation.

  • Posts: 9
  • Thank you received: 2

No, I am using kstars-bleeding-2.9.2-7.2. I suppose that the bleeding version is very close to a nightly build?
6 years 1 week ago #23483

Please Log in or Create an account to join the conversation.

To try out the feature, you'd either have to build KStars from GIT, or wait until the next release. But it looks like this feature will be limited anyway. So the premise is that as SOON as the capture is done, it immediately starts the next capture and THEN uploads/saves the image. So this would be useful IF the capture time is longer than the transfer time. So if each image is 10 seconds long, but the transfer takes 2-3 seconds, then fine. But if the capture time is 0.5 seconds, but it takes 1 second to upload the image to the client, then this would fail miserably.

But even if you're taking 5 minutes captures (like what I usually do), it can help. In my case, it takes 20 seconds to upload the image back to KStars. So with the regular method, after an image is capture AND uploaded, the next image capture begins, but with rapid looping turned on, then it begins immediately. Therefore, you can save 1 minute of time every 15 minutes of capture.
The following user(s) said Thank You: Emmanuel Bertin
6 years 1 week ago #23484

Please Log in or Create an account to join the conversation.

  • Posts: 9
  • Thank you received: 2
OK thanks, I will try with the GIT version! Transfer time with a 4MB image is significantly less than 0.5s when writing directly to an SSD, even using CFITSIO :-). Well anyway for now the limiting factor seems to be the ASIGetExpStatus() issue.
btw I hope you will be able to reproduce the "Failed to save file to" bug with large sequences in the "Upload: Client" mode. In practice this is the most annoying issue with high acquisition rates.
Thanks again!
- Emmanuel.
6 years 1 week ago #23485

Please Log in or Create an account to join the conversation.

  • Posts: 9
  • Thank you received: 16
An ASI software engineer replied to my post on the ZWO forum indicating that the long delay for for the exposure completion status to switch from working to success was expected.

This is a bit disappointing, but without access to the code I cannot guess what it is doing that is taking so long.
The following user(s) said Thank You: Emmanuel Bertin
6 years 1 week ago #23566

Please Log in or Create an account to join the conversation.

  • Posts: 9
  • Thank you received: 2
I suspected that the "Failed to save file to" in "Upload: Client" mode was due to Ekos running out of file descriptors. This seems indeed to be the case. The problem vanishes when I push the maximum number of open file descriptors to 4096 per user (was 1024 by default on my system). Sounds like a close() is missing somewhere :).
Last edit: 6 years 1 week ago by Emmanuel Bertin.
6 years 1 week ago #23585

Please Log in or Create an account to join the conversation.

Well, where is the close() missing? I know this has been reported before, but whenever a new FITS is used, the previous one is always closed first.
6 years 1 week ago #23588

Please Log in or Create an account to join the conversation.

  • Posts: 9
  • Thank you received: 16
I experimented with many short exposures while monitoring the files held open by kstars, indiserver, and indi_asi_ccd.

I saw no open file changes for indiserver or indi_asi_ccd.

For kstars after the first exposure there were 6 additional files held open: a socket, a pair of event fds, /dev/urandom, and the image file (open twice).

When I took many images the number of open files remained the same, with only the open image file changing.

However, if the number of fits viewers increases, then the number of files help open also increases -- I can only reproduce this when I trigger the rapid looping bug where the image download takes longer than the image exposure, which we know is an issue.

When you think kstars or indi is holding open too many files please get the pids for kstars, indiserver and indi_asi_ccd, and then for each please list the open files:

ls -l /proc/<pid>/fd

Where <pid> is the process pid -- you may want to redirect the output of each ls command to a file.

Then post here the results of what you've found.

Thanks!
The following user(s) said Thank You: Emmanuel Bertin
Last edit: 6 years 1 week ago by Leonard Bottleman.
6 years 1 week ago #23596

Please Log in or Create an account to join the conversation.

Time to create page: 0.389 seconds