×
INDI Library v1.8.5 Released (19 Apr 2020)

April 2020 release of INDI Library v1.8.5 introduces new drivers while providing fixes and improvements to existing devices and core framework.

scheduler: guiding aborted but capture continues

1 month 1 week ago
sterne-jaeger
Platinum Boarder
Platinum Boarder
Posts: 593
Karma: 6
More
scheduler: guiding aborted but capture continues #56082
Steve, I hope I do not get onto your nerves, but the two first links point to the same log file, right? I've analyzed it already, but do you have more of them?

TSA-120 + FSQ-85 + GSO 150/750 | Avalon Linear + M-zero | Moravian G2-8300 + ASI 1600mm pro + ASI 294mc pro | KStars/INDI on Raspberry Pi 4 with Raspbian 10

Please Log in or Create an account to join the conversation.

1 month 1 week ago
alacant
Platinum Boarder
Platinum Boarder
Posts: 581
More
Topic Author
scheduler: guiding aborted but capture continues #56097
Hi Wolfgang
You NEVER get on my nerves-. Ever!

Here are the 4 logs I have:
drive.google.com/drive/folders/1h4IL45vq...YR2PEbiW?usp=sharing

Of course, when the issue occurs, I'll post the logs here; last night was OK. Tonight I'll use the stable version again. That seems to throw up the issue.

Oh, and I see the git has some guiding files changed... Could there be a clue there? But I'll stick to my stable install for noe.

Cheers and TIA,
Steve

ubuntu 18.04
700d, eq6, t7m
The following user(s) said Thank You sterne-jaeger

Please Log in or Create an account to join the conversation.

1 month 1 week ago
sterne-jaeger
Platinum Boarder
Platinum Boarder
Posts: 593
Karma: 6
More
scheduler: guiding aborted but capture continues #56101
Hi Steve,
thanks, now I could track it down, all cases show the same behavior. The problem starts always with a dithering error resulting in the following behavior:
  1. Capture module recognizes a guiding error and aborts the capturing sequence
  2. Scheduler recognizes the capture failure and tries to recover
  3. Scheduler restarts guiding
  4. Guide issues the warning "PHD2: Guiding is already running."
  5. Capture and Scheduler recognize that guiding has restarted
  6. Scheduler starts the capture sequence
  7. Guide reports a problem "Unknown PHD2 event: ConfigurationChange", reports "PHD2: Guiding Stopped" and changes its state to "Autoguiding aborted"
  8. Capture starts capturing

#4 and #7 are strange, I do not have an explanation yet. But it might be that another problem appears that leads to the situation. Dithering is failing because of a lost star event. But the log shows an oscillation between "PHD2: Star Lost. Trying to reacquire." and "PHD2: Star found, guiding resumed." This might be the root cause.

I have no clue whether this is a problem that results from a recent code change or whether this is a problem that already exists longer. Indeed, I did a change to the PHD2 module in April fixing a guiding problem. Maybe it has a undetected side effect.

My next step will be trying to reproduce it.

If you have the time, you could do me a favor and try a very recent build, since I refactored the Capture module to make it more robust. It would be interesting for me if this latest change also shows this behavior.

Cheers
Wolfgang

TSA-120 + FSQ-85 + GSO 150/750 | Avalon Linear + M-zero | Moravian G2-8300 + ASI 1600mm pro + ASI 294mc pro | KStars/INDI on Raspberry Pi 4 with Raspbian 10
The following user(s) said Thank You alacant

Please Log in or Create an account to join the conversation.

1 month 1 week ago
alacant
Platinum Boarder
Platinum Boarder
Posts: 581
More
Topic Author
scheduler: guiding aborted but capture continues #56103
Hi Wolfgang

you could do me a favor and try a very recent build,


No problem. Unfortunately it's cloudy tonight. I'll use the time to build the latest from git and hope I can reproduce the issue.
Cheers

ubuntu 18.04
700d, eq6, t7m

Please Log in or Create an account to join the conversation.

1 month 1 week ago
sterne-jaeger
Platinum Boarder
Platinum Boarder
Posts: 593
Karma: 6
More
scheduler: guiding aborted but capture continues #56107
Hi Steve,
there is a unusual thing in your PHD log. It looks like dithering provokes sometimes a star lost event.

The weirdest thing happens around 2020-06-23T02:11:25, one of the events where guiding fails and capturing continues. PHD2 first reports "Settling failed" but before it reports guiding steps. This confuses the Ekos guiding, since it receives one guiding step AFTER PHD2 reports "Settling failed".

This is a concurrency issue of PHD2. I know I fixed another concurrency issue in April, so maybe this is a similar case.

I am quite sure that there is something special in your PHD2 setup that leads to these "star lost" events although the sky looks good. If you take a look at the PHD2 log for that event above, there is sharp spike in the guiding star mass. I guess this does not come from clouds but has a technical source.

So please investigate that. What is the reason that dithering leads sometimes to a lost star event? Maybe the focus is too sharp, maybe you need to change settings for guiding star selection, ...

In parallel, I will check this concurrency issue.

Good night
Wolfgang

TSA-120 + FSQ-85 + GSO 150/750 | Avalon Linear + M-zero | Moravian G2-8300 + ASI 1600mm pro + ASI 294mc pro | KStars/INDI on Raspberry Pi 4 with Raspbian 10
The following user(s) said Thank You alacant

Please Log in or Create an account to join the conversation.

1 month 1 week ago 1 month 1 week ago by alacant.
alacant
Platinum Boarder
Platinum Boarder
Posts: 581
More
Topic Author
scheduler: guiding aborted but capture continues #56115
Thanks again Wolfgang

OK. I have the latest git built for tonight. Forecast is clear but perhaps best not to mention that...

I have sent a query to the PHD2 Devs and mentioned INDI, so hoping to avoid a deluge of windows related stuff;)
My first thought was -and I've a feeling the PHD2 guys are gonna say the same- to cancel star mass detection in PHD2. It's a simple toggle in the advanced configuration dialogue. But then we wouldn't know if it had had any effect. Yes or no on that one?

The PHD2 thread is here:
groups.google.com/forum/?utm_medium=emai...xnAR-1k/cV0wxwEPAwAJ
but of course I'll report any activity here too.

Cheers,
Steve

ubuntu 18.04
700d, eq6, t7m

Please Log in or Create an account to join the conversation.

1 month 1 week ago
DerPit
Platinum Boarder
Platinum Boarder
Posts: 446
Karma: 1
More
scheduler: guiding aborted but capture continues #56116
Just two remarks, maybe they add some info:
I sometimes had issues with PHD2 when using subframe readout, it would - for one frame - get a blank image and choke on that.
Also, variable seeing had triggered a 'Star mass changed' error one too often, so I'm only using star profile now. No guiding issues with PHD2 on my side. But I'm only using the capture module, not the scheduler. I'm running git versions, mostly up to date ones (at most 1 week old at times).

openSUSE Tumbleweed KStars git INDI git
GPDX+EQMOD, CEM60EC, ASI1600+EFW+EAF+ASI290 mini
The following user(s) said Thank You alacant

Please Log in or Create an account to join the conversation.

1 month 1 week ago
alacant
Platinum Boarder
Platinum Boarder
Posts: 581
More
Topic Author
scheduler: guiding aborted but capture continues #56118
@DerPit
Hi and thanks for your message.
Question: If you're not using the scheduler, does capture alone recover from a guiding failure?
TIA,
Steve

ubuntu 18.04
700d, eq6, t7m

Please Log in or Create an account to join the conversation.

1 month 1 week ago
DerPit
Platinum Boarder
Platinum Boarder
Posts: 446
Karma: 1
More
scheduler: guiding aborted but capture continues #56120
No, it'll abort, and sit there :(
So not a solution to your problem in itself, but maybe a way to make PHD2 not fail guiding. The only guiding failures I had in the last months were when I was tracking a target until it vanished behind a building.....

openSUSE Tumbleweed KStars git INDI git
GPDX+EQMOD, CEM60EC, ASI1600+EFW+EAF+ASI290 mini
The following user(s) said Thank You alacant

Please Log in or Create an account to join the conversation.

1 month 1 week ago
sterne-jaeger
Platinum Boarder
Platinum Boarder
Posts: 593
Karma: 6
More
scheduler: guiding aborted but capture continues #56121
I also use PHD2 for guiding and it is running smoothly. I always use the scheduler, mostly with at least two jobs per night.

We have put a lot of work into making the scheduler more robust, but it is tricky. Therefore I would always recommend using the scheduler when running unattended.

Capture itself handles guiding deviations, i.e. it aborts a capture if the deviation is too high and restarts if it is back in the range. As soon as an error occurs - i.e. star lost timeout, it stops. Here the scheduler comes into the game. The scheduler recognises the situation and either starts the next job or restarts the job - depending on the settings.

Wolfgang

TSA-120 + FSQ-85 + GSO 150/750 | Avalon Linear + M-zero | Moravian G2-8300 + ASI 1600mm pro + ASI 294mc pro | KStars/INDI on Raspberry Pi 4 with Raspbian 10

Please Log in or Create an account to join the conversation.

1 month 1 week ago 1 month 1 week ago by alacant.
alacant
Platinum Boarder
Platinum Boarder
Posts: 581
More
Topic Author
scheduler: guiding aborted but capture continues #56144
Hi Wolfgang, everyone

Here is the esl and log with the latest git. Guiding failed at around 04:40 and there is a dangerous issue that the scheduler allowed tracking after sunrise without parking the mount as requested. I've started a new thread on that:
www.indilib.org/forum/ekos/7250-latest-g...t-sunrise.html#56143

Here are the logs anyway:
log: drive.google.com/drive/folders/1vg9JM_c1...EmVpcLXQ?usp=sharing

Cheers and clear skies,
Steve

ubuntu 18.04
700d, eq6, t7m

Please Log in or Create an account to join the conversation.

1 month 1 week ago
alacant
Platinum Boarder
Platinum Boarder
Posts: 581
More
Topic Author
scheduler: guiding aborted but capture continues #56146
Hi
Here is the reply to the dither question from the PHD2 devs:

The settle failure at 02:11 actually has a very simple explanation. The dither settle timeout of 60s is too short given the time the mount actually needs to settle after large dithers.

Here are the details of what we see in the log:

A dither request came in to PHD2 at 02:10:25.298, requesting a 6.0 pixel dither with settling criteria of less than 3.0 pixels error for at least 18 seconds, with a timeout after 60 seconds. The 60-second timeout meant that the deadline for settling to complete is 02:11:25.298.
In the camera frames up to 02:11:07.492 the guide error was > 3 pixels.
At 02:11:09.498 the guide error dropped below the 3.0 pixel threshold. It needs to stay below the 3.0 px threshold for 18s to meet the requested settling criteria.
The guide distance continued to stay below the 3.0 px threshold until phd2 handled a camera frame at 02:11:25.485. The gude error had stayed below the 3.0 px threshold for just under 16 seconds, not quite to the 18s required, however the deadline for settling had passed, and phd2 reported that settling had failed -- indeed it had failed to settle within the 60s time limit.

There's an easy fix for this: update your settle timeout to a larger value. I would not hesitate to set it to a much larger value, like 300s or even 600s.


Wolfgang: I can change the settings but does this help the capture starting without guiding issue? I don't think so.
Cheers,
Steve

ubuntu 18.04
700d, eq6, t7m

Please Log in or Create an account to join the conversation.

Time to create page: 1.052 seconds