×

INDI Library v2.0.6 is Released (02 Feb 2024)

Bi-monthly release with minor bug fixes and improvements

Polar Alignment tool not waiting for mount to stop

  • Posts: 25
  • Thank you received: 8
I'm unable to use the polar alignment tool because the second exposure is always taken before the mount stops moving. The initial exposure is fine, but the second is always just streaks. I'm still working on getting the development environment working so I can debug this kind of thing, but after looking at the code I can see two cases where this could happen:

1. It may be that there is a race condition if the telescope's "isSlewing()" doesn't return true immediately after a "slew()" command is issued. If this was true the alignment state would transition directly to taking the second image without waiting. This seems unlikely since the Telescope::Slew state is set immediately in Ekos and then sent to the "SLEW" switch. If the switch was somehow

2. There isn't any error checking on the Slew() result, so it's possible that an error condition while requesting the slew happened silently. There are a lot of places that return "false" without any error logging. However, the slew does actually happen which is why I'm seeing streaks so this might not be it.

I haven't tried running any of the other lx200 drivers yet. I'm using a Losmandy Gemini 2 plugged into an RPi running indiserver. It's all connected over wired Ethernet to the Ekos VM running in VirtualBox on a Mac. I tried to make it as complicated as possible!

Any debugging ideas would be welcome.
8 years 7 months ago #4686

Please Log in or Create an account to join the conversation.

  • Posts: 25
  • Thank you received: 8
I spent a few hours debugging this last night after getting kstars building in a debug environment. For some reason the problem showed up this time with the process getting stuck, seemingly waiting for the mount to make the 30' move - only a kstars restart fixes it. Both issues are caused by the same bug and it does appear to be a race condition, but I haven't been able to clearly identify the problem or a fix.

I added logging to the align module and ran the same process through my Gemini 2 and the Telescope Simulator. The problem seems to be caused by the state machine (i.e. azStage) assuming it will continue to get frequent EQUATORIAL_EOD_COORD updates after issuing the Sync() and Slew() commands, but in my case it only gets two - which would be enough, except that the driver does not report that the scope is slewing after the first EQUATORIAL_EOD_COORD event is received, so the transition into AZ_SLEWING state doesn't happen until the second EQUATORIAL_EOD_COORD event - but by that time the slew has actually completed so the system gets stuck in the AZ_SLEWING state.

The local simulator seems to get much more frequent EQUATORIAL_EOD_COORD updates so the problem doesn't happen.

Gemini 2 (lx200generic)
2015-08-11T23:47:04 AZ_SYNCING -> AZ_SLEWING
2015-08-11T23:47:04 AZ_SYNCING mode
2015-08-11T23:47:04 AZ Stage: 2
2015-08-11T23:47:04 Mount at RA: 18:20:06   DEC: -02:52:35
2015-08-11T23:47:04 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-11T23:46:58 AZ_SYNCING -> X
2015-08-11T23:46:58 AZ_SYNCING mode
2015-08-11T23:46:58 AZ Stage: 2
2015-08-11T23:46:58 Mount at RA: 18:22:07   DEC: -02:52:35
2015-08-11T23:46:58 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-11T23:46:57 Solver completed in 4 seconds.
2015-08-11T23:46:57 Slewing 30 arcminutes in RA...
2015-08-11T23:46:57 Synchronizing position and slewing...
2015-08-11T23:46:57 measureAzError(): state: 1
2015-08-11T23:46:57 Processing solution for polar alignment...
2015-08-11T23:46:57 Solver RA (275.32) DEC (-2.8889) Orientation (-0.17158) Pixel Scale (3.5923)
2015-08-11T23:46:57 Field 1: solved with index index-4110.fits.
2015-08-11T23:46:56 log-odds ratio 105.538 (6.83035e+45), 13 match, 0 conflict, 22 distractors, 17 index.
2015-08-11T23:46:55 Solving...
2015-08-11T23:46:55 Reading sort column "FLUX"
2015-08-11T23:46:55 simplexy: found 458 sources.
2015-08-11T23:46:53 Downsampling by 2...
2015-08-11T23:46:53 Extracting sources...
2015-08-11T23:46:53 Reading input file 1 of 1: "/tmp/fitsM12367"...
2015-08-11T23:46:53 /usr/bin/solve-field --no-verify --no-plots --no-fits2fits --resort --downsample 2 -O -L 72.0217 -H 105.383 -u aw -3 274.925 -4 -13.7867 -5 30 -W /tmp/solution.wcs /tmp/fitsM12367
2015-08-11T23:46:53 Starting solver...
2015-08-11T23:46:53 Image received.
2015-08-11T23:46:43 Capturing image...
2015-08-11T23:46:43 Solving first frame near the meridian.
2015-08-11T23:46:42 measureAzError(): state: 0


Simulator
2015-08-12T07:45:45 AZ Stage: 0
2015-08-12T07:45:45 Mount at RA: 00:14:05   DEC: 15:16:17
2015-08-12T07:45:45 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:45 AZ Stage: 0
2015-08-12T07:45:45 Mount at RA: 00:14:05   DEC: 15:16:17
2015-08-12T07:45:45 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:45 AZ Stage: 0
2015-08-12T07:45:45 Mount at RA: 00:14:05   DEC: 15:16:17
2015-08-12T07:45:45 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:45 processTelescopeNumber(): EQUATORIAL_PE
2015-08-12T07:45:45 processTelescopeNumber(): EQUATORIAL_PE
2015-08-12T07:45:45 processTelescopeNumber(): EQUATORIAL_PE
2015-08-12T07:45:44 AZ Stage: 0
2015-08-12T07:45:44 Mount at RA: 00:13:41   DEC: 15:16:17
2015-08-12T07:45:44 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:44 AZ Stage: 0
2015-08-12T07:45:44 Mount at RA: 00:13:41   DEC: 15:16:17
2015-08-12T07:45:44 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:44 AZ Stage: 0
2015-08-12T07:45:44 Mount at RA: 00:13:41   DEC: 15:16:17
2015-08-12T07:45:44 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:43 AZ Stage: 0
2015-08-12T07:45:43 Mount at RA: 00:13:17   DEC: 15:16:17
2015-08-12T07:45:43 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:43 AZ Stage: 0
2015-08-12T07:45:43 Mount at RA: 00:13:17   DEC: 15:16:17
2015-08-12T07:45:43 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:43 AZ Stage: 0
2015-08-12T07:45:43 Mount at RA: 00:13:17   DEC: 15:16:17
2015-08-12T07:45:43 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:42 AZ Stage: 0
2015-08-12T07:45:42 Mount at RA: 00:12:53   DEC: 15:16:17
2015-08-12T07:45:42 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:42 AZ Stage: 0
2015-08-12T07:45:42 Mount at RA: 00:12:53   DEC: 15:16:17
2015-08-12T07:45:42 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:42 AZ Stage: 0
2015-08-12T07:45:42 Mount at RA: 00:12:53   DEC: 15:16:17
2015-08-12T07:45:42 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:41 AZ Stage: 0
2015-08-12T07:45:41 Mount at RA: 00:12:29   DEC: 15:16:17
2015-08-12T07:45:41 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:41 AZ Stage: 0
2015-08-12T07:45:41 Mount at RA: 00:12:29   DEC: 15:16:17
2015-08-12T07:45:41 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:41 AZ Stage: 0
2015-08-12T07:45:41 Mount at RA: 00:12:29   DEC: 15:16:17
2015-08-12T07:45:41 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:41 Solver completed in 2 seconds.
2015-08-12T07:45:41 Slewing back to original position...
2015-08-12T07:45:41 Calculating azimuth alignment error...
2015-08-12T07:45:41 measureAzError(): state: 6
2015-08-12T07:45:41 Processing solution for polar alignment...
2015-08-12T07:45:40 Starting solver...
2015-08-12T07:45:40 Image received.
2015-08-12T07:45:36 AZ Stage: 6
2015-08-12T07:45:36 Mount at RA: 00:12:05   DEC: 15:16:17
2015-08-12T07:45:36 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:36 AZ Stage: 6
2015-08-12T07:45:36 Mount at RA: 00:12:05   DEC: 15:16:17
2015-08-12T07:45:36 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:36 Capturing image...
2015-08-12T07:45:36 Solving second frame near the meridian.
2015-08-12T07:45:36 measureAzError(): state: 4
2015-08-12T07:45:36 AZ_SLEWING -> AZ_SECOND_TARGET
2015-08-12T07:45:36 AZ Stage: 3
2015-08-12T07:45:36 Mount at RA: 00:12:05   DEC: 15:16:17
2015-08-12T07:45:36 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:36 processTelescopeNumber(): EQUATORIAL_PE
2015-08-12T07:45:36 processTelescopeNumber(): EQUATORIAL_PE
2015-08-12T07:45:36 processTelescopeNumber(): EQUATORIAL_PE
2015-08-12T07:45:35 AZ_SLEWING -> X
2015-08-12T07:45:35 AZ Stage: 3
2015-08-12T07:45:35 Mount at RA: 00:12:28   DEC: 15:16:17
2015-08-12T07:45:35 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:35 AZ_SLEWING -> X
2015-08-12T07:45:35 AZ Stage: 3
2015-08-12T07:45:35 Mount at RA: 00:12:28   DEC: 15:16:17
2015-08-12T07:45:35 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:35 AZ_SLEWING -> X
2015-08-12T07:45:35 AZ Stage: 3
2015-08-12T07:45:35 Mount at RA: 00:12:28   DEC: 15:16:17
2015-08-12T07:45:35 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:34 AZ_SLEWING -> X
2015-08-12T07:45:34 AZ Stage: 3
2015-08-12T07:45:34 Mount at RA: 00:12:52   DEC: 15:16:17
2015-08-12T07:45:34 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:34 AZ_SLEWING -> X
2015-08-12T07:45:34 AZ Stage: 3
2015-08-12T07:45:34 Mount at RA: 00:12:52   DEC: 15:16:17
2015-08-12T07:45:34 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:34 AZ_SLEWING -> X
2015-08-12T07:45:34 AZ Stage: 3
2015-08-12T07:45:34 Mount at RA: 00:12:52   DEC: 15:16:17
2015-08-12T07:45:34 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:33 AZ_SLEWING -> X
2015-08-12T07:45:33 AZ Stage: 3
2015-08-12T07:45:33 Mount at RA: 00:13:16   DEC: 15:16:17
2015-08-12T07:45:33 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:33 AZ_SLEWING -> X
2015-08-12T07:45:33 AZ Stage: 3
2015-08-12T07:45:33 Mount at RA: 00:13:16   DEC: 15:16:17
2015-08-12T07:45:33 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:33 AZ_SLEWING -> X
2015-08-12T07:45:33 AZ Stage: 3
2015-08-12T07:45:33 Mount at RA: 00:13:16   DEC: 15:16:17
2015-08-12T07:45:33 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:32 AZ_SLEWING -> X
2015-08-12T07:45:32 AZ Stage: 3
2015-08-12T07:45:32 Mount at RA: 00:13:40   DEC: 15:16:17
2015-08-12T07:45:32 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:32 AZ_SLEWING -> X
2015-08-12T07:45:32 AZ Stage: 3
2015-08-12T07:45:32 Mount at RA: 00:13:40   DEC: 15:16:17
2015-08-12T07:45:32 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:32 AZ_SYNCING -> AZ_SLEWING
2015-08-12T07:45:32 AZ_SYNCING mode
2015-08-12T07:45:32 AZ Stage: 2
2015-08-12T07:45:32 Mount at RA: 00:13:40   DEC: 15:16:17
2015-08-12T07:45:32 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:32 AZ_SYNCING -> X
2015-08-12T07:45:32 AZ_SYNCING mode
2015-08-12T07:45:32 AZ Stage: 2
2015-08-12T07:45:32 Mount at RA: 00:14:05   DEC: 15:16:17
2015-08-12T07:45:32 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:32 AZ_SYNCING -> X
2015-08-12T07:45:32 AZ_SYNCING mode
2015-08-12T07:45:32 AZ Stage: 2
2015-08-12T07:45:32 Mount at RA: 00:14:05   DEC: 15:16:17
2015-08-12T07:45:32 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:32 AZ_SYNCING -> X
2015-08-12T07:45:32 AZ_SYNCING mode
2015-08-12T07:45:32 AZ Stage: 2
2015-08-12T07:45:32 Mount at RA: 00:14:05   DEC: 15:16:17
2015-08-12T07:45:32 processTelescopeNumber(): EQUATORIAL_EOD_COORD
2015-08-12T07:45:32 processTelescopeNumber(): EQUATORIAL_PE
2015-08-12T07:45:32 processTelescopeNumber(): EQUATORIAL_PE
2015-08-12T07:45:32 processTelescopeNumber(): EQUATORIAL_PE
2015-08-12T07:45:32 Solver completed in 2 seconds.
2015-08-12T07:45:32 Slewing 30 arcminutes in RA...
2015-08-12T07:45:32 Synchronizing position and slewing...
2015-08-12T07:45:32 measureAzError(): state: 1
2015-08-12T07:45:32 Processing solution for polar alignment...
2015-08-12T07:45:30 Starting solver...
2015-08-12T07:45:30 Index files index-4201 to index-4202 are missing. Astrometry.net would not be able to adequately solve plates until you install the missing index files. Download the index files from http://www.astrometry.net
2015-08-12T07:45:30 Image received.
2015-08-12T07:45:27 Capturing image...
2015-08-12T07:45:27 Solving first frame near the meridian.
2015-08-12T07:45:25 measureAzError(): state: 0

I'm going to continue to add more logging to the alignment module and add a "Verbose" checkbox that works like the solver's option. I remember reading another thread in which someone had the same problem, but I can't seem to find it. This would at least make it easier to help identify problems in the future.

Here's my GitHub clone kstars/aligndebug
The following user(s) said Thank You: Chris Ryan
8 years 7 months ago #4717

Please Log in or Create an account to join the conversation.

  • Posts: 205
  • Thank you received: 19
Thanks for looking into this - I could never work out what exactly was going on when the same problem happened to me.

If any of the other drivers work, I'd like to know. I never seemed to have any luck with them.
8 years 7 months ago #4718

Please Log in or Create an account to join the conversation.

  • Posts: 25
  • Thank you received: 8
It took a while to figure out what was going on - I'm used to debugging code in an office, not while sitting outside in a lawn chair at midnight!

Now that I have a better understanding of what's going on I'm going to take a stab at the state management code and try to make it more resilient to the ordering of events from the telescope. It might take a while because I'm still learning the codebase and properly handling asynchronous communication is pretty tricky. I'd really like to get a quick workaround in place ASAP, maybe one of the main developers has some suggestions?

It's still not completely clear what the specific issue is that causes the original bug that I started this thread about. Once I had all the debugging in place I only got the second problem of the state machine getting stuck before even starting the second exposure. I'm assuming it's the same problem and it's all dependent upon the ordering of EOD events and isSlewing() state. If the telescope driver is replicating state in an unexpected order it's very easy to create weird synchronization issues. What I really need to figure out is if there's a reliable way to detect the start and end of a slew without relying on the exact ordering of those two values.
8 years 7 months ago #4720

Please Log in or Create an account to join the conversation.

  • Posts: 25
  • Thank you received: 8
I'm still working on this one. I realized I can work on it during the day using the CCD Simulator on the RPI. I've figured out the root cause of the problem - the lx200driver is having a TTY timeout when reading the result of the get-distance-to-target-as-a-series-of-hashes command "#:D#". This is the command used to figure out if you've reached your target. As a result, all slews work, but fail to signal that they have completed. This is why the alignment tool gets stuck after the first image. It also probably means this bug is unrelated to the one in the topic title since it has the opposite effect. For some reason there isn't any error logging in the shared lx200 communication code, I might submit a patch with everything I've added to help anyone else who runs across these types of problems.

I have to learn a new codebase every time I descend into a new tier of the architecture, but now I've got a reasonable debugging environment set up so it's quicker to iterate. The simplicity of the indiserver/driver architecture definitely helps out.
2015-08-15T22:01:31: Driver ./indi_lx200generic: isSlewComplete Command [#:D#]
2015-08-15T22:01:36: Driver ./indi_lx200generic: tty_read_section: Timed out after reading 0 bytes [code -4]
2015-08-15T22:01:36: Driver ./indi_lx200generic: isSlewComplete Reading command [#:D#] result failed with -4
8 years 7 months ago #4725

Please Log in or Create an account to join the conversation.

  • Posts: 25
  • Thank you received: 8
I've got a functional workaround for this problem on my mount. After more testing it looks like there's a bug in the Gemini 2 firmware and it won't respond to the #:D# command at all. I reproduced the problem in minicom.

github.com/seanhoughton/indi/commit/da96...7e2e0a79e433662af146

The lx200basic driver doesn't have this problem because it uses distance to target threshold to trigger the slewing completion instead of the bars command. However, it does not inherit from INDI::Telescope so it doesn't work with any plate solving tools because it's missing the telescope properties. My fix to the lx200generic driver just copies this technique and it works perfectly on my mount.

Note: this fix could actually cause the original problem of the scope still slewing while taking the second exposure if the threshold is too high. This value it tunable on the lx200basic driver but my fix hardcodes it at a small value.
8 years 7 months ago #4726

Please Log in or Create an account to join the conversation.

  • Posts: 205
  • Thank you received: 19
Sounds like good progress.

Would it be possible to make the hardcoded amount a config item for the driver?
8 years 7 months ago #4727

Please Log in or Create an account to join the conversation.

  • Posts: 25
  • Thank you received: 8
I confirmed on the Gemini-2 forums that the
:D#
command is not supported. I'll clean up the patch and make the accuracy a parameter next week. In the meantime you can manually change the threshold and re-build the driver I linked on github.

Last night I managed to get really good polar alignment in only about 20 minutes - around 4x faster than my usual pace with drift alignment.



I took some 120s unguided test images that we're much better than anything I've ever tried before without a guider. Note, these are completely unprocessed except for a quick histogram transformation.

The following user(s) said Thank You: Jasem Mutlaq
Last edit: 8 years 7 months ago by Sean Houghton. Reason: Clarify that the images are test images
8 years 7 months ago #4730
Attachments:

Please Log in or Create an account to join the conversation.

  • Posts: 205
  • Thank you received: 19
That's a great result. What focal length are you using?
8 years 7 months ago #4731

Please Log in or Create an account to join the conversation.

  • Posts: 25
  • Thank you received: 8
That's taken with a Tak FS-102 at f/6 (620mm) and an ST-8300M. It's not exactly a long focal length, and it was binned 2x. Still, it puts my previous polar alignment efforts to shame. I think it was probably aligned past the point where flexure and non-orthogonality affect the result more than polar alignment.

I've created a new lx200gemini driver derived from lx200generic. Jasem has been helping out with suggestions and reviewing the code. The tolerance value is a parameter so you can tune it for your mount. I'll probably add some control over the PEC training as well now that there's a good spot for the code and properties.
8 years 7 months ago #4817

Please Log in or Create an account to join the conversation.

Time to create page: 0.403 seconds