Lee Peterson replied to the topic 'Ekos Scheduler did not wake up' in the forum. 1 year ago

I am happy to report that everything ran perfectly last night. By unchecking the "Park Dome" on the scheduler GUI, the scheduler shutdown the system properly and then woke up as planned.

Thank you everyone for your help!

Read More...

Lee Peterson replied to the topic 'Ekos Scheduler did not wake up' in the forum. 1 year ago

Thank you, Jasem!

I saw your commit with the changes and I would like to test it tonight. But looking at the binary-factory Jenkins dashboard for MacOS, It doesn't look like the recently built commit 918bc20506c4b139ca35ca4d53372496b73b05e0 corresponds to any commit on invent.kde.org/education/kstars. So there must be something up with the git SHA. At any rate, is your commit in the latest build #1853? If so I'll test it tonight.

Oh and by the way I'd like to add my observatory info to my signature, but the link in your signature is broken. Can you tell me where to do that on this site?

Read More...

Lee Peterson replied to the topic 'Ekos Scheduler did not wake up' in the forum. 1 year ago

Hello Hy,

I've studied the code and logic, and had a look at the iOptron RS-232 syntax. I ran another schedule last night the same as before, and encountered the same stuck state. Here's what I've found:

- The iOptron error I mentioned above did not recur. The debug output was indicating that the mount responses to get_status and get_position were mixed up. That is, the command string ":GLS#" for status was returning the response for the command string ":GEP#" and visa versa. That looks like a mount firmware problem, or perhaps something wrong in a FIFO buffer somewhere. At any rate, the error did not repeat, and so I'll ignore it for now.

- What I did see is that, when the scheduler went to sleep, it parked the mount, emitting a "Mount parked" info string into the log. It then went into a perpetual "Checking shutdown state" loop. Looking at the code, the next state would have been SHUTDOWN_PARK_DOME. I presume it got stuck there as the message "Checking shutdown state" repeated all night.

I have a dome, but that is on another system. This particular system does not have a dome. But to my surprise, the schedule I am running has "Park Dome" checked. I generate my .esl files in python using templates. I bet I imported a template .esl file that was generated on my other system that has a dome.

So my hypothesis is that the scheduler was trying to park a dome that does not exist. By checking "Park Dome" in the GUI, the scheduler state machine would see that parkDomeCheck->isChecked() is True. But inside parkDomeCheck, domeInterface.isNull() also returns True since there is no dome. But the immediate return from domeInterface.isNull() does not set the shutdownState to SHUTDOWN_SCRIPT, which would be the next step. Instead, it stays in SHUTDOWN_PARK_DOME, causing the loop to repeat.

Tonight I will have a chance to run this again with "Park Dome" unchecked in the schedule. Weather and full moon might wash out tonight's run, so it might be a couple days.

Read More...

Lee Peterson replied to the topic 'Ekos Scheduler did not wake up' in the forum. 1 year ago

Morning Hy,

Unfortunately last night's run crashed again in the middle of the first run due to a mount error. I bet this is behind the problem with the scheduler not waking up. Here is the error message that repeated every second all night:

iOptron v3 : "[ERROR] bool IOPv3::Driver::getCoords(double*, double*, IOPv3::IOP_PIER_STATE*, IOPv3::IOP_CW_STATE*): Expected 20 bytes but received 23. "

This particular mount is an iOptron CEM 40 EC. Previously I had been communicating with it via a powered USB bus, and had problems with it dropping intermittently over the past year, often requiring a restart of ekos. I recently tracked down that there was some EMI issue with the mount's USB controller almost every time it slewed. I surmise that was due to an issue in the mount's power system or something emitted by the stepper motors. It actually was creating issues with all the USB devices. So I moved the mount comm over to the wifi interface and it solved the EMI problem and had been running fine, apparently.

But now I think the wifi interface has created another issue with getting the mount state, maybe. I found this interesting comment in the source code.

github.com/indilib/indi/blob/master/driv...e/ioptronv3.cpp#L808

That's right where where the status gets set in the driver. This particular stellar mate has indilib 1.9.7 installed. I'm going to try some more debugging runs and I'll let you know if I can see what is happening.

Read More...

Lee Peterson replied to the topic 'Ekos Scheduler did not wake up' in the forum. 1 year ago

That would not surprise me (retired hardware programmer from academia and NASA, et al). I will disable that logging and re-run tonight exactly from yesterday's schedule.esl

Read More...

Lee Peterson replied to the topic 'Ekos Scheduler did not wake up' in the forum. 1 year ago

Ah. If I use reply instead of quick reply it is an option. But unfortunately its size exceeds that allowed by this site. So I have temporarily assigned it to this public dropbox. Please let me know when you have it so that I can clean it out.

www.dropbox.com/s/ovqxlnj2e4cj9f2/log_20-42-15.txt?dl=0

Read More...

Lee Peterson replied to the topic 'Ekos Scheduler did not wake up' in the forum. 1 year ago

I have one, but there are about 100K lines about the mount debugging between the park command when it went to sleep, and when I woke it up in the AM. I've been debugging mount commands, but I will turn those off tonight. I will upload that log if there is a way to do that without polluting the chat. I don't see a way to upload the file independently of the chat. Is there?

Read More...

Lee Peterson replied to the topic 'Ekos Scheduler did not wake up' in the forum. 1 year ago

You know that might be it. I love the greedy scheduler and always enable it, but maybe since job 2 was second in the list ... I'll try making it the priority tonight, with verbose logs enabled, and see what happens.

Thanks!!

Read More...

Lee Peterson replied to the topic 'Ekos Scheduler did not wake up' in the forum. 1 year ago

Very good point. I thought maybe it was obvious to some of you. I will enable verbose logs and re-try tonight. Thank you!

Read More...

Lee Peterson created a new topic ' Ekos Scheduler did not wake up' in the forum. 1 year ago

I'm really enjoying kstars/ekos and finally have my hardware stable enough for everything to run all night.

Last night I tried to run two targets. One was scheduled to run from astronomical twilight to 2330, when it fell below the 20 degree limit my site requires. The other was scheduled to start at 0230 in the morning, when it rose above the 20 degree limit.

The first target was acquired perfectly. The mount parked as planned, and the system went to sleep. But it never woke up for the second target.

This morning it looked like everything was fine. There were no hardware errors in ekos or in the dmesg or in the syslog. Nothing had crashed, but the scheduler was still asleep.

Any ideas on what might have prevented the scheduler from waking? I have a watchdog, but it did not trigger and is currently set to not do anything. And the weather limits were perfect all night.

Read More...

Lee Peterson replied to the topic 'Nexdome Beaver will not slave' in the forum. 1 year ago

Fixed! (Also posted on cloudy nights)

It seems that the dome will not start following the mount unless the mount is in its programmed park position. What I mean is that the mount needs to be in the position programmed as "park" in the hand controller. I have an ioptron CEM120EC2, so no doubt this is related to that driver plus the NexDome beaver.

The root cause was that I set that "park" position to west side vertical to prevent mechanical interference with the dome. Its a tight fit. However, I ordinarily start my observing session from the "zero" position since that makes it easier for me to fit into the dome and access my computer. In kstars, when I select my alignment target near the pole, it thinks that the mount is parked. But the dome apparently does not. So when I use kstars to move the mount to a target, the dome ignores it.

Here's what works:

- Assume the mount is pointing somewhere other than its programmed "park" position
- In the ekos dome tab, park the dome
- In the ekos mount tab, park the mount (and get out of the way!)
- In the ekes dome tab, unpark the dome. Now, the dome moves to align with the mount park position
- In the ekos mount tab, unpark the mount (this might work too from kstars, but I haven't tried that one)

Now the dome is slaved to the mount and one can proceed as normal.

If the developers see this, maybe a new feature might be a button under the ekos/dome tab to have the dome re-sync and follow the mount from any state or position. Just a suggestion.

Read More...

Lee Peterson created a new topic ' Nexdome Beaver will not slave' in the forum. 1 year ago

I've also posted this on cloudynights.

I have a new NexDome with the Beaver controller. The software is served off a Stellarmate and controlled remotely via MacOS using kstars and ekos.

Last night I had trouble getting the dome to slave to the mount. Eventually (and quite accidentally) I managed to get it to work. I got almost 8 hours of images.

But tonight I cannot repeat my success. I have tried disable/enable slaving, pressing abort, park, unpark. Nothing is working.

Any ideas out there?

Read More...

Lee Peterson replied to the topic 'NexDome Park before Close' in the forum. 1 year ago

Solved. I found the setting under the options tab, mount policy, mount locks.

I’ll test to see what it does with the rain sensor after I install it.

Read More...

Lee Peterson created a new topic ' NexDome Park before Close' in the forum. 1 year ago

Hi Everyone,

I have a new NexDome housing an iOptron RC14 OTA. I am concerned that there is not enough mechanical clearance between the telescope and the dome's shutter motor. With some adjustments, I can get the OTA to barely clear the shutter motor when horizontal, and by ensuring a minimum altitude of 20 degrees I can clear the motor with some margin. But I am concerned that the margin would be too little if the dome were parked and closed with the telescope in an arbitrary position.

One solution for me would be to always park the telescope before parking and closing the dome. Is there a way to ensure this within Ekos?

My system uses the NexDome beaver Indilib driver. I see that there is a "park before close" option in the driver parameters, but I presume this is to park the dome in azimuth first, then close the shutter. That is, it does not park the telescope first. Is this correct?

I also intend to use the rain sensor, so I wonder if the parking order is the same when it is triggered as in a normal condition.

Any other solution ideas would be greatly welcomed!

Read More...

I noticed anomalous behavior on 2 different Macs both running kstars 3.6.0 and macOS 12.6.1. The drift plot is no longer available under the ekos start & stop pane. In fact, only the drift graphics (rms error vs time) can be shown. Clicking on the arrows to shift the plot left or right does not work.

I have hesitated to update to 3.6.1. Perhaps this is fixed there?

Read More...