Welcome, Guest
Username: Password: Remember me
25 Jul 2018
Glad to announce of release of INDI Library v1.7.4 on 2018-07-25. A few drivers were added in this release as we continue to improve & stabilize existing drivers.
Read More...

TOPIC: Perfecting the Scheduler

Re:Perfecting the Scheduler 6 months 14 hours ago #25389

Have you looked at CCACHE?

Please Log in or Create an account to join the conversation.

Jasem Mutlaq
Support INDI & Ekos; Get StellarMate Astrophotography Gadget.
How to Submit Logs when you have problems?
Add your observatory info

Re:Perfecting the Scheduler 6 months 12 hours ago #25391

I am actually using ccache in the project configuration. But it's not resulting in the acceleration I'm used to. Apart from the configuration issue where enabling the option calls "ccache /usr/lib/ccache/g++" which is a weird though perhaps harmless process duplication, cache hits are really low. But even at the dependency enumeration, cmake sometimes rebuild files I see no reason for it to. Probably my configuration...

-Eric

Please Log in or Create an account to join the conversation.

HEQ5-Pro - Atik 314E - Orion ED80T - DMK21 on Orion 50mm
DIY 3D-printed Moonlite and FWheel RGB/LPR
KStars and indiserver on two Atom 1.6GHz 1GB RAM Linux, VPN remote access

Re:Perfecting the Scheduler 6 months 1 hour ago #25415

As if slow builds weren't enough, I suffered a VM corruption this evening :) but my changes are being tested, scheduler is sleeping until twilight before capturing M51.

-Eric

Please Log in or Create an account to join the conversation.

HEQ5-Pro - Atik 314E - Orion ED80T - DMK21 on Orion 50mm
DIY 3D-printed Moonlite and FWheel RGB/LPR
KStars and indiserver on two Atom 1.6GHz 1GB RAM Linux, VPN remote access

Re:Perfecting the Scheduler 5 months 3 weeks ago #25631

Hi folks,
I am not sure whether I got all points right in the thread, so please excuse, if I bring up a topic here that is already addressed.

But first of all: the scheduler is THE killer application for me. Thanks a lot, it brings kstars on a completely new level.

Back to my problem. As far as I understood, when a job fails after several retries (for example during focussing), the job is marked as "ABORTED". But this might be too strict for example if clouds pass by.

So it could happen, that a certain job could not be executed for a while. If other jobs are present that could run, OK, let them run. But what happens, if all jobs are aborted? In my case, I only have one single job. In this case, EKOS is terminated, right? At least this happens in my case.

What would be the suggested behavior: I would like having a "retry after a certain amount of time" and restart the aborted job.

Does that make sense to you?
Cheers
Wolfgang

Please Log in or Create an account to join the conversation.

Re:Perfecting the Scheduler 5 months 3 weeks ago #25635

@sterne-jaeger this absolutely makes sense, and that's what I work towards to. Ideally (my view at least), the scheduler should be able to plan ahead as many observations as possible, and be robust to as many error situations as possible. But we can't go too fast with features.

The recent changes were targeting scheduler jobs and sequence jobs, so that captures were counted properly. There are still issues, for instance when there is no filter wheel, and side effects, for instance when duplicated jobs all complete at the same time, but that will be solved soon.

The other activity was targeting job evaluation, so that jobs are aborted when the scheduler stops and re-evaluated when the scheduler restarts. The idea is to adjust the algorithm to always restart aborted jobs while the scheduler runs, either with a limited number of retries or with a lower priority. Here too, there are side effects to the preliminary fixes, for instance scheduled jobs are now assigned a fixed start time, but there is no provision yet to start such scheduled jobs in advance. This will be solved soon too.

-Eric
The following user(s) said Thank You: El Corazon

Please Log in or Create an account to join the conversation.

HEQ5-Pro - Atik 314E - Orion ED80T - DMK21 on Orion 50mm
DIY 3D-printed Moonlite and FWheel RGB/LPR
KStars and indiserver on two Atom 1.6GHz 1GB RAM Linux, VPN remote access

Re:Perfecting the Scheduler 5 months 3 weeks ago #25638

Thanks, Eric! The scheduler already works really well as is, thanks to your and Jasem's efforts.

The priorities as I see them are:
#1 and of foremost importance by far: Ensure that no imaging time is wasted during a clear night!

That already can be achieved with the scheduler as is, as long as a series of backup jobs are scheduled. That, together with the ability to reevaluate virtually assures that every minute of darkness is used for photon collection, even though the targets may not be worked off in order of priority. During a clear night, the scheduler works very predictably, at least in my hands.
The other issues can be solved by scheduling the same target repeatedly with different priorities. That can take care of the passing cloud contingency.

At any rate, the scheduler is vastly improved over what it could do only a few weeks ago. By all means, take your time with adding the remaining tweaks.

And yes, the scheduler is THE killer app!

Jo

Please Log in or Create an account to join the conversation.

Re:Perfecting the Scheduler 5 months 3 weeks ago #25728

I'm testing my recent additional changes on the scheduler, and I observe that when using the filter wheel embedded in the CCD simulator, my sequence files requesting L, R, G, and B never process L.

I can see the capture tab requesting change to Red, Green and Blue, but when it is time to set the Luminosity filter, (1) there is no log in the capture tab, (2) the CCD states it is moving to filter 7, which is LPR, and (3) the capture is stored in folder "Blue".

This issue is causing a side effect to the scheduler, which cannot consider the job complete when there is a missing Luminosity set. BUT there seems to be an additional issue to counterbalance this, as the capture module notifies completion to the scheduler, which can thus continue. As a consequence, when starting the scheduler anew, all sequence jobs are considered uncompleted and restart.

This issue didn't arise when using my physical setup in the basement. However, there was another problem with the path to which the capture was stored, which did not contain a filter name when no filter driver was connected. I worked around this with a filter wheel simulator (not to be confused with the CCD simulator embedding a filter wheel above), but I'll have to come back to this later on.

-Eric

Please Log in or Create an account to join the conversation.

HEQ5-Pro - Atik 314E - Orion ED80T - DMK21 on Orion 50mm
DIY 3D-printed Moonlite and FWheel RGB/LPR
KStars and indiserver on two Atom 1.6GHz 1GB RAM Linux, VPN remote access

Re:Perfecting the Scheduler 5 months 3 weeks ago #25730

I think I found the reason : the CCD simulator has a "Luminance" filter, and the filter wheel simulator has a "Luminosity" filter. Hence sequence files are somehow incompatible between my physical setup and my simulated setup.

-Eric

Please Log in or Create an account to join the conversation.

HEQ5-Pro - Atik 314E - Orion ED80T - DMK21 on Orion 50mm
DIY 3D-printed Moonlite and FWheel RGB/LPR
KStars and indiserver on two Atom 1.6GHz 1GB RAM Linux, VPN remote access

Re:Perfecting the Scheduler 5 months 3 weeks ago #25734

I'll update Luminosity --> Luminance in the filter wheel simulator as well.

EDIT: Actually, they should be the same. You probably have this saved as config from a long time ago.
The following user(s) said Thank You: TallFurryMan

Please Log in or Create an account to join the conversation.

Jasem Mutlaq
Support INDI & Ekos; Get StellarMate Astrophotography Gadget.
How to Submit Logs when you have problems?
Add your observatory info
Last Edit: by knro.

Re:Perfecting the Scheduler 5 months 3 weeks ago #25735

I'll have to check: I created the sequence while connected to my physical setup, before I realized there was no filter wheel in said setup. Thus I think the unexpected filter name is not sourcing from the INDI setup, but from the capture tab itself. But that's maybe what you mean.

-Eric

Please Log in or Create an account to join the conversation.

HEQ5-Pro - Atik 314E - Orion ED80T - DMK21 on Orion 50mm
DIY 3D-printed Moonlite and FWheel RGB/LPR
KStars and indiserver on two Atom 1.6GHz 1GB RAM Linux, VPN remote access

Re:Perfecting the Scheduler 5 months 2 weeks ago #25805

@knro any pointers on how I can write tests for the scheduler? I'm preparing several scheduler/sequence files and I'd like to automate their execution and verification.

-Eric

Please Log in or Create an account to join the conversation.

HEQ5-Pro - Atik 314E - Orion ED80T - DMK21 on Orion 50mm
DIY 3D-printed Moonlite and FWheel RGB/LPR
KStars and indiserver on two Atom 1.6GHz 1GB RAM Linux, VPN remote access

Re:Perfecting the Scheduler 5 months 2 weeks ago #25809

  • kecsap
  • kecsap's Avatar
  • Offline
  • Expert Boarder
  • Expert Boarder
  • Posts: 86
  • Karma: 4
  • Thank you received: 34
@TallFurryMan: Writing tests for the KStars GUI is difficult, I recommend to forget about it for the scheduler.
The following user(s) said Thank You: TallFurryMan

Please Log in or Create an account to join the conversation.

Time to create page: 0.166 seconds

Login



3rd Party

Choose from the numerous 3rd party INDI drivers to suit your needs!

Got Problem?

Check out the FAQ, the forum, and the bug tracking system to resolve any issues you might have!
You can also subscribe to INDI newsletter and development mailing lists to get the latest updates on INDI!


Gallery

Replica

Why INDI

Replica