×

INDI Library v2.0.7 is Released (01 Apr 2024)

Bi-monthly release with minor bug fixes and improvements

Read More...

New All Sky camera management software: indi-allsky

  • Posts: 257
  • Thank you received: 66

Replied by Aaron Morris on topic New All Sky camera management software: indi-allsky

That looks fantasic! You have your focus really dialed in.

I recently added the ability to "stretch" images. I might provide some additional contrast enhancement with the milky way.
11 months 1 week ago #92641

Please Log in or Create an account to join the conversation.

  • Posts: 535
  • Thank you received: 109

Replied by Jim on topic New All Sky camera management software: indi-allsky

I updated and see the options. It has been cloudy ever since, of course
11 months 1 week ago #92709

Please Log in or Create an account to join the conversation.

  • Posts: 83
  • Thank you received: 4

Replied by Markus Kempf on topic New All Sky camera management software: indi-allsky

I got indi-allsky running with a remote indi_pylibcamera (PiHQ with RPI Zero W) for about two weeks now. Today I did an update (git pull) of indi-allsky. Since this time, the program no longer adjusts the exposure time. It stays at the minimum time.
further bugs:
- System menu does not work, I get Internal Server Error. All other menus do work.
- the image white balance is again quite bad. It was perfect with indi-allsky using libcamera directly. Might be a problem of indi_pylibcamera
- there is an error message in the log about a write attempt to /var/lib/containers/storage/overlay but indi-allsky is not running in a container...
May 10 20:09:51 localhost [ERROR] MainProcess allsky._startImageWorker() #768: Image worker exception: 
May 10 20:09:51 localhost [ERROR] MainProcess allsky._startImageWorker() #768: Image worker exception: PermissionError: [Errno 13] Permission denied: '/var/lib/containers/storage/overlay'
May 10 20:09:51 localhost [ERROR] MainProcess allsky._startImageWorker() #768: Image worker exception:     st = os.statvfs(path)
May 10 20:09:51 localhost [ERROR] MainProcess allsky._startImageWorker() #768: Image worker exception:   File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/psutil/_psposix.py", line 175, in disk_usage
May 10 20:09:51 localhost [ERROR] MainProcess allsky._startImageWorker() #768: Image worker exception:     return _psplatform.disk_usage(path)
May 10 20:09:51 localhost [ERROR] MainProcess allsky._startImageWorker() #768: Image worker exception:   File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/psutil/__init__.py", line 2000, in disk_usage
May 10 20:09:51 localhost [ERROR] MainProcess allsky._startImageWorker() #768: Image worker exception:     disk_usage = psutil.disk_usage(fs.mountpoint)
May 10 20:09:51 localhost [ERROR] MainProcess allsky._startImageWorker() #768: Image worker exception:   File "/home/admin/indi-allsky/indi_allsky/image.py", line 652, in processImage
May 10 20:09:51 localhost [ERROR] MainProcess allsky._startImageWorker() #768: Image worker exception:     self.processImage(i_dict)
May 10 20:09:51 localhost [ERROR] MainProcess allsky._startImageWorker() #768: Image worker exception:   File "/home/admin/indi-allsky/indi_allsky/image.py", line 258, in saferun
May 10 20:09:51 localhost [ERROR] MainProcess allsky._startImageWorker() #768: Image worker exception:     self.saferun()
May 10 20:09:51 localhost [ERROR] MainProcess allsky._startImageWorker() #768: Image worker exception:   File "/home/admin/indi-allsky/indi_allsky/image.py", line 229, in run
May 10 20:09:51 localhost [ERROR] MainProcess allsky._startImageWorker() #768: Image worker exception: Traceback (most recent call last):

CS, Markus
11 months 1 week ago #92746

Please Log in or Create an account to join the conversation.

  • Posts: 257
  • Thank you received: 66

Replied by Aaron Morris on topic New All Sky camera management software: indi-allsky

Can you check your filesystems to see if you are out of space anywhere? indi-allsky is checking the disk usage using the psutil module and it is getting an permission denied error. If not, that is fine. I have merged an update to catch this exception. This will also fix the System menu (which is making the same disk check).

For the exposure, try setting the default exposure to something like 0.01 and see if that helps get things going.

And color balance... were you using using JPEG mode with libcamera? If so, that enables the auto white balance mode which corrects the colors (but also causes other problems). indi_pylibcamera returns RAW data which does not have the auto-white balance enabled. I will research to see if it possible to re-enable that.
Last edit: 11 months 1 week ago by Aaron Morris.
11 months 1 week ago #92747

Please Log in or Create an account to join the conversation.

  • Posts: 83
  • Thank you received: 4

Replied by Markus Kempf on topic New All Sky camera management software: indi-allsky

you got it right, /tmp was full. I deleted dozens of tmp*.fit files created by indi-allsky and it should have enough space again. The exposure error went away with the sunset happening right now. Yes, I have used/tested libcamera with JPEG mode. It provided excellent pictures. Would be nice if the same quality could be achieved with raw. The secret sauce of the PiHQ camera firmware should not be a well hidden enigma.
After the git pull and restart the System menu is working now, but the May 10 20:58:32 localhost [ERROR] ImageWorker001 image.processImage() #655: PermissionError: [Errno 13] Permission denied: '/var/lib/containers/storage/overlay' is still there.
With default exposure 0.01 it works now again.

CS, Markus
11 months 1 week ago #92748

Please Log in or Create an account to join the conversation.

  • Posts: 257
  • Thank you received: 66

Replied by Aaron Morris on topic New All Sky camera management software: indi-allsky

I have implemented subtracting the offset from the image from indi_pylibcamera. I had already implemented from the libcamera-still side (it was automatically subtracted in JPEG mode).

I do not know why libcamera implemented this crazy offset... the standard offset for most of their camera modules is 4096 (in a 16-bit space). This basically means the image has a minimum RGB color of #101010, a light gray. It gives the image a hazy look. If you take dark frames, this would be eliminated, but the offset is so high, it needs to be subtracted if you do not have them.

I am not a big fan of the auto white balance from libcamera. It does give proper color balance, but it has a very nasty side effect at night. During the day, there is enough light, libcamera can give proper color offsets on the first exposure. However, at night, there is so little light, it has to repeat the exposure internally 3-6 times to return proper balance. This means a 30s exposure can take 120 seconds to complete. This is one of the reasons I implemented SCNR (from the PixInsight docs) to reduce the green bias from the camera.

Edit: I also increased the max gain for imx477 and imx378 to 22.
The following user(s) said Thank You: Markus Kempf
Last edit: 11 months 1 week ago by Aaron Morris.
11 months 1 week ago #92753

Please Log in or Create an account to join the conversation.

  • Posts: 83
  • Thank you received: 4

Replied by Markus Kempf on topic New All Sky camera management software: indi-allsky

Aaron, thanks a lot for the quick updates! It's running with the latest release and the pic looks crisper with the offset correction. I use it with a circular fisheye lens and the area outside the image circle is pitch black now and not middle grey. For the daytime pictures it would be good to use the libcamera auto white balance mode and for nighttime you may switch to another WB mode. btw the ImageWorker001 image.processImage() #659: PermissionError: [Errno 13] Permission denied: '/var/lib/containers/storage/overlay' error is still in the log.

CS, Markus
11 months 1 week ago #92759

Please Log in or Create an account to join the conversation.

  • Posts: 257
  • Thank you received: 66

Replied by Aaron Morris on topic New All Sky camera management software: indi-allsky

The error is caught and just logged. Can you show me the output of "df"? I may just hard code the exclude of that filesystem.

indi_pylibcamera does not currently support changing the AWB modes. I may try to make the modification myself and submit a PR.

Try using the the SCNR option on "Maximum Neutral" it almost completely eliminates the green bias from the images, although I realize the colors are not quite the same.
11 months 1 week ago #92764

Please Log in or Create an account to join the conversation.

  • Posts: 83
  • Thank you received: 4

Replied by Markus Kempf on topic New All Sky camera management software: indi-allsky

I already use the SNCR option with Maximum neutral. The df output:
Dateisystem      1K-Blöcke     Benutzt   Verfügbar Verw% Eingehängt auf
udev               3957356           0     3957356    0% /dev
tmpfs               797788       12456      785332    2% /run
/dev/sda2        106377928    81302296    19625700   81% /
tmpfs              3988924          88     3988836    1% /dev/shm
tmpfs                 5120           4        5116    1% /run/lock
/dev/sda1           523248         152      523096    1% /boot/efi
tmpfs              3988924      505876     3483048   13% /tmp
/dev/sdb1       1921724676   430857688  1490850604   23% /srv/dev-disk-by-label-SSData
disks          36960024576 20394953216 16565071360   56% /disks
tmpfs               797784           0      797784    0% /run/user/501
tmpfs               797784           0      797784    0% /run/user/0

but after this mornings git pull, I get a new error with gunicorn:
Mai 12 10:21:11 openmediavault gunicorn[2679441]: [ERROR] MainProcess app.log_exception() #1744: Exception on /indi-allsky/ [GET]
                                                  Traceback (most recent call last):
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1964, in _exec_single_context
                                                      self.dialect.do_execute(
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 748, in do_execute
                                                      cursor.execute(statement, parameters)
                                                  sqlite3.OperationalError: no such column: camera.hidden
 
                                                  The above exception was the direct cause of the following exception:
 
                                                  Traceback (most recent call last):
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/flask/app.py", line 2528, in wsgi_app
                                                      response = self.full_dispatch_request()
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/flask/app.py", line 1825, in full_dispatch_request
                                                      rv = self.handle_user_exception(e)
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
                                                      rv = self.dispatch_request()
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
                                                      return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
                                                    File "/home/admin/indi-allsky/indi_allsky/flask/misc.py", line 22, in decorated_view
                                                      return current_app.ensure_sync(func)(*args, **kwargs)
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/flask/views.py", line 104, in view
                                                      self = view.view_class(  # type: ignore[attr-defined]
                                                    File "/home/admin/indi-allsky/indi_allsky/flask/base_views.py", line 147, in __init__
                                                      self.setupSession()
                                                    File "/home/admin/indi-allsky/indi_allsky/flask/base_views.py", line 161, in setupSession
                                                      self.camera = self.getCameraById(session['camera_id'])
                                                    File "/home/admin/indi-allsky/indi_allsky/flask/base_views.py", line 92, in getCameraById
                                                      camera = IndiAllSkyDbCameraTable.query\
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/sqlalchemy/orm/query.py", line 2752, in first
                                                      return self.limit(1)._iter().first()  # type: ignore
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/sqlalchemy/orm/query.py", line 2855, in _iter
                                                      result: Union[ScalarResult[_T], Result[_T]] = self.session.execute(
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 2229, in execute
                                                      return self._execute_internal(
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 2124, in _execute_internal
                                                      result: Result[Any] = compile_state_cls.orm_execute_statement(
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/sqlalchemy/orm/context.py", line 253, in orm_execute_statement
                                                      result = conn.execute(
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1414, in execute
                                                      return meth(
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 486, in _execute_on_connection
                                                      return connection._execute_clauseelement(
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1638, in _execute_clauseelement
                                                      ret = self._execute_context(
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1842, in _execute_context
                                                      return self._exec_single_context(
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1983, in _exec_single_context
                                                      self._handle_dbapi_exception(
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2326, in _handle_dbapi_exception
                                                      raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1964, in _exec_single_context
                                                      self.dialect.do_execute(
                                                    File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 748, in do_execute
                                                      cursor.execute(statement, parameters)
                                                  sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such column: camera.hidden
                                                  [SQL: SELECT camera.id AS camera_id, camera.uuid AS camera_uuid, camera.name AS camera_name, camera.driver AS camera_driver, camera."friendlyName" AS "camera_friendlyName", camera."createDate" AS "camera_createDate", camera."connectDat>
                                                  FROM camera
                                                  WHERE camera.id = ?
                                                   LIMIT ? OFFSET ?]
                                                  [parameters: (2, 1, 0)]
                                                  (Background on this error at: https://sqlalche.me/e/20/e3q8)
Mai 12 10:21:12 openmediavault gunicorn[2679441]: [ERROR] MainProcess app.log_exception() #1744: Exception on /indi-allsky/ [GET]
                                                  Traceback (most recent call last):
11 months 1 week ago #92787

Please Log in or Create an account to join the conversation.

  • Posts: 257
  • Thank you received: 66

Replied by Aaron Morris on topic New All Sky camera management software: indi-allsky

I made a database schema change in the last update. Just re-run setup.sh and you should be all set.
11 months 1 week ago #92792

Please Log in or Create an account to join the conversation.

  • Posts: 83
  • Thank you received: 4

Replied by Markus Kempf on topic New All Sky camera management software: indi-allsky

thanks, does work now.

CS, Markus
11 months 1 week ago #92795

Please Log in or Create an account to join the conversation.

  • Posts: 83
  • Thank you received: 4

Replied by Markus Kempf on topic New All Sky camera management software: indi-allsky

I get this error after days of running setup.sh on my RPI Zero W:
**** Setup DB ****
./setup.sh: Zeile 1804: 3257 Ung▒ltiger Maschinenbefehl flask db init

CS, Markus
10 months 5 hours ago #93548

Please Log in or Create an account to join the conversation.

Time to create page: 1.345 seconds