First of all many thanks for working on this. I am really impressed by the progress you made!

I made some experiments in Python, mainly to understand:

• how libcamera works,
  
• how to select and configure a camera,
  
• what is needed to get raw data and what format it has,
  
• how to configure libcamera for fast exposure (running the next exposure while the actual one gets processed and transmitted to client) and
  
• if libcamera can do very long exposures and how to switch between exposure times.

I did this on a Raspberry Pi Zero with HQ camera (the hardware I use for astro photography). Attached are the Python code and its output.Maybe it can help you to implement the libcamera staff or to make own experiments.

I would like to, but can't

sudo apt-get -y install libcap-dev python3-libcamera
pip install picamera2

libcamera-hello
->
[24:45:34.494209213] [24736] INFO Camera camera_manager.cpp:299 libcamera v0.0.2+47-0684c373

then

python
Python 3.9.2 (default, Mar 12 2021, 04:06:34)
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> import datetime
>>> import pprint
>>> from picamera2 import Picamera2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/pi/.local/lib/python3.9/site-packages/picamera2/__init__.py", line 5, in <module>
from .picamera2 import Picamera2, Preview
File "/home/pi/.local/lib/python3.9/site-packages/picamera2/picamera2.py", line 21, in <module>
from picamera2.encoders import Encoder, H264Encoder, MJPEGEncoder, Quality
File "/home/pi/.local/lib/python3.9/site-packages/picamera2/encoders/__init__.py", line 3, in <module>
from .jpeg_encoder import JpegEncoder
File "/home/pi/.local/lib/python3.9/site-packages/picamera2/encoders/jpeg_encoder.py", line 3, in <module>
import simplejpeg
File "/home/pi/.local/lib/python3.9/site-packages/simplejpeg/__init__.py", line 1, in <module>
from ._jpeg import encode_jpeg
File "simplejpeg/_jpeg.pyx", line 1, in init simplejpeg._jpeg
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 48 from C header, got 40 from PyObject

Hello, I'm very impressed with this thread.

Regarding the options parsing, It seems that option variable has to be created in the context of Libcamera Encoder(). ( I'm learning and my C++ is rusty so I apologize in advance)

The following code snipet works for me:
<code>
#include <string>
#include <sstream>
#include <vector>

using namespace std;

vector<string> split(string str, char delimiter) {
vector<string> out;
stringstream ss(str);
string token;
while(getline(ss, token, delimiter)) {
out.push_back(token);
}
return out;
}

int getArgv(char** argv, string str) {
vector<string> splitStr = split(str, ' ');
for (long unsigned int i = 0; i < splitStr.size(); i++) {
argv = new char[splitStr.length() + 1];
strcpy(argv, splitStr.c_str());
}
return splitStr.size();
}


INDILibCamera::INDILibCamera()
{
// char str1[] = "-n -t 0 -o -\n --awbgains 1,1 --immediate -v 2";
char str1[] = "-n -t 0 -o -\n --awbgains 1,1 -v 2 --width 1920 --height 1080";
char *argv[64] = {};
int argc = getArgv(argv, str1);


setVersion(LIBCAMERA_VERSION_MAJOR, LIBCAMERA_VERSION_MINOR);
signal(SIGBUS, default_signal_handler);
//m_StillApp.reset(new LibcameraApp(std::make_unique<StillOptions>()));

LibcameraEncoder *lib_cam_enc = new LibcameraEncoder();
VideoOptions *options = lib_cam_enc->GetOptions();


if (options->Parse(argc, argv))
{
if (options->verbose >= 2)
options->Print();
}


m_StillApp.reset(lib_cam_enc);
m_VideoApp.reset(m_StillApp.get());
}
</code>

Note: Because of my rusty c++, I used ChatGPT to create functions getArgv() and split(). Is that ok?

@ANJO
It seems some Python dependencies are broken. I installed pycamera2 with "sudo apt-get" and not with pip:
The "simplejpeg" library that fails in your system comes from package "python3-simplejpeg" on my Raspberry Pi. Maybe you need to reinstall it too with "sudo apt-get". Maybe you will also need an upgrade/update of the whole system.

I do not think anyone can mind ChatGPT or other AI, I even use Copilot officially in my work. This is the future, better get used to it before rather than later

I got it to work by commenting out a few imports and places. But yeah, it's probably pip vs apt. In particular as I use pip with venv.

OK, the most recent commit does:

- fix dng files
- fix fits processing
- added "gain" control
- added bits per pixel ( in image info
- should set bounds correctly (can't subframe yet, though, as I'm not sure what should be set to what)
- consolidates setting of options
- replace use of mmap when putting together the image, this fixes a crash that will occur randomly

Issues and TODOs:
- FITS files are still very dark, no idea why.
- gain apparently only works with jpeg (still and stream), not raw files. At least it doesn't show any changes
- for the correct "main" sizes, we should probably have a selection
- for subframes I think we need to use "rio" in the 0...1 range, but I'm not sure on what base
- there should be brightness etc controls, like in v4l2 but the creation there seems to be very dynamic. not sure if this applies here, too.

In general, I found playing with the command line apps pretty inconsistent.

On the bright side, the IMX290 seems to have 7s exposures which is a lot more than the usb imx 290 cam has (156ms or so, the rest with stacking).

Awesome progress Anjo! Do you think you can submit a PR now to INDI so that we can perhaps make this available for more users to test?

@Jasem can you give me some pointers on how the new properties are supposed to be used?

Here you go, but I won't be doing cosmetics just yet. There's just too much stuff I'm not sure if it's ok or not.

github.com/indilib/indi-3rdparty/pull/697

Here's some more properties, I guess this can be used to create them dynamically. I think it's "low,high,current":

>>> pprint.pprint(picam2.camera_controls)
{'AeConstraintMode': (0, 3, 0),
'AeEnable': (False, True, None),
'AeExposureMode': (0, 3, 0),
'AeMeteringMode': (0, 3, 0),
'AnalogueGain': (1.0, 16.0, None),
'AwbEnable': (False, True, None),
'AwbMode': (0, 7, 0),
'Brightness': (-1.0, 1.0, 0.0),
'ColourCorrectionMatrix': (-16.0, 16.0, None),
'ColourGains': (0.0, 32.0, None),
'Contrast': (0.0, 32.0, 1.0),
'ExposureTime': (0, 66666, None),
'ExposureValue': (-8.0, 8.0, 0.0),
'FrameDurationLimits': (33333, 120000, None),
'NoiseReductionMode': (0, 4, 0),
'Saturation': (0.0, 32.0, 1.0),
'ScalerCrop': (libcamera.Rectangle(0, 0, 0, 0),
libcamera.Rectangle(65535, 65535, 65535, 65535),
libcamera.Rectangle(0, 0, 0, 0)),
'Sharpness': (0.0, 16.0, 1.0)}

You can create them using XML skeleton file or directly in the source using INDI::PropertyNumber. I you have static number of properties that will be always there then you can declare in the header INDI::PropertyNumber ControlsNP {10} for example for 10 controls and then in initProperties you define them