×

INDI Library v2.0.6 is Released (02 Feb 2024)

Bi-monthly release with minor bug fixes and improvements

Weather radio don't want connect

  • Posts: 108
  • Thank you received: 4

Replied by Gunter on topic Rain sensor

Hi there,

just for the records, there is another GitHub project by hdiessner.


github.com/hdiessner/Allskycam-heating
The following user(s) said Thank You: Adrian
3 years 2 months ago #65469

Please Log in or Create an account to join the conversation.

  • Posts: 89
  • Thank you received: 16
I am having some trouble with my new build.

I have bought the anenometer and it works nicely on my protoboard with the other sensors, BME280, TSL2591 and MLX90614.

I used the sketches from cactus.io to check the anenometer and they read nicely.

My oled is an SH1106, which works nicely with u8g2, so no luck until I get an SSD1306

With version 1.10 and 1.11 from github I can't compile the sketch without #def USE_OLED due to some missing preprocessser directives
in the ino, there is also a typo in the JsonDocument declaration at the start of getSensorData, see patch


My three I2C sensors read nicely with weatherradio unless I use the anenometer.

I thought this might be due to my hardware setup so I wrote another script and the anenometer works (vane and windspeed) with the sensors
so I am trying to figure out where the problem is.

In weatherradio if I run the anenometer on its own
{
"Davis Anemometer": {
"init": true,
"direction": 60,
"avg speed": 4.924791,
"min speed": 0.252556,
"max speed": 11.95702,
"rotations": 130
}
}

If I add an I2C sensor (BME280)
Weather Radio V 1.10 (should say 1.11)
{
"Davis Anemometer": {
"init": true,
"direction": 98,
"avg speed": 0,
"min speed": 0,
"max speed": 0,
"rotations": 0
},
"BME280": {
"init": true,
"Temp": 16.58,
".2793
}
}
Woops what is that ".2793 ????

Worse with 2 sensors BME and MLX
Weather Radio V 1.10
{
"Davis Anemometer": {
"init": true,
"direction": 104,
"avg speed": 0,
"min speed": 0,
"max speed": 0,
"rotations": 0
},
"BME280": {
"init": true,
"Temp": 16.49,
8.42871
},
"MLX90614": {

}
}
Woops strange character before 8.4

With three sensors
Weather Radio V 1.10
{
"Davis Anemometer": {
"init": true,
"direction": 105,
"avg speed": 0,
"min speed": 0,
⸮⸮W⸮U⸮
⸮⸮7U⸮]Ul"Timing": 1
}
}

An example output with my test script

BME280
T = 16.48 *C
P = 1028.22 hPa
H = 48.18 %

MLX90614
Ambient T = 16.35 *C
Object T = 16.23 *C

TSL2591
[ 33511 ms ] IR: 213 Full: 13960074 Visible: 906 Lux: 2.526531

DAVIS VANE
424 149 SE

DAVIS WIND
60 45.00

{
"BME280": {
"init": true,
"Temp": 16.48,
"Pres": 1028.209,
"Hum": 48.17773
},
"MLX90614": {
"init": true,
"T amb": 16.31,
"T obj": 16.32999
},
"TSL2591": {
"init": true,
"Lux": 2.440772,
"Visible": 874,
"IR": 205,
"Gain": 32,
"Timing": 1
}
}

Not sure if any you have seen this. Are any of you using the anenometer with the i2c sensors connected to the same arduino?

My RG-11 has arrived, but I won't start with that until I have figured this out.

Kind regards

Adrian
3 years 2 months ago #65660
Attachments:

Please Log in or Create an account to join the conversation.

  • Posts: 1185
  • Thank you received: 370
Adrian,
many thanks for reporting this. Version 1.11 is currently unstable and under development, so better not use it in production. I applied your patch, so syntactically everything should be fine.

For the rest, I need to create an appropriate test bread board. Indeed, in my private environment I have one separate Arduino running only the Davis Wind sensor. But this is only due to my installation, since the wind vane sits on the roof top and the weather station is in the garden in a separate installation.

Btw: the INDI service for the rain sensor isn't ready yet, I am currently working on in and am planning to release it with V 1.11.

Cheers
Wolfgang
The following user(s) said Thank You: Adrian
3 years 2 months ago #65673

Please Log in or Create an account to join the conversation.

  • Posts: 16
  • Thank you received: 3
Adrian, I've been trying to diagnose similar issues as I've been adding in my dew heater code. I haven't solved this but my thoughts are that it could be:

1. Hardware problem - I'm not that neat with my soldering etc and the problems sometimes seem intermittent so I wonder whether it's just something particularly on the i2c bus. I've been gradually redoing my setup, but not sure whether this is the problem.

2. JsonDocument issue - I've been trying to understand JsonDocument to understand whether enough space is being allocated because of string usage and whether we're hitting the memory limit of StaticJsonDocument as more sensors are added. It's a confusing library, but I think the answer here is it's okay.

3. Code problems - I've been through the code in detail, whilst there are some issues in the new update like the missing preprocessor lines you identified, and some missing allocation of space in JsonDocuments, everything looks fine and practically I see similar problems with older versions of the code. So I don't think it's anything that's been recently added.

4. Memory limits of the microcontroller - I notice you're using an Uno same as me. My dew heater code is including math.h as does the Davis library, and I've wondered whether that means there's just not enough left with an Uno when it comes round to allocating memory for the Json strings.

It's been very difficult to narrow it down as it's not particularly repeatable and removing code in various places seems to fix the issue. My guess is it's probably memory, but I'm not an experienced C programmer. It's good to know though that it's not just me :)

-Mark
The following user(s) said Thank You: Adrian
3 years 2 months ago #65674

Please Log in or Create an account to join the conversation.

  • Posts: 89
  • Thank you received: 16
Wolfgang and Mark,

I suspect a memory problem. With three I2C sensors and the Davis sensor
Sketch uses 23780 bytes (73%) of program storage space. Maximum is 32256 bytes.
Global variables use 961 bytes (46%) of dynamic memory, leaving 1087 bytes for local variables. Maximum is 2048 byte

This is why it works when I use the Davis on its own, and not when I add more sensors.

I don't understand the Static Json docsize declaration size based upon JSON_OBJECT_SIZE (will have to read up on it), it
can't know what the lenght of the variables are and what strings they will hold, right!
The script creates this then serializes this into a string which is returned.

My w json serialized string is 313 bytes long!
{"Davis Anemometer":{"init":true,"direction":96,"avg speed":0,"min speed":0,"max speed":0,"rotations":0},"BME280":{"init":true,"Temp":12.25,"Pres":1024.765,"Hum":52.88281},"MLX90614":{"init":true,"T amb":12.45001,"T obj":12.91},"TSL2591":{"init":true,"Lux":2.062501,"Visible":799,"IR":211,"Gain":32,"Timing":1}

If I type p I get strange results, but with w it can look right if I fiddle with the docSize.
Weather Radio V 1.10
{"Davis Anemometer":{"init":true,"direction":96,"avg speed":0,"min speed":0,"max speed":0,"rotations":0},"BME280":{"init":true,"Temp":12.25,"Pres":1024.765,"Hum":52.88281},"MLX90614":{"init":true,"T amb":12.45001,"T obj":12.91},"TSL2591":{"init":true,"Lux":2.062501,"Visible":799,"IR":211,"Gain":32,"Timing":1}}
{
  "Davis Anemometer": {
    "init": true,
    "direction": 96,
    "avg speed": 0,
    "min speed": 0,
    "max speed": 0,
    "rotations": 0
  },
  "BME280": {
    "init": true,
    "Temp": 12.25,
    "Pres": 1024.765,
    "Hum": 52.88281
  },
  "MLX90614": {
    "init": true,
    "T amb":
    "Visible": 799,
    "IR"iming": 1
  }
}
Again note that the last line in pretty is taking the start of the IR for the MLX sensor and ending it with the last field of the TSL sensor.

My impression is lack of memory on the microcontroller.

Adrian
3 years 2 months ago #65700

Please Log in or Create an account to join the conversation.

  • Posts: 89
  • Thank you received: 16
Seems docSize needs to include the json part and the string storage.

github.com/bblanchon/ArduinoJson/issues/151

For my 313 byte serial output the doc size is calculated as 152 so it is unlikely to be able to hold the information.
Weather Radio V 1.10
152 <=== Serial.println(docSize)
{
  "Davis Anemometer": {
    "init": true,
    "direction": 87,
    "avg speed": 0,
    "min speed": 0,
    "max speed": 0,
    "rotations": 0
  },
  "BME280": {
    "init": true,
    "Temp": 14.42,
    "Pres": 1024.433,
    "Hum": 50.88184
  },
  "MLX90614": {
    "init": true,
    "T amb": 14.82999,
    "T obj": 14.19
  },
  "TSL2591":<Rni
  }
}
3 years 2 months ago #65704

Please Log in or Create an account to join the conversation.

  • Posts: 1185
  • Thank you received: 370
Ah, interesting, that issue may explain it. I had the same problem with getCurrentConfig()
String getCurrentConfig() {
  const int docSize = JSON_OBJECT_SIZE(7) + // max 7 configurations
                      JSON_OBJECT_SIZE(2) + // DHT sensors
                      JSON_OBJECT_SIZE(3) + // Davis Anemometer
                      JSON_OBJECT_SIZE(1) + // Water sensor
                      JSON_OBJECT_SIZE(2) + // Rain Sensor
                      JSON_OBJECT_SIZE(3) + // WiFi parameters
                      JSON_OBJECT_SIZE(1) + // Arduino
                      JSON_OBJECT_SIZE(3) + // OTA
                      JSON_OBJECT_SIZE(2);  // buffer
  StaticJsonDocument <docSize> doc;
...
As you can see, I simply added an additional buffer of JSON_OBJECT_SIZE(2) - I found it out by trial and error...

Wolfgang
3 years 2 months ago #65710

Please Log in or Create an account to join the conversation.

  • Posts: 16
  • Thank you received: 3
This is the best description I found of how to use the library: arduinojson.org/v6/how-to/determine-the-...of-the-jsondocument/ My reading of this was that if all strings used for keys are const char* then we won't need extra space, otherwise serialisation needs space to copy all the strings.

-Mark
3 years 2 months ago #65712

Please Log in or Create an account to join the conversation.

  • Posts: 89
  • Thank you received: 16
Slight progress

There is a clue to setting the Static JsonDocument in the webpages

arduinojson.org/v6/assistant/

I set mine to 194 to allow for the JSON data and the strings
How to work this out on the fly for different sensor and outputs?

The main problem with the outputting is with serializeJsonPretty, this puts out a very long output with all those nice spaces and ^M at the end.
This seems to overrun the memory which is put into String result="";
I am not sure how the String object looks after its memory, or if you have to reserve memory for it yourself, and free it up afterwards etc.

You can specify how much to output in the serialze functions

I tried to output to a buffer and set a size limit on this
#include <ArduinoJson.h>
#define DESTBUFSIZE 320
char destBuf[DESTBUFSIZE];
 
#ifdef __arm__
// should use uinstd.h to define sbrk but Due causes a conflict
extern "C" char* sbrk(int incr);
#else  // __ARM__
extern char *__brkval;
#endif  // __arm__
 
int freeMemory() {
  char top;
#ifdef __arm__
  return &top - reinterpret_cast<char*>(sbrk(0));
#elif defined(CORE_TEENSY) || (ARDUINO > 103 && ARDUINO != 151)
  return &top - __brkval;
#else  // __arm__
  return __brkval ? &top - __brkval : &top - __malloc_heap_start;
#endif  // __arm__
}
 

In the getSensorData I changed it to use the buffer and permitted length.
void getSensorData(bool pretty) {
 
and
 
  int ret;
 
  destBuf[0] = 0;// Null terminator
 
  if (pretty)
    ret = serializeJsonPretty(weatherDoc, destBuf, DESTBUFSIZE);
  else
    ret = serializeJson(weatherDoc, destBuf, DESTBUFSIZE);
 
  Serial.print("ret :"); Serial.println(ret);
  Serial.print("Free Memory: "); Serial.println(freeMemory());
  Serial.print("DestBuf :"); Serial.println(destBuf);

Then in the w and p commands
    case 'w':
      //Serial.println(getSensorData(false));
      getSensorData(false);
      Serial.println(destBuf);
      break;
    case 'c':
      Serial.println(getCurrentConfig());
      break;
    case 'p':
      //Serial.println(getSensorData(true));
      getSensorData(true);
      Serial.println(destBuf);
      break;

Not sure if this is the best approach. I do remember trying to understand the firmata stuff on the old meteostation and wringing my hands.
3 years 2 months ago #65715

Please Log in or Create an account to join the conversation.

  • Posts: 89
  • Thank you received: 16
Of course the Wemos D1 mini (4Mb) pro (16Mb) is flush with memory compared to the Uno (32K flash and 2K SRAM)

So if a Wemos D1 mini pro is used there will be no shortage of ram compared to the UNO.
3 years 2 months ago #65753

Please Log in or Create an account to join the conversation.

  • Posts: 1185
  • Thank you received: 370
Adrian,
I filled a breadboard with a BME280, a DHT11, a MLX90614, a TSL2591, a RG-11, a Davis Wind sensor and a OLE display. At least on a Wemos D1 mini, serializing works fine:
{
 
  "Davis Anemometer": {
    "init": true,
    "direction": 3,
    "avg speed": 0,
    "min speed": 0,
    "max speed": 0,
    "rotations": 0
  },
 
  "Rain Sensor": {
    "init": true,
    "rainfall": 0,
    "count": 0
  },
 
  "BME280": {
    "init": true,
    "Temp": 21.08,
    "Pres": 974.0237,
    "Hum": 26.83203
  },
 
  "DHT": {
    "init": true,
    "Temp": 19.9,
    "Hum": 36
  },
 
  "MLX90614": {
    "init": true,
    "T amb": 21.01,
    "T obj": 20.55001
  },
 
  "TSL2591": {
    "init": true,
    "Lux": 1.487565,
    "Visible": 505,
    "IR": 108,
    "Gain": 32,
    "Timing": 1
  }
}

Therefore I suspect it is a memory issue or a cabling problem.

Wolfgang
3 years 2 months ago #65806

Please Log in or Create an account to join the conversation.

  • Posts: 89
  • Thank you received: 16
Thanks for looking at that Wolfgang, you are very kind to do all that work.

Do you have an Uno to try with?

Like Mark I doubted my wiring so I had redone it several times before I looked at the code.

Yes, it definitely seems to be due to lack of memory on the UNO.

Options:
- Use bigger memory device, like your Wemos D1
- Limit sensors used
- Spread sensors over several microcontroller boards
- It may be possible to save some memory, so that lower powered devices can be used

The main memory hogs are the JSON calls that are copied back to a large long string. The JSON object is big and the string output to the serial port is larger.
In addition, reading about the String object suggests that it is somewhat notorious for giving memory problems.

I have been looking at the code and I am trying to get rid of the String objects and the ArduinoJSON library.

I see the same information is in three places - the basic structure storing the results for each sensor, the JSON object
and the large character buffer that is built up to send to the serial port.

I have written a shortJSON library to generate the JSON strings. I store these in a character buffer for each sensor (the memory is assigned at the start and continuously reused) I just allocated a little more characters than are needed to accomodate the sensor readings.
At the moment I call the function to update this character buffer every time
when the sensor is updated (I call it after the structure is updated eg mlxData.variable.= mlxreadCAll(object).
This is a little wasteful in processing time but the strings are just there and ready to go when they are requested.

These are just Serial.print(MLXstring); on the calls to w, p etc. from the serial console.

I have these working for davis, mlx, tsl and bme, no problem with funny output or memory shortage so far, so seems stable with less memory usage.

Before I move on to the other sensors I will see if I can get indi to talk to the new code.

I will upload the code to github so you can have a look at it.

Kind regards,

Adrian
3 years 2 months ago #65811

Please Log in or Create an account to join the conversation.

Time to create page: 1.878 seconds