OTGW hangs with WDT reset!

This Forum is about the Opentherm gateway (OTGW) from Schelte

Moderator: hvxl

mike7
Member
Member
Posts: 102
Joined: Mon Dec 02, 2013 8:45 am

OTGW hangs with WDT reset!

Post by mike7 »

Schelte,

OTGW hung after WDT reset, no message traffic logged by otmonitor.

Code: Select all

18:42:53.747990  T00050000  Read-Data   Application-specific flags: 00000000 0
18:42:53.915467  BC0050000  Read-Ack    Application-specific flags: 00000000 0
18:42:54.747950  T00110000  Read-Data   Relative modulation level: 0.00
18:42:54.928987  BC0110000  Read-Ack    Relative modulation level: 0.00
18:42:55.643704  AFOpenTherm Gateway 4.2.5
18:42:55.653292  WDT reset!
18:43:05.823705  Command (via websocket from 172.17.0.3:33640): SR=10:42,0
18:43:07.762148  Command (via websocket from 172.17.0.3:33644): UI=10
18:43:09.792581  Command (via websocket from 172.17.0.3:33646): SR=48:65,35
18:43:11.790784  Command (via websocket from 172.17.0.3:33648): UI=48
18:43:16.800776  Command (via websocket from 172.17.0.3:33652): SR=49:85,30
18:43:18.799805  Command (via websocket from 172.17.0.3:33656): UI=49
20:30:00.122980  Command (via websocket from 172.17.0.3:36216): SW=50
23:30:00.369659  Command (via websocket from 172.17.0.3:40532): HW=0
I've got timeout response to commands.

Unfortunately otgw is on remote location and I can't check connection between boiler and thermostat.
As I know LED "Transmit.." was off, LED "Trouble" was on.
After power recycle I've got:

Code: Select all

08:53:01.060446  00000000000000000110000
08:53:01.084953  00enTherm Gateway 4.2.5
08:53:01.097237  WDT reset!
08:53:01.139930  00000000000000000110000
08:53:01.167578  00enTherm Gateway 4.2.5
08:53:01.179653  WDT reset!
08:53:08.205454  0
0.............................................................................................................
..............................................................................................................
..................................................................OpenTherm Gateway 4.2.5
08:53:08.255320  B40001300  Read-Ack    Status: 00010011 00000000
08:53:09.108236  T10010000  Write-Data  Control setpoint: 0.00
08:53:09.288105  BD0010000  Write-Ack   Control setpoint: 0.00
08:53:10.108401  T00050000  Read-Data   Application-specific flags: 00000000 0
08:53:10.282108  BC0050000  Read-Ack    Application-specific flags: 00000000 0
08:53:11.108387  T00110000  Read-Data   Relative modulation level: 0.00
08:53:11.294532  BC0110000  Read-Ack    Relative modulation level: 0.00
08:53:12.108181  T80001300  Read-Data   Status: 00010011 00000000
08:53:12.289122  B40001300  Read-Ack    Status: 00010011 00000000
What is WDT reset, is it watchdog? If yes what is timeout?
I don't understand why I've got 2 resets after power recycle.

Thank you for your work and support
hvxl
Senior Member
Senior Member
Posts: 1965
Joined: Sat Jun 05, 2010 11:59 am
Contact:

Re: OTGW hangs with WDT reset!

Post by hvxl »

"WDT reset" indeed means that the watchdog timer kicked in. The timeout for the watchdog timer is 16.38 ms. Beware that after a watchdog reset, the gateway boots into monitor mode.

Normally a watchdog timer reset indicates a firmware bug. But this firmware has been running for months in hundreds of gateways without problems and you don't seem to be doing anything unusual. So the most likely cause of the reset is a flaky bit in the eeprom.

Judging by your last piece of log, the gateway is running again. But as mentioned, that's probably in monitor mode (check with PR=M). Try to switch it back to gateway mode (GW=1). If you get another WDT reset, I would suggest to reflash the firmware.
Schelte
mike7
Member
Member
Posts: 102
Joined: Mon Dec 02, 2013 8:45 am

Re: OTGW hangs with WDT reset!

Post by mike7 »

Strange, there were no messages after "WDT reset", I hope otmonitor should log messages when GW in monitor mode.
Before power recycling I tried GW=R command without success. Boiler problem LED also was on ...

Looks like I need to reflash firmware.
sincze
Starting Member
Starting Member
Posts: 47
Joined: Sat Jun 14, 2014 7:18 pm
Location: Breda

Re: OTGW hangs with WDT reset!

Post by sincze »

For some reason I now experience the following:

Code: Select all

2017-09-08 06:09:29.401  OTGW: OpenTherm Gateway 4.2.5
2017-09-08 06:09:29.401  OTGW: WDT reset!
Every day this happens at random times. I already checked the wiring.
THe OTGW has been running fine for over a year now. This started to happen exactly AFTER I had the MAINTENANCE man fix a leakage in the Heating System so I guessed he switched it off.
Normally before maintenance I power down the OTGW and then the Heating system. I think now only the heating system was switched off.

To fix it. I need to disconnect the power from OTGW and reconnect it. Than it starts running again for x hours.
I have no error messages 01/02/03/04 in the OT Monitor. Maybe I should have the monitor run for several days to see if I can catch an error.

Seems I had this issue before.... and it solved itself for some reason...
domoticaforum.eu/viewtopic.php?f=70&amp ... set#p77767


Suggestions are welcome.
sincze
Starting Member
Starting Member
Posts: 47
Joined: Sat Jun 14, 2014 7:18 pm
Location: Breda

Re: OTGW hangs with WDT reset!

Post by sincze »

So according to everything I could find a watchdog RESET could also mean a problem with the firmware.

As the incident started after technician looked at my Ferroli without removing the power from OTGW I did the following:
- Upload diagnostic.hex firmware
- Walked through the tests, all seems okay. Checked reference voltage.
- Upload 4.2.5 firmware. (in addition now all previously stored AA commands / mistakes are removed from EEPROM).

The system is now running for over 24 hours.
I'll add the missing values using AA commands later as my Ferroli Megadens supports the values and shows them in the OTGW monitor nicely.
sincze
Starting Member
Starting Member
Posts: 47
Joined: Sat Jun 14, 2014 7:18 pm
Location: Breda

Re: OTGW hangs with WDT reset!

Post by sincze »

Unfortunately this did not work. Still experiening a Watchdog.
Only a powercycle brings life back to the OTGW.
Suggestions are welcome. So to recap. The gateway is working for hours almost 24 hours. It stops showing the WDT reset warning. No error 01,02,03 or 04. Reference value should be okay. Also confirmed by diagnostic firmware.
sincze
Starting Member
Starting Member
Posts: 47
Joined: Sat Jun 14, 2014 7:18 pm
Location: Breda

Re: OTGW hangs with WDT reset!

Post by sincze »

Well just a brief update.
Just Modified the gateway a bit with some fancy parts to see what is going on.

Add some PIN headers
Added Pin Headers to the mainboard
Added Pin Headers to the mainboard
OTGW-Pins-1.jpg (43.23 KiB) Viewed 14666 times
Added RCA connectors
RCA Connectors for better connection and easy mount/unmount
RCA Connectors for better connection and easy mount/unmount
OTGW-Connectors-1.jpg (38.16 KiB) Viewed 14666 times
Add some LIGHTS
Added lights to the cover
Added lights to the cover
OTGW-Lights-1.jpg (24.94 KiB) Viewed 14666 times

In this case now I can actually see what is happening.
The following occurs.

The FERROLI boiler tends to reboot itself sometimes. ( I can see and hear this, display lights up and it starts it's automatic 'ontluchting' procedure).
After completion the boiler continues to work. However now with the communication leds attached to the OTGW I can see that sometimes when this happens the OTGW stops flashing communication LEDS and raises the WDT Reset in the log.
I've attached a reset button to the OTGW and once pressed the OTGW continues operation as normal

As I have no 01/02/03/04 error messages I must assume the Firmware contains some sort of error/bug.
Can this 'time-out' for rebooting boilers perhaps be implemented to that it stops raising the WDT Reset. Error message?

Many many thanks.
hvxl
Senior Member
Senior Member
Posts: 1965
Joined: Sat Jun 05, 2010 11:59 am
Contact:

Re: OTGW hangs with WDT reset!

Post by hvxl »

So sometimes the reboot of the Ferroli boiler causes the OTGW to become stuck (leading to a WDT reset), but not always? If you can provide a couple of logs of both situations, we may be able to determine what may be the trigger for the problems. Also statistics may be helpful. What's the ratio of troublesome versus trouble-free reboots?
Schelte
sincze
Starting Member
Starting Member
Posts: 47
Joined: Sat Jun 14, 2014 7:18 pm
Location: Breda

Re: OTGW hangs with WDT reset!

Post by sincze »

Excellent I will start logging.
At the moment as I now have the pic reset completely automated I have 7 pic resets a day. (I received a notification with the actual count of that day)
It will reset the pic if no data was received for 15 minutes.
Just post the logs in here is okay as attachment?
Any specific log settings required?

Regarding the ratio it is hard to say as it also happens in the middle of the night. And when I am not home so I can't hear the boiler reset actually happening. With 7 pic resets a day that seems a lot of times for boiler resets. However I just don't know that exact answer. I can set an additional alarm.. If no data was received for 1 minute and I am home. Run to the boiler.
sincze
Starting Member
Starting Member
Posts: 47
Joined: Sat Jun 14, 2014 7:18 pm
Location: Breda

Re: OTGW hangs with WDT reset!

Post by sincze »

As the log file was a bit to big I added it to a google drive.
https://drive.google.com/file/d/0B1iXMF ... sp=sharing

Anything we can use in it??

I'll keep the log running.
sincze
Starting Member
Starting Member
Posts: 47
Joined: Sat Jun 14, 2014 7:18 pm
Location: Breda

Re: OTGW hangs with WDT reset!

Post by sincze »

I've just added another log file (12 resets in a day).
Is the log in the corret format?

Unfortunately I am unable to determine the root cause.
The auto PIC reset however brings the device back online. :lol: but then again. Maybe it can be solved in the firmware as well.
the WDT reset! could be an auto-reset of the PIC :)

drive.google.com/file/d/0B1iXMFkXDcbJYU ... sp=sharing

I'll add a warning to my monitor so that in case I did not receive an update I will run to the boiler to see if I can find something suspicious before it resets the PIC.

This simple relais is my PIC reset it is operated by a Mysensors Node. Currently in "Try-Out" Position aka test-setup. :)
PIC Reset
PIC Reset
PIC Reset.jpeg (63.75 KiB) Viewed 14327 times
sincze
Starting Member
Starting Member
Posts: 47
Joined: Sat Jun 14, 2014 7:18 pm
Location: Breda

Re: OTGW hangs with WDT reset!

Post by sincze »

As I noticed the OTGW PIC hangs as well without a restart of the boiler I tried a spare OTGW that I borrowed from a friend.
Just to rule out if it is my own OTGW that is causing all this
Spare Parts
Spare Parts
OTGW - Spare-r.jpg (20.67 KiB) Viewed 14059 times
This borrowed 'beast' still has OTGW firmware 4.2.3.
I only had to mount 2 RCA plugs that I had somewhere to make a connection with the existing wiring.
The boiler has been running overnight without any problems. Normally the gateway would be have to reset between 7-20 times a day,,, (As it was not processing any data no led lights blinking until a reset)

Today I added a few commands using AA this morning to have the spare part providing me exactly the same data as my OTGW 4.2.5 was providing me.
Let's see what happens. As the 4.2.5. is working without any error 01/02/03/04 for a while and randomly hangs.. What would be the most obvious part to look for that could be faulty?
As the WDT does not provide the 'cause' of hanging.

Suggestions are welcome.
sincze
Starting Member
Starting Member
Posts: 47
Joined: Sat Jun 14, 2014 7:18 pm
Location: Breda

Re: OTGW hangs with WDT reset!

Post by sincze »

The spare OTGW was working without any issues.

I reflashed the 'production' OTGW with the same version. Added the values I wanted to see in otmonitor with 'AA=xx".

Reconnected the cables and the OTGW is working now over 26 hours without a reset.

Well what do you know... :) finally solved ?? I will let you know next week.
hvxl
Senior Member
Senior Member
Posts: 1965
Joined: Sat Jun 05, 2010 11:59 am
Contact:

Re: OTGW hangs with WDT reset!

Post by hvxl »

The changes between version 4.2.3 and 4.2.5 are very small. I don't see anything in there that seems like it could have introduced a reason for a WDT reset.
Schelte
sincze
Starting Member
Starting Member
Posts: 47
Joined: Sat Jun 14, 2014 7:18 pm
Location: Breda

Re: OTGW hangs with WDT reset!

Post by sincze »

hvxl wrote:The changes between version 4.2.3 and 4.2.5 are very small. I don't see anything in there that seems like it could have introduced a reason for a WDT reset.
Me neither. However it could be the PIC content got corrupted?
Even after reflashing with the 'do not keep the options' ??

The only thing I changed was:
- Get Spare OTGW...
- Attach spare to Boiler.. spare was running 4.2.3. and it did not crash,

Reflashed my own gateway with 4.2.3 using a raspberry Pi in stead of otmonitor on windows via a ser2lan connection.
And the thing has been running.
Only 1 or 2 automated resets in a week now.

I'm thinking about reflashing to 4.2.5.
Only thing i am currently looking at is:

Code: Select all

2017-12-03 20:36:18.546 OTGW: PR: BV
2017-12-03 20:36:18.645 (GW OpenTherm) Temp (Max. Vloerverwarming)
2017-12-03 20:36:18.736 (GW OpenTherm) General/Percentage (Maximum Relative Modulation Level)
2017-12-03 20:36:18.826 (GW OpenTherm) Thermostat (Room Setpoint)
2017-12-03 20:36:18.941 (GW OpenTherm) General/Percentage (Relative modulation level)
2017-12-03 20:36:19.056 (GW OpenTherm) General/Pressure (CH Water Pressure)
2017-12-03 20:36:19.175 (GW OpenTherm) Temp (Room Temperature)
2017-12-03 20:36:19.296 (GW OpenTherm) Temp (Boiler Water Temperature)
2017-12-03 20:36:19.403 (GW OpenTherm) Temp (DHW Temperature)
2017-12-03 20:36:19.534 (GW OpenTherm) Temp (OTGW - Outside Temperature)
2017-12-03 20:36:19.660 (GW OpenTherm) Temp (Return Water Temperature)
2017-12-03 20:36:19.766 (GW OpenTherm) Thermostat (DHW Setpoint)
2017-12-03 20:36:19.869 (GW OpenTherm) Thermostat (Max_CH Water Setpoint)
If I could just figure out what setting was causing:

Code: Select all

2017-12-03 20:36:18.546 OTGW: PR: BV
That would be ideal.
Post Reply

Return to “Opentherm Gateway Forum”