OTGW hangs with WDT reset!

This Forum is about the Opentherm gateway (OTG) from Schelte

Moderator: hvxl

OTGW hangs with WDT reset!

Postby mike7 » Sat Jun 18, 2016 5:57 am

Schelte,

OTGW hung after WDT reset, no message traffic logged by otmonitor.
Code: Select all
18:42:53.747990  T00050000  Read-Data   Application-specific flags: 00000000 0
18:42:53.915467  BC0050000  Read-Ack    Application-specific flags: 00000000 0
18:42:54.747950  T00110000  Read-Data   Relative modulation level: 0.00
18:42:54.928987  BC0110000  Read-Ack    Relative modulation level: 0.00
18:42:55.643704  AFOpenTherm Gateway 4.2.5
18:42:55.653292  WDT reset!
18:43:05.823705  Command (via websocket from 172.17.0.3:33640): SR=10:42,0
18:43:07.762148  Command (via websocket from 172.17.0.3:33644): UI=10
18:43:09.792581  Command (via websocket from 172.17.0.3:33646): SR=48:65,35
18:43:11.790784  Command (via websocket from 172.17.0.3:33648): UI=48
18:43:16.800776  Command (via websocket from 172.17.0.3:33652): SR=49:85,30
18:43:18.799805  Command (via websocket from 172.17.0.3:33656): UI=49
20:30:00.122980  Command (via websocket from 172.17.0.3:36216): SW=50
23:30:00.369659  Command (via websocket from 172.17.0.3:40532): HW=0


I've got timeout response to commands.

Unfortunately otgw is on remote location and I can't check connection between boiler and thermostat.
As I know LED "Transmit.." was off, LED "Trouble" was on.
After power recycle I've got:
Code: Select all
08:53:01.060446  00000000000000000110000
08:53:01.084953  00enTherm Gateway 4.2.5
08:53:01.097237  WDT reset!
08:53:01.139930  00000000000000000110000
08:53:01.167578  00enTherm Gateway 4.2.5
08:53:01.179653  WDT reset!
08:53:08.205454  0
0.............................................................................................................
..............................................................................................................
..................................................................OpenTherm Gateway 4.2.5
08:53:08.255320  B40001300  Read-Ack    Status: 00010011 00000000
08:53:09.108236  T10010000  Write-Data  Control setpoint: 0.00
08:53:09.288105  BD0010000  Write-Ack   Control setpoint: 0.00
08:53:10.108401  T00050000  Read-Data   Application-specific flags: 00000000 0
08:53:10.282108  BC0050000  Read-Ack    Application-specific flags: 00000000 0
08:53:11.108387  T00110000  Read-Data   Relative modulation level: 0.00
08:53:11.294532  BC0110000  Read-Ack    Relative modulation level: 0.00
08:53:12.108181  T80001300  Read-Data   Status: 00010011 00000000
08:53:12.289122  B40001300  Read-Ack    Status: 00010011 00000000


What is WDT reset, is it watchdog? If yes what is timeout?
I don't understand why I've got 2 resets after power recycle.

Thank you for your work and support
mike7
Member
Member
 
Posts: 68
Joined: December 2013

Re: OTGW hangs with WDT reset!

Postby hvxl » Sat Jun 18, 2016 9:11 am

"WDT reset" indeed means that the watchdog timer kicked in. The timeout for the watchdog timer is 16.38 ms. Beware that after a watchdog reset, the gateway boots into monitor mode.

Normally a watchdog timer reset indicates a firmware bug. But this firmware has been running for months in hundreds of gateways without problems and you don't seem to be doing anything unusual. So the most likely cause of the reset is a flaky bit in the eeprom.

Judging by your last piece of log, the gateway is running again. But as mentioned, that's probably in monitor mode (check with PR=M). Try to switch it back to gateway mode (GW=1). If you get another WDT reset, I would suggest to reflash the firmware.
Schelte
hvxl
Senior Member
Senior Member
 
Posts: 1026
Joined: June 2010

Re: OTGW hangs with WDT reset!

Postby mike7 » Sat Jun 18, 2016 9:54 am

Strange, there were no messages after "WDT reset", I hope otmonitor should log messages when GW in monitor mode.
Before power recycling I tried GW=R command without success. Boiler problem LED also was on ...

Looks like I need to reflash firmware.
mike7
Member
Member
 
Posts: 68
Joined: December 2013

Re: OTGW hangs with WDT reset!

Postby sincze » Fri Sep 08, 2017 8:24 am

For some reason I now experience the following:

Code: Select all
2017-09-08 06:09:29.401  OTGW: OpenTherm Gateway 4.2.5
2017-09-08 06:09:29.401  OTGW: WDT reset!


Every day this happens at random times. I already checked the wiring.
THe OTGW has been running fine for over a year now. This started to happen exactly AFTER I had the MAINTENANCE man fix a leakage in the Heating System so I guessed he switched it off.
Normally before maintenance I power down the OTGW and then the Heating system. I think now only the heating system was switched off.

To fix it. I need to disconnect the power from OTGW and reconnect it. Than it starts running again for x hours.
I have no error messages 01/02/03/04 in the OT Monitor. Maybe I should have the monitor run for several days to see if I can catch an error.

Seems I had this issue before.... and it solved itself for some reason...
domoticaforum.eu/viewtopic.php?f=70& ... set#p77767


Suggestions are welcome.
sincze
Starting Member
Starting Member
 
Posts: 32
Joined: June 2014
Location: Breda

Re: OTGW hangs with WDT reset!

Postby sincze » Wed Sep 20, 2017 10:49 am

So according to everything I could find a watchdog RESET could also mean a problem with the firmware.

As the incident started after technician looked at my Ferroli without removing the power from OTGW I did the following:
- Upload diagnostic.hex firmware
- Walked through the tests, all seems okay. Checked reference voltage.
- Upload 4.2.5 firmware. (in addition now all previously stored AA commands / mistakes are removed from EEPROM).

The system is now running for over 24 hours.
I'll add the missing values using AA commands later as my Ferroli Megadens supports the values and shows them in the OTGW monitor nicely.
sincze
Starting Member
Starting Member
 
Posts: 32
Joined: June 2014
Location: Breda

Re: OTGW hangs with WDT reset!

Postby sincze » Mon Sep 25, 2017 11:54 am

Unfortunately this did not work. Still experiening a Watchdog.
Only a powercycle brings life back to the OTGW.
Suggestions are welcome. So to recap. The gateway is working for hours almost 24 hours. It stops showing the WDT reset warning. No error 01,02,03 or 04. Reference value should be okay. Also confirmed by diagnostic firmware.
sincze
Starting Member
Starting Member
 
Posts: 32
Joined: June 2014
Location: Breda

Re: OTGW hangs with WDT reset!

Postby sincze » Thu Sep 28, 2017 8:35 am

Well just a brief update.
Just Modified the gateway a bit with some fancy parts to see what is going on.

Add some PIN headers
OTGW-Pins-1.jpg
Added Pin Headers to the mainboard
OTGW-Pins-1.jpg (43.23 KiB) Viewed 834 times

Added RCA connectors
OTGW-Connectors-1.jpg
RCA Connectors for better connection and easy mount/unmount
OTGW-Connectors-1.jpg (38.16 KiB) Viewed 834 times

Add some LIGHTS
OTGW-Lights-1.jpg
Added lights to the cover
OTGW-Lights-1.jpg (24.94 KiB) Viewed 834 times



In this case now I can actually see what is happening.
The following occurs.

The FERROLI boiler tends to reboot itself sometimes. ( I can see and hear this, display lights up and it starts it's automatic 'ontluchting' procedure).
After completion the boiler continues to work. However now with the communication leds attached to the OTGW I can see that sometimes when this happens the OTGW stops flashing communication LEDS and raises the WDT Reset in the log.
I've attached a reset button to the OTGW and once pressed the OTGW continues operation as normal

As I have no 01/02/03/04 error messages I must assume the Firmware contains some sort of error/bug.
Can this 'time-out' for rebooting boilers perhaps be implemented to that it stops raising the WDT Reset. Error message?

Many many thanks.
sincze
Starting Member
Starting Member
 
Posts: 32
Joined: June 2014
Location: Breda

Re: OTGW hangs with WDT reset!

Postby hvxl » Sun Oct 08, 2017 3:10 pm

So sometimes the reboot of the Ferroli boiler causes the OTGW to become stuck (leading to a WDT reset), but not always? If you can provide a couple of logs of both situations, we may be able to determine what may be the trigger for the problems. Also statistics may be helpful. What's the ratio of troublesome versus trouble-free reboots?
Schelte
hvxl
Senior Member
Senior Member
 
Posts: 1026
Joined: June 2010

Re: OTGW hangs with WDT reset!

Postby sincze » Mon Oct 09, 2017 9:14 am

Excellent I will start logging.
At the moment as I now have the pic reset completely automated I have 7 pic resets a day. (I received a notification with the actual count of that day)
It will reset the pic if no data was received for 15 minutes.
Just post the logs in here is okay as attachment?
Any specific log settings required?

Regarding the ratio it is hard to say as it also happens in the middle of the night. And when I am not home so I can't hear the boiler reset actually happening. With 7 pic resets a day that seems a lot of times for boiler resets. However I just don't know that exact answer. I can set an additional alarm.. If no data was received for 1 minute and I am home. Run to the boiler.
sincze
Starting Member
Starting Member
 
Posts: 32
Joined: June 2014
Location: Breda

Re: OTGW hangs with WDT reset!

Postby sincze » Wed Oct 11, 2017 11:13 pm

As the log file was a bit to big I added it to a google drive.
https://drive.google.com/file/d/0B1iXMF ... sp=sharing

Anything we can use in it??

I'll keep the log running.
sincze
Starting Member
Starting Member
 
Posts: 32
Joined: June 2014
Location: Breda

Re: OTGW hangs with WDT reset!

Postby sincze » Thu Oct 19, 2017 10:54 am

I've just added another log file (12 resets in a day).
Is the log in the corret format?

Unfortunately I am unable to determine the root cause.
The auto PIC reset however brings the device back online. :lol: but then again. Maybe it can be solved in the firmware as well.
the WDT reset! could be an auto-reset of the PIC :)

drive.google.com/file/d/0B1iXMFkXDcbJYU ... sp=sharing

I'll add a warning to my monitor so that in case I did not receive an update I will run to the boiler to see if I can find something suspicious before it resets the PIC.

This simple relais is my PIC reset it is operated by a Mysensors Node. Currently in "Try-Out" Position aka test-setup. :)

PIC Reset.jpeg
PIC Reset
PIC Reset.jpeg (63.75 KiB) Viewed 496 times
sincze
Starting Member
Starting Member
 
Posts: 32
Joined: June 2014
Location: Breda

Re: OTGW hangs with WDT reset!

Postby sincze » Sat Nov 11, 2017 10:56 am

As I noticed the OTGW PIC hangs as well without a restart of the boiler I tried a spare OTGW that I borrowed from a friend.
Just to rule out if it is my own OTGW that is causing all this
OTGW - Spare-r.jpg
Spare Parts
OTGW - Spare-r.jpg (20.67 KiB) Viewed 228 times


This borrowed 'beast' still has OTGW firmware 4.2.3.
I only had to mount 2 RCA plugs that I had somewhere to make a connection with the existing wiring.
The boiler has been running overnight without any problems. Normally the gateway would be have to reset between 7-20 times a day,,, (As it was not processing any data no led lights blinking until a reset)

Today I added a few commands using AA this morning to have the spare part providing me exactly the same data as my OTGW 4.2.5 was providing me.
Let's see what happens. As the 4.2.5. is working without any error 01/02/03/04 for a while and randomly hangs.. What would be the most obvious part to look for that could be faulty?
As the WDT does not provide the 'cause' of hanging.

Suggestions are welcome.
sincze
Starting Member
Starting Member
 
Posts: 32
Joined: June 2014
Location: Breda

Re: OTGW hangs with WDT reset!

Postby sincze » Sun Nov 12, 2017 10:01 pm

The spare OTGW was working without any issues.

I reflashed the 'production' OTGW with the same version. Added the values I wanted to see in otmonitor with 'AA=xx".

Reconnected the cables and the OTGW is working now over 26 hours without a reset.

Well what do you know... :) finally solved ?? I will let you know next week.
sincze
Starting Member
Starting Member
 
Posts: 32
Joined: June 2014
Location: Breda


Return to Opentherm Gateway Forum

Who is online

Users browsing this forum: No registered users and 1 guest