Boiler runaway when removing OTGW power

[fox]

Hello,
I have a Viessmann Vitodens 100-W B1KF-25 boiler.
I use the OTGW in standalone mode.
I like the failsafe design of the CS command (expire after 1min).
I'm also trying on my side to keep that failsafe philosophy upstream (when implementing my MQTT control topic logic).

During my tests, I also tried to remove the power supply from the OTGW (remove USB cable).

After 15sec, an error code shows up on the boiler (95) meaning something like: "You asked me to obey the OpenTherm bus, but you didn't connect any room unit there. Either plug a OT thermostat or change the boiler mode."
My boiler then turns on Central Heating with an arbitrarily fixed control setpoint of 60°C.
After restoring power to the OTGW, the error code goes away and the boiler obeys again the OpenTherm bus (Central Heating it shut off and the setpoint is normal again).

If I understand correctly, it seems this behavior is what the standard wants?

"The boiler unit must support an important installation feature which allows the terminals at the boiler to be short-circuited to simulate a heat demand such as can be done with existing on/off boilers. The boiler unit should interpret the short-circuit as a heat demand within 15secs of the short-circuit being applied."
[...]
"It is allowable that this can implemented by a software-detection method. The software short-circuit condition is defined as a low-voltage state (Vlow) with no valid communications for at least 5 seconds."

I'm surprised because it's talking about short-circuit (i.e. connecting the wires together), but actually, not connecting anything without short-circuit also produces "a low-voltage state with no valid communications".

Or do I misunderstand the standard and it's a boiler bug?

This behavior seems very dangerous.
I understand for the short-circuit (it's also the logic used by OTGW slave interface). But not for unconnected bus or shutdown Master.
Does anyone have suggestions on how to mitigate this risk?
I'm thinking about
- Powering the OTGW from the same "source" than the boiler itself (for example, by plugging the Raspberry Pi power transformer on the same 230V socket as the boiler itself. Raspberry Pi Hub powers OTGW.
That way it's less likely to have the boiler with power and the OTGW without (but still possible unfortunately).
- Making all boiler electricity go through a relay that can be opened remotely. But this requires manual action to 1) notice runaway and 2) cut boiler power. Not very good...

[hvxl]

The OTGW was specifically designed so it will present a high-voltage state (Vhigh) to the boiler when it is powered off. Power is required to pull the line low. Also, "not connecting anything without short-circuit" doesn't produce "a low-voltage state with no valid communications". The boiler is supposed to provide a high enough voltage for the thermostat to be able to limit it to the desired Opentherm level. That means it should be at least 18V. An unpowered OTGW would only pull that down to around 17V. That doesn't qualify as a low-voltage state.

The specs really only specify what should be done in case of a short-circuit/low-voltage state. It doesn't actually say what must be done in case of an open/high-voltage state. So it can be argued that the boiler doesn't violate the specs. Although it seems obvious that it should do the opposite in that case. It definitely looks like your boiler doesn't follow at least the intention of the specs in this situation.

Most boilers combine the opentherm and on/off connections. I understand from one of your other posts that is not the case with your boiler. Maybe that's why it behaves unusual. But it clearly detected that there is a problem. Reacting to that by blasting the heat is a very poor design choice, to put it mildly.

If even an open line still causes the boiler to start heating, there is not much else you can do than somehow power off the boiler in case of problems.

The only real fail-safe solution I can think of that could counteract the bad behavior of your boiler would be a radically different OTGW design. It would have to harness power from the boiler to run the PIC, so it can always send OT messages to the boiler, as long as that is powered up. External power to the OTGW would still be required for the thermostat side, serial communication, LEDs, etc. Achieving optical isolation between the boiler and the USB connector would also be much harder than with the current design.

[fox]

I tested again. Actually, I was wrong.
Yes, the flame turns on when I remove OTGW power, but it's not quite a runaway!

Actually, as I spend most of the time in HW=0 mode (eco), my hot water is often not hot.
Unplugging the OTGW power has the effect of the boiler defaulting back to HW=1 (comfort mode). So the flame turns on, and we can hear the sound of the heat exchanger pump.
I was mislead by the fact that, the boiler flow temperature went up to 60°C (and flow temperature has a shared mission between CH and DHW).
My conclusion last time was also influenced my the fact that all my TRVs were closed.
But this time they were open and I could see that radiators kept cold actually, and flame turns off relatively quickly when DHW water temperature is 50°C.
That's great because if power is ever partially cut in my house, I won't have dangerous issues.


But I noticed: if my serial program is still up, but I just cut OTGW USB power, whenever I plug it back, CH will turn on!
After analysis:
- When gateway is powered up, CH default to 1.
- I always send CS=<current CS wanted> (no matter whether or not CH is activated). I understand the fail-safe design, but I don't really like that when we stop to send CS=, it goes back to zero. Because then, how do we know the real CS? My program always take B messages to determine the real state of the boiler to reflect it to MQTT status topic. If I let the gateway timer expire, it will try to send (an invalid) value of CS in MSgID 1. And because my boiler's minimum setpoint is 20°C, boiler will start reporting 20°C. So my MQTT topic state changes, saying the CS is now 20°C, although I never wanted to modify it. I prefer to continue seeing the original intent (for example, 40°C).
- I correctly repush all the wanted state (from MQTT control topic to OTGW) if I lose MQTT connection when I recover it, but I haven't implemented repush if I lose serial and recover it yet (hence I don't repush that I want CH=0 if OTGW disappears and reappears, but the gateway starts with CH=1 ).

On my side I will fix my code:
- Resend the complete wanted state to the OTGW if it disappeared and reappeared
- Stop to send CS when I don't want heating, and stop to use boiler messages to determine the status of the CS.

How do you think about:
1) Defaulting CH to 0 when booting up in standalone mode (at least if I don't repush the correct CH=1 wanted state, it means I have a bug in my program, but I think it's better for a house to be cold when we want it hot rather than the opposite).
2) Rethinking the way we do about the fail safe mechanism for the CH/CS. It sounds more logical to me to say that we need to repeat "CH=1" every minute, if not gateway switches back to CH=0 (in standalone), or to whatever thermostat want when there is a thermostat. That way we don't have to "fiddle" with the setpoint when it was actually not the intent of anyone to change it.
Couldn't we say that gateway takes ownership of the master CH bit if a CH command was received within the last minute? And if the timer expires, we let the thermostat control the bit again?
I've never tested OTGW with a thermostat so I'm not sure, but as thermostat is also a master, I guess it sends wanted CS regularly over the line, even during the time we take control of the CH master bit.
So as soon as manual CH command has expired, we also immediately know the new CS to apply (the one coming from the thermostat).

[fox]

No, no, no, I waited for longer time. The boiler will eventually turn on the heating!

1) OTGW power is removed
2) Boiler detects there is no thermostat anymore
3) It enables CH and DHW comfort
4) But because hot water always has priority over heating, it first heats the water
5) Once water is hot, it turns off flame
6) And if we wait for 5 more minutes, it will turn on heating pump, radiators really become hot.
7) Boiler temperature falls because as TRVs are open, a lot of cold radiator water enter the boiler
8 ) And guess what... Boiler eventually re-ignite the flame, but this time the 3way vane is in CH position.

So to sum up, the fallback behavior of this boiler is the worst possible.

It's extremely hard to see what it's doing because, when in OT mode, the boiler stops to display the CS and the SW, but displays instead the current flow temperature and the current DHW temperature.

[hvxl]

Your observation is not what is supposed to happen.

• When the gateway is powered up, CH defaults to 1 and CS defaults to 0, which means thermostat controlled. With CS thermostat controlled, the CH setting is irrelevant. The OTGW will always forward the CHenable bit received from the thermostat, or 0 in standalone mode.
• When you send CS commands and then stop, after a minute the gateway will go back to passing the values from the thermostat, or 0 in standalone mode. If the gateway sends an invalid value, then that's a bug.

After a boot the CH setting does not matter because CS is 0. The idea is that when you send a CS command, because you want heating, the CH enable bit (ID0:HB0) also gets set. This avoids confusion about why the boiler does not start heating when you tell it to, by setting a (high) CS. So, especially in standalone mode, send a CS value when you want heating. Send CS=0 when you don't want heating. That way you can leave CH=1.

The normal way to control a boiler is to have a PID controller that regularly calculates the desired control setpoint. If you do this calculation at least once a minute and send a CS command with the result, the timer will not expire. If the PID controller produces a value that should not result in heating (PIDout < Tboiler), send CS=0. It is very unusual to have the control setpoint fixed and then basically control the boiler with the CH enable bit as if it were an on/off boiler.

The gateway currently takes ownership of the master CH bit if a CS command was received within the last minute. And if the timer expires, we let the thermostat control the bit again. As I said, sending a CS value every minute, or more frequently, is almost natural. You may never need to touch the CH setting. So it makes absolutely no sense to me to use the CH command instead of the CS command to restart the timer.

When you use a thermostat, you don't need to know the new CS to apply when the timer expires. The OTGW already goes back to forwarding the control setpoint and CH enable bit from the thermostat. No external intervention is required.

[fox]

You are re-stating how the OTGW currently works. I agree with you that it works like you said, there is no misunderstanding about this part.
I'm just trying to explain why I feel this is not natural.

But I'm only taking the point of view of a pure standalone usage, where the OTGW is the only master in the story, in that case it IS a thermostat itself actually, so I'm never "overriding anything" as I'm always the only thermostat.

There are 3 strange things for me.

1. The fact that OTGW asks my boiler to change its internal CS once I stop emitting CS commands to the OTGW.
2. The fact that I need to repeat CS to enable my heating (repeating CH would be more logical to me).
3. The fact that in standalone mode, CH defaults to one.

1. OTGW asks boiler to set a 0 CS when stopping emitting CS commands

Oct 31 13:45:14 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Sending command 'CS=30.0'
Oct 31 13:45:14 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'CS: 30.00\r\n''
Oct 31 13:45:15 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'B401A3633\r\n''
Oct 31 13:45:15 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'R00210000\r\n''
Oct 31 13:45:16 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'B4021002F\r\n''
Oct 31 13:45:16 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'R100E1400\r\n''
Oct 31 13:45:17 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'BD00E1400\r\n''
Oct 31 13:45:17 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'R80130000\r\n''
Oct 31 13:45:18 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'B40130000\r\n''
Oct 31 13:45:18 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'R00000000\r\n''
Oct 31 13:45:19 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'BC0000000\r\n''
Oct 31 13:45:19 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'R80190000\r\n''
Oct 31 13:45:19 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'B401937E6\r\n''
Oct 31 13:45:20 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'R10011E00\r\n''
Oct 31 13:45:20 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'BD0011E00\r\n''
[...]
Oct 31 13:46:49 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'R00110000\r\n''
Oct 31 13:46:50 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'BC0110000\r\n''
Oct 31 13:46:50 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'R801A0000\r\n''
Oct 31 13:46:51 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'B401A3566\r\n''
Oct 31 13:46:51 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'R00210000\r\n''
Oct 31 13:46:52 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'B4021002F\r\n''
Oct 31 13:46:52 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'R100E1400\r\n''
Oct 31 13:46:53 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'BD00E1400\r\n''
Oct 31 13:46:53 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'R80130000\r\n''
Oct 31 13:46:54 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'B40130000\r\n''
Oct 31 13:46:54 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'R00000000\r\n''
Oct 31 13:46:55 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'BC0000000\r\n''
Oct 31 13:46:55 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'R80190000\r\n''
Oct 31 13:46:56 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'BC0193733\r\n''
Oct 31 13:46:56 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'R10010000\r\n''
Oct 31 13:46:57 raspi1B otgw2mqtt[9959]: otgw2mqtt.otgw - DEBUG - Received frame 'b'BD0011400\r\n''

I can make an analogy with a water tap to try to better convey what I want to say:
A tap water has two independent settings.
- Horizontally, (a kind of CS), we control the temperature of the tap
- Vertically (a kind of CH) we control whether water is flowing or not
I would find it strange if closing the tap water would always additionally reset the temperature setting all to the right.
When we close a water tap, we can keep the previous temperature setting unchanged, we can for example close it AND let it in the middle position.
(CS=40 and CH=0).
This is convenient, because, even if a closed tap set to the right seems to behave the same as a closed tap set to the middle (i.e. no water is flowing anyway), but it's not exactly the same. When an application reads the current state of the closed tap, it reads something that we never intended to set.


2. The fail safe mechanism wants us to repeat CS to maintain Central Heating enabled, instead of repeating CH
This is less important than the first issue. It's just, as a standalone-only user, I would find it more logical to repeat:

CS=40 -> Okay, CS is set to 40 but this doesn't mean we want to enable heating
CH=1 I wan to enable heating
CH=1 Yes, my program is still up, I still want the heating to be on
CH=1 Yes, I still want the heating to be on
CH=1 Yes, I still want the heating to be on
CH=1 Yes, I still want the heating to be on
[...] No News of the user for one minute, we automatically switch back to CH=0

rather than repeating a CS command.
Because the name of the command should correspond to what it's doing: CH=1 reads like "Central Heating Activated".

For me, CS and CH should be completely independent.
- CS should only choose the control point, never turn on of turn off the heating.
- CH should only turn on or off the heating, never change the control point.
CH alone should be enough to turn on the heating, without having to repeat a CS which may be already set to the correct value.

But here, CS alone can stop the heating! (I mean if we stop issuing commands.)

What I propose (I'm only talking of the fail safe mechanism when in standalone mode), is to repeat CH commands.
Once we stop repeating it, CH will automatically become 0.
This feels natural, because, we are manipulating something meaning "Turn on the heating", and if we stop to say this, then the "heating stops".
It's the same "theme". And it has nothing to do with the CS. Don't know if I'm being clear.

3. The fact that in standalone mode, CH defaults to one when booting
I know that Central Heating won't really be activated in the current design, because if we don't emit CS commands, it doesn't heat.
But I would find it more logical that in standalone mode, CH defaults to 0. Because we are going to take control anyway, we will use CH to turn on or off the heating. And we will use CS only to change the water temperature, not to turn on or off the heating (if we implement the change n°2 I propose).


Again, I don't know if what I said in this post has a meaning when a real thermostat is plugged in.
But if we focus in standalone mode only, this is feedback I wanted to make.

You have a more global view, with all the features and usages of the OTGW (especially the use case with 2 buses used at the same time with a real thermostat).
Maybe some things I say don't have meaning in thermostat mode. But I believe they have a meaning in standalone mode.
I know the gateway was initially not designed to be used in standalone mode, this is a feature that you added much later.

[fox]

About this:

No, no, no, I waited for longer time. The boiler will eventually turn on the heating!

1) OTGW power is removed
2) Boiler detects there is no thermostat anymore
3) It enables CH and DHW comfort
4) But because hot water always has priority over heating, it first heats the water
5) Once water is hot, it turns off flame
6) And if we wait for 5 more minutes, it will turn on heating pump, radiators really become hot.
7) Boiler temperature falls because as TRVs are open, a lot of cold radiator water enter the boiler
8 ) And guess what... Boiler eventually re-ignite the flame, but this time the 3way vane is in CH position.

So to sum up, the fallback behavior of this boiler is the worst possible.

Here, I was just talking about a bug in my boiler. When removing OTGW power, my boiler
- sets comfort mode with water to 50°C
- activates central heating with water to 60°C

This is totally unrelated to the OTGW. This is a bug of my boiler.
I have a way to mitigate this but it's not satisfactory, it's a dirty hack: I can configure the old ON/OFF thermostat input (which not the same input as the OT bus on my boiler) to behave as a "boiler lookout demand". Closing that switch forbids the flame to turn on, no matter what.

[hvxl]

I'm just trying to explain why I feel this is not natural.

I can make an analogy with a water tap to try to better convey what I want to say:
A tap water has two independent settings.
- Horizontally, (a kind of CS), we control the temperature of the tap
- Vertically (a kind of CH) we control whether water is flowing or not
I would find it strange if closing the tap water would always additionally reset the temperature setting all to the right.
When we close a water tap, we can keep the previous temperature setting unchanged, we can for example close it AND let it in the middle position.
(CS=40 and CH=0).
This is convenient, because, even if a closed tap set to the right seems to behave the same as a closed tap set to the middle (i.e. no water is flowing anyway), but it's not exactly the same. When an application reads the current state of the closed tap, it reads something that we never intended to set.

I understood what you were saying the first time. What I was trying to explain is that a boiler isn't normally controlled that way. Your analogy to a mixer tap doesn't represent how a central heating system works. It is more like a car, where CS is the gas pedal and CH is the clutch. To control the speed of the car, you adjust the gas. Add more to accelerate, reduce when you approach the desired speed. Only when you don't want to move while the motor is on, you press the clutch. It would not be good if the engine would run at some high revs when you start it, just because that was used previously. Similarly, a central heating system is controlled using CS. Set it higher to increase the temperature, reduce it when the temperature reaches the setpoint. When no heating is desired, switch off CH enable. When the temperature drops below the setpoint, heating is initiated again. But slowly, not blasting at a high level that happened to be used before. Now, with the OTGW there are two ways the CH enable can be switched off: Using the CH=0 command (when CS is running stationary at 10°C or so), or when CS=0 (the engine is switched off).

I will admit that my analogy also doesn't fit exactly, but I hope you can understand the gist. And this is how normal thermostats work. It is also how I expect the OTGW is used when acting as the thermostat, in stand-alone mode. This is why the commands have been designed the way they are. The current way works best for the standard way of controlling a heating system. I will not change that to accommodate your alternative way. That would make it inconvenient for people who control their system the normal way.

[fox]

I understand your analogy, but I don't agree.

When a thermostat uses the outside temperature to fix the setpoint, for example, it could send this computed setpoint to the boiler continuously and adjust it all the time, regardless of whether the heating is on or off.
It really depends of its choice. This shouldn't be enforced.

When a thermostat is controlling the boiler, OTGW never resets the last setpoint sent by the thermostat right? It doesn't interfere.
In my case, the thermostat is me. But the gateway resets the setpoint I set.
There is no reason not to respect the last setting sent by a thermostat (no matter if the thermostat is OpenTherm or serial). Both are thermostats.

It would not be good if the engine would run at some high revs when you start it, just because that was used previously.

This is not always right. Freedom should be let to the thermostat to decide what to do, not enforced.
A boiler is not like a car engine: for example lets say you have HW=0 and your DHW is at 18°C. If you suddenly take a shower, you will see 100% modulation at first. This is not ideal (thermal shock), it would have been better to increase that slower, but the boiler will do this if needed on occasion.
A thermostat (including mine) knows exactly what is the last setting it sent. So it's not just "random" to start the boiler with that last setting, if it happens, it's because the thermostat algorithm wanted it. Otherwise the thermostat would change the setpoint to what it wants.
- Sometimes, we want a very slow and comfortable heating (using intertia, for example the CS adjusted with outside temperature, with a very low CS).
It's good for a lot of reasons (condensing range, less gas consumption, less thermal shock, less boiler cycling). But it has one drawback: the inertia.
- So, sometimes, we could want on occasion to do a boost for whatever reason, and why not right at startup?
Maybe if I left the house for 15 days to go on vacation, and if I come back earlier than planned and forgot to tell my system I will come back, I could need a boost: start the boiler with a 60°C setpoint so that we quickly reach the target and don't need to wait for 24h for the inertia to come up.

In my opinion, a device allowing control to something shouldn't decide/limit in the place of the implementer.

That's why I decided on my side to go around this by always sending CS= command, regardless of whether CH is 1 or 0.
My MQTT state topic always reflects what the boiler says its settings are. This is the most important to me. The state of the boiler.
If I stop sending CS, OTGW sends a CS of 0 to the boiler. I don't want that: the boiler corrects this to 20°C, because 0°C is not a legal value for this boiler, and my state topic changes to 20°C, although I never wanted the setpoint to be 20°C.
My main concern is that we really send this "0" setpoint to the boiler. Are you sure real thermostats do this?


It's okay to disagree on this. ^^ I already have a way to make things work for me, it's already great!
Anyway, it was interesting to make the feedbacks and share the different views. Thank you!

[hvxl]

My main concern is that we really send this "0" setpoint to the boiler. Are you sure real thermostats do this?

Yes. Real thermostats send some low value along with CHenable=0. Honeywell and Siemens tend to use 10, Remeha and Toon picked 6, and Tado and Anna do actually send 0. So 0 seems like a reasonable value for the OTGW to send in standalone mode.