Creating my own OTGW firmware for the NodoShop version

This Forum is about the Opentherm gateway (OTGW) from Schelte

Moderator: hvxl

rvdbreemen
Starting Member
Starting Member
Posts: 45
Joined: Sun Jan 19, 2020 2:04 pm

Re: Creating my own OTGW firmware for the NodoShop version

Post by rvdbreemen »

franz wrote:Hej,

thanks for the reply. I have 0.7.8+702 (25-02-2021) now. I tried the api first, no success. The website shows:
OTGW firmware

[/api/v1/otgw/command/TT=10] is not a valid
Is that how it is supposed to work? Haven't tried MQTT yet again.
Hi Franz,
It's a POST command, if you type it in your browser then you will get this error. You need to use a HTTP POST command, try:
curl -X POST otgw.local/api/v1/otgw/command/TT=10

You can do this from any batch script or python script you write.
Greetings,
Robert
rvdbreemen
Starting Member
Starting Member
Posts: 45
Joined: Sun Jan 19, 2020 2:04 pm

Re: Creating my own OTGW firmware for the NodoShop version

Post by rvdbreemen »

franz wrote: /api/v1/otgw/command/"OT=20"
/api/v1/otgw/command/OT20
/api/v1/otgw/command/{OT=20}

Maybe I am just blind?
No you are not blind, just needs more documentation on the wiki I guess.
But like I explained, it needs to be an HTTP POST or HTTP PUT. Not a GET (aka using a browser).

Hope this explains,
Robert
rvdbreemen
Starting Member
Starting Member
Posts: 45
Joined: Sun Jan 19, 2020 2:04 pm

Re: Creating my own OTGW firmware for the NodoShop version

Post by rvdbreemen »

Announcing the 9th release of the Nodoshop firmware.

This is a major release to improve the integration with Homeassistant (breaking change).

Due to the nature of the change it's a breaking change, it is reimplementing the homeassistant integration and completly refactored the topic naming convention.

In this release:
  • change of naming convention of MQTT topics for otgw
  • add OTGW as a device and group all exposed entities as childs
  • add climate (thermostat) enity, uses temporary temperature
  • override (OTGW TT command) (Home Assistant Core v2021.2.0+)
  • new restapi - clean json array for easy integration with telegraf (/api/v1/otgw/telegraf)
  • fix to core ot message processing (asf flags en oem code related)
  • fix for issue #14 - watchdog reset on MQTT timeout (>5 sec)
  • fix for ASF flas in web UI (and restapi otmonitor)
  • added a LLMNR to improve integration with Windows 10 & Linux (just goto: http://otgw/)
Shout out to:
  • @sjorsjuhmaniac (for creating the improved homeassistant integration).
  • @Martinvdm & @tjfsteele (for testing and remote debugging support).
If you have ideas, feedback, issues, or pull request, you should visit the github repo. Do you want to chat in realtime, just go over to Discord.

Download the binary's to upgrade right here: 9th release v0.8.0
franz
Starting Member
Starting Member
Posts: 6
Joined: Sun Feb 14, 2021 12:44 pm

Re: Creating my own OTGW firmware for the NodoShop version

Post by franz »

Hej Robert,

thanks, that helped.

And thanks again for your work.

Cheers,
Franz
rvdbreemen
Starting Member
Starting Member
Posts: 45
Joined: Sun Jan 19, 2020 2:04 pm

Re: Creating my own OTGW firmware for the NodoShop version

Post by rvdbreemen »

The OTGW Nodoshop ESP 8266 firmware has just been released. This is version 0.9.0. It's been a while since I last updated this topic on with the release information. Since the v0.8.0 release we have seen 6 minor releases of the firmware adding small features, fixing bugs and improving stability.

So this release contains several small improvements, enhancements and some bug fixes. Lets sum it up:
  • New: NTP hostname is now a setting in the webui
  • New: Time/Date is now set using serial commands (SC, and SR=21 and SR=22)
  • Fixed: OT status (incl. VH and Solar) message decoding
  • Fixed: Status bit decoding in webUI
  • Improved: Auto-wifi connection recovery (ESP based)
  • Improved: Reboot 15 min after Wifi loss
  • Changed: Removed ezTime NTP library, moved over to ESP native ConfigTime NTP and AceTime
  • Improved: Ton of small stability improvements.
To find out what changed since version 0.8.0, you can best go to the github release page:
https://github.com/rvdbreemen/OTGW-firmware/releases

For more information on this release:
https://github.com/rvdbre...mware/releases/tag/v0.9.0

As always I am open to any questions, complaints, issues you gind, pull-request you have for me or compliments. The easiest way to give feedback is through Github or on the Discord community.

Kinds regards,
Robert
rvdbreemen
Starting Member
Starting Member
Posts: 45
Joined: Sun Jan 19, 2020 2:04 pm

Re: Creating my own OTGW firmware for the NodoShop version

Post by rvdbreemen »

The 17th release of the firmware for the ESP8266 based Nodoshop is here. Just before Christmas, something that started a couple of weeks ago with the idea to introduce new set commands feature, read more here. This feature is going to simplify the coding of automation using the home automation solutions out there.

The biggest change to the code is the improvements I made to the OpenTherm message parsing. With the endless help of some of the community testers, and a patient Schelte Bron helped me understand the inner working of OpenTherm decoding this release improves the OT decoding logic by a lot.

Also a ton of small fixes, improvements, cleanup of code, and the introduction of a "bootlog" function to simplify debugging over time.

Here is the overview for this release:
  • new feature: set commands for MQTT to simplify control of the OTGW
  • improved: the OT decoding algorithm, so values on MQTT, REST, and WebUI now should be more reliable
  • new feature: reset bootlog to the filesystem, for debugging purposes
  • improved: stability, due to removal of ESP based auto-wifi-reconnect
  • Improved: no messages on versions when not connected to the internet
  • added: proper msg id 100: remote override room setpoint flags decoding
  • added: missing some msg ids to OT mapping in the firmware
  • improved: cleaned up debug logging information
Just go here to download the next release: https://github.com/rvdbreemen/OTGW-firm ... tag/v0.9.1

So yet another release to enjoy for all of you, hope you like this Christmas release 2021. As always issues should be logged using github issue tracker, so I won't forget to fix them, also feature suggestions, questions or remarks are always welcome. As are pull-request to the code.

Last but not least, the best place to find me and many other, is the discord community server, join the discussion right here: https://discord.gg/zjW3ju7vGQ
rvdbreemen
Starting Member
Starting Member
Posts: 45
Joined: Sun Jan 19, 2020 2:04 pm

Re: Creating my own OTGW firmware for the NodoShop version

Post by rvdbreemen »

Releasing version 19... Just finished release the ESP firmware for Nodoshop OTGW, some minor bugs had to fixed to make sure it works for most classic integrations from home automation products like, Domoticz.

Go here to find the binary release: https://github.com/rvdbreemen/OTGW-firm ... tag/v0.9.3

This release included the following fixed:
  • Bugfix: Reverting change to the size of the input buffer, as the 128 bytes buffer broke PS=1 responses.
  • New setting added: A new setting for reboot detections cycle of HA. It turns out that in rare cases we do not see HA go offline, then turning off the feature should fix that. Default is detecting the HA reboot cycle.
  • Bugfix: Now parsing of top topic when changed to something that contained a "/", like "Myhome/OTGW"
  • Bugfix: Fixed version detecting in PIC hex files
  • Added: version detection of pic banner in OT processing loop
  • Removed: interface.hex and diagnostic.hex from the filesystem. For now included in the filesystem as hidden files, downloading, renaming, and uploading would make them visible again.
If you missed the previous release, just know, it turns out that it's the most stable release of the ESP firmware so far. Improving the OT message decoding and handling of the gateway overriding. So lots of improvements on that end.

Hope you like the ESP firmware, if you have any issues with it, you can report them on GitHub thru the issues or join the discord server. As always, fixes, issues, pull request, features, just ask, and it will be talked about.

Kind regards,
Robert
rvdbreemen
Starting Member
Starting Member
Posts: 45
Joined: Sun Jan 19, 2020 2:04 pm

Re: Creating my own OTGW firmware for the NodoShop version

Post by rvdbreemen »

Releasing v0.9.5 the 21th release. Just released the latest ESP firmware for the Nodoshop OTGW. This time around I put together a minor release based on the contribution of the community, thru some pull request, feedback and discussion on discord.

What's Changed
  • Improved: web UI was improved by @DutchessNicole. Added dropdown, small CSS improvements were added.
  • Improved: added User Agent to HTTP request when querying Schelte Bron's website for firmware updates (pic16f88 only for now)
  • Improved: now reporting pic-type in device information, in preparation of 6.x flashing / upgrades
  • Fixed: prevent bootloop https spamming of Bron Schelte's website
  • Fixed: home assistant integration of device was improved, so it now returns to "online" instead of "true" after a reboot (found by sergantd)
  • Fixed: Update mqttha.cfg by @STemplar in #112
  • Fixed: Change MQTT Control_setpoint_for_2e circuit by @Martinvdm in #113
Go here to find the binary release: https://github.com/rvdbreemen/OTGW-firm ... tag/v0.9.5

This release was made possible by users from the OTGW community once more. Hope you like the ESP firmware, if you have any issues with it, you can report them on GitHub thru the issues or join the discord server. As always, fixes, issues, pull request, features, just ask, and it will be talked about.

Want to give me feedback on the ESP firmware, just join the discord community right here: https://discord.gg/zjW3ju7vGQ

Kind regards,
Robert
brommetje
Starting Member
Starting Member
Posts: 11
Joined: Sun Oct 28, 2018 3:31 pm

Re: Creating my own OTGW firmware for the NodoShop version

Post by brommetje »

Hi All,
Just started with HASS and flashed nodeshop otgw with firmware created by @rdvbremen but now I have some starter questions.
Configured in the gui mqtt also running under HASS as a add-on and see a lot of entities (around 175)

Question do I also have to install the integration OpenTherm Gateway?

Want to start step by step :
How can I change P4 ECO-stand, there are 3 options 0, 1 or 3 from HASS, because of the gas prices?
Change the outside temperature from a outside weather station?
Change the room temperature?
Do I have to use mqtt commands or the integration OTGW, not clear for me at all?
Maybe someone has examples how to do this?

Regards,
Martin
dr.knor
Starting Member
Starting Member
Posts: 9
Joined: Tue Nov 21, 2017 11:34 am

Re: Creating my own OTGW firmware for the NodoShop version

Post by dr.knor »

Templar wrote: Sat Jan 16, 2021 4:27 pm Hi rvdbreemen,

I've got a slightly differend hardware setup. The classic OTGW hardware with the Recom DC/DC convertor mod. And instead of the MAX232 IC, I connected a ESP8266-12E VCC=5V. On it I run the ESP-LINK: Wifi-Serial Bridge w/REST&MQTT V2.2.3. To use your Nodemcu firmware the only thing I need to do is altering these PIN definitions in the OTGW-Core.ino file? It has been 4 years ago I set this up..

Code: Select all

#define PIN_I2C_SDA 4
#define PIN_I2C_SCL 5
I see no reply on this, but I actually got this to work fairly easily on my classic OTGW. I use an ESP-01 on an ESP-01 UART 3,3/5V adapter module. Powered from the OTGW (using a Recom DC/DC indeed). The code is based on rvdbreemen's FW version 0.9.5, with the following modifications.

1. Arduino IDE board selection "Generic ESP 8266 Module". The flash size is just 1MB for this unit (Nodo shop version has 4MB). I selected FS:256K, which leaves ~374K for OTA (not using OTA, and probably would not work in this configuration, i guess)
2. Removed all references and connections to the I2C external watchdog (including the SDA and SCL pin references, and Wire commands), and use the ESP.h SW watchdog in its place (modified the feedwatchdog command) with a 3000 ms timeout.
3. Changed the pin definitions for the LEDs and the PICreset pin. Actually no LEDs are connected now, and still need/want to implement a reset line (with voltage level shift) from the ESP to the OTGW, but without it the FW also works fine (for my purposes at least). I currently only have a reset button connected to the OTGW.

I followed the 'How to compile' instructions on rvdbreemen's Github, with all the proper library selections. The FW runs perfectly and stable. I use it to make an MQTT binding to Domoticz (used the OTGW-Domoticz-MQTT-Client for that). I am still using 4.2.5 FW for the OTGW and the standard Domoticz binding (via PS=1) only works with version 5 and up. I made one more hack in the ESP FW to only send opentherm messages via MQTT that start with R (so messages to boiler that are modified by the OTGW). The PIC FW that I use was modified to override the Tset signal for zone control using the SB feature and in this way I can also monitor that function via MQTT, without having to send a serial command to the OTGW first (to check the status of the GPIO pins in this case). I parse the MQTT opentherm messages with NodeRed (actually copied the parsing code from the ESP FW to do that).

Previously, I flashed the PIC via OTmonitor over wifi (via the jeelabs ESP-link FW that you mention). I hope that flashing still works with the ESP FW in between (via port 25238), or maybe via the PIC firmware upload facility in the ESP FW, but I have not tried it and no need to modify the PIC FW at this moment. I am happy with the setup. Nice stuff rvdbreemen and hvxl!
rvdbreemen
Starting Member
Starting Member
Posts: 45
Joined: Sun Jan 19, 2020 2:04 pm

Re: Creating my own OTGW firmware for the NodoShop version

Post by rvdbreemen »

Nice one. Or of course possible to hack your own based on mine. Thanks for sharing.
hvxl
Senior Member
Senior Member
Posts: 1965
Joined: Sat Jun 05, 2010 11:59 am
Contact:

Re: Creating my own OTGW firmware for the NodoShop version

Post by hvxl »

dr.knor wrote: Fri Feb 17, 2023 4:08 pm Previously, I flashed the PIC via OTmonitor over wifi (via the jeelabs ESP-link FW that you mention). I hope that flashing still works with the ESP FW in between (via port 25238)
That doesn't work on rvdbreemen's FW version 0.9.5. It has been fixed in FW version 0.10.X. So if that is something you need, you'll have to use 0.10.X, or at least backport the relevant changes to OTGW-Core.ino.
Schelte
dr.knor
Starting Member
Starting Member
Posts: 9
Joined: Tue Nov 21, 2017 11:34 am

Re: Creating my own OTGW firmware for the NodoShop version

Post by dr.knor »

Thank you! That is good to know. I will make the change. I plan to move to 0.10.X later, but seeing the ambitions of the project and the growing program size, I will have to stop updating at some point with the 1MB ESP-01.

By the way, I plan to make a reset line from the ESP to the PIC reset input, to replace the current button switch. But I was wondering what the exact purpose is of that reset line. I see it is used in the ESP FW code (PIC reset on ESP boot for example), but the setup also works without it. I did notice in my old setup that, after power up, I always had to do a manual PIC reset to open up communication towards the PIC (so the PS=1 command would not work before I did the reset).

I suppose this 'problem' would be solved by having a reset line? And also, would there then still be a need for a manual PIC reset switch? More specifcally, during PIC FW upload, is a manual reset action still requested to go into programming mode? Or could you do that in another way (e.g. by executing a reset command over the serial connection, or via the ESP FW)?
hvxl
Senior Member
Senior Member
Posts: 1965
Joined: Sat Jun 05, 2010 11:59 am
Contact:

Re: Creating my own OTGW firmware for the NodoShop version

Post by hvxl »

Here's the situation with the firmware versions you are using:

When the PIC boots, it first enters the boot loader. The boot loader listens for one second if new firmware is being uploaded. When the ESP boots, it sends some output on the serial interface at 74880 baud. This causes a framing error on the AUSART of the PIC, which expects 9600 baud. The normal firmware is able to handle a framing error correctly, but the boot loader doesn't. But because the boot loader reads the AUSART receive register, the framing error indication is lost. So when the regular firmware starts, it is unaware that there has been a framing error and it doesn't run the code to recover from such a situation.

Resetting the PIC after the ESP has booted clears the AUSART and no new framing error occurs in the boot loader, so then communication works. Resetting the ESP when the PIC is running is no problem. The normal PIC firmware recovers correctly from any 74880 baud data. The problem really only happens when the ESP and PIC boot at the same time, i.e. on power up.

The problem described above has been fixed in firmware 5.5 (and 6.2). The regular firmware now always completely resets the AUSART after the boot loader finishes.

To perform a PIC FW upload, OTmonitor sends a reset command to the PIC. As long as the PIC is running properly working firmware, a hardware reset is not necessary. You only need a hardware reset in case a firmware upgrade fails and you are left with a PIC that contains broken firmware, that doesn't react to a reset command anymore.

Conclusion: If you upgrade your PIC firmware to 5.5 or later, you shouldn't need to hook up the PIC reset pin to an ESP pin. And you may only need the manual reset in the rare situation that a FW upgrade fails.
Schelte
dr.knor
Starting Member
Starting Member
Posts: 9
Joined: Tue Nov 21, 2017 11:34 am

Re: Creating my own OTGW firmware for the NodoShop version

Post by dr.knor »

Your explanation could not be clearer, thanks again! I know what to do in the summer season (when I can disconnect the OTGW without risk of disrupting current operation): upgrade the PIC FW. Then I can leave out the reset line, although I already soldered an extra level-switch circuit to GPIO0 on the adapter board in the weekend. I will just have to find another purpose for that! Cheers.
Post Reply

Return to “Opentherm Gateway Forum”