A new event architecture

[hplus]

Adding the same formula editor as the one in the condition inside the device.value or the globalvar field in the action editor is already a great step forward and is almost as powerful as scripting and is easy to do. I can even do myself i think... even if i don't understand any scripting language. Just by copy pasting parts of your code....

[RDNZL]

You mean inside the <% %> tags? Hmmm

I think about adding a eval field for each action record/line.
Instead of true false actions.

So you can do:
If dev_43_value1 = "on" -> run action #1
If dev_43_value1 = "off" -> run action #2
Or just Run action #3 with eval code

Would that be a good alternative/idea?

[airox]

You can attach actions on the true side and on the false side. This is really what you want because often things need to be handled when the condition becomes false or true.

I thought about a few optimizations you can do when you need to fetch a lot of event data. This is something I never implemented because it's still fast enough for me (after adding indexes on all tables):

I'm really interested in the experiences you guys also have in other event architectures.

Having actions for true and false conditions is a very good idea!
I now have 2 events for a lot of things, "if device value = on then" and seperate "if device value is off then"...
I'm going to try to implement that.

A step further like hplus suggested earlier -having conditions for each action- is almost like having scripting capabilities, but I have not succeeded to implement that (i can only eval() basic gambas code, which returns true/false)

About your event architecture:
If I understand correctly you parse all your event conditions whenever there is something going on, either a device value changes, a lastchanged/seen time changes, time changes... you don't have trigger types on which you filter as first step, right?

I did that so triggertype 1 = cron/time, 2 = device value.. x = globalvar changes (for e-mail counters, callerid etc) and then I only checked if device and field matched, before i continued the parse...
Now (thanks to suggestions of hplus) I introduced multitrigger (as a test) so events can be triggered on various types.

We must gtalk some time , but we had that idea earlier if I remember correctly, a bit lack of time here..

One thing to remember when implementing actions on the true and false side is that you need to check if the know what the last outcome of the condition was and if the condition while evaluated is different then this one. Only then can you trigger the actions. So you need to log it somehow.

In my event architecture I'm also using types of triggers to filter for. But they depend on the way the events are coming in:
- Values (reported device values which change)
- Email (incoming email)
- Recurring (time based)
- Sunset, Sunrise, Date, Date time, Time and dayofweek (time based)
- Condition based

I'm querying the database by searching for the type of event and in case of values I search if the trigger of this event has the device mentioned in it. This results in a single query which filters out a lot. I only need to check if the conditions of the event will also evaluate to true. So far nothing really different I guess.

The real problem arises with condition based events in my architecture. I can handle things like this:

Code: Select all

<conditionBasedEvent id="LIGHTS_WHEN_NECESSARY" name="Zet lichten woonkamer aan wanneer nodig">
      <conditions>
         <equal>
            <flagValue id="SECURITY">uit</flagValue>
            <flagValue id="PRESENCE">ja</flagValue>            
            <lightDark id="LICHTSENSOR_WOONKAMER">0</lightDark>
         </equal>
         <conditions logical="OR">
            <equal>
               <motion id="BEWEGINGSENSOR_WOONKAMER">1</motion>
               <motion id="BEWEGINGSENSOR_KEUKEN">1</motion>
            </equal>
         </conditions>
      </conditions>
      <actionsWhenTrue>
         <switchOn device="TVLAMPEN" />
         <dim value="47" device="KEUKENLAMP" />
      </actionsWhenTrue>
      <actionsWhenFalse>
         <!-- switchOff device="TVLAMPEN" />
         <dim value="0" device="KEUKENLAMP" /  -->
      </actionsWhenFalse>
   </conditionBasedEvent>


In which you can see I am allowing conditions within conditions. So you can create complex statements, just like in programming languages. This is saved in an object model (and then saved to the database). Because you can have conditions inside conditions and so on it is very hard to query so I never investigated it further.

What I meant in my previous post is that I take care that I don't have to go to the database for fetching the trigger and conditions as defined by the user. I keep it all in memory. A roundtrip to the database is what hits you the hardest. And the need for it on every incoming device value is like zero

The things I found to take the most time when a (for example rfxcom) driver reports a value:
- Saving the value inside the large big table with all the values (history table) and updating the indexes of this table.
- Saving logs of action being taken place because of a triggered event (images, log lines, etc)

Like always, everything which is written to disk takes time. And with my little amount of events (around 30, of all the types above) and the new memory queue based implementation I'm using it's all fast enough. As in... I don't need the extra speed right now. I do have this listed on my backlog. It definitely would be nice to see how fast one can get for domotica

We indeed need to talk on gtalk or something, maybe we need to organize a domotica software thingie or something

[hplus]

I meant putting a formula editor inside the "To" field of the "set Device" action tab and the "set Globalvar" tab.
This solution can already do many things without changing too much things.
For other type of action, we should think to a similar solution

[Digit]

I also keep the complete events in memory. For me the biggest time savers are:
- No disk I/O needed for event processing/evaluation;
- The fact that a device value 'knows' from itself whether it's being used in an event or not. So in my case event processing only kicks in when evaluation is really necessary. With >750 device values at this moment and only 15 of them being used in events, this has a big impact.

[RDNZL]

I don't even have the device values in memory right now.

I guess I start doing that when Gambas3 becomes stable, it support structures, arrays of different types, so I can mimic the device sql record in memory easily.

I have tested with a few actions inside the eval field, and that works ok.
Will try if I can get all current actions working from inside the eval field.

I think I leave all like it is, (keep multitriggers) but just add an action type 'Eval Script' or something like that in which you can do your scripting if needed.

Screenshot-Edit Condition.png

Edit: Actually it's not needed to add an action type, or change anything big, you can put the above in a condition already and don't create an action for that event.
Only need to check the action subs for private/public and that they return booleans, and define them in the treeview for easy adding.

[hplus]

Ok why not...
It would just have been more coherent to have this in the action list editor.
I mean : keep the conditions the way they are.... the multitrigger (i've almost managed to have the timecron inside the multitrigger, promise i'll send you the code)
Add what you described just here in the action editor. Like this one can have several actions (ordered action list) and can have delay between the etc....
You'll get the best of the two worlds and a powerful event engine.

[hplus]

I've just sent you the code.

It's incredible, i used to have 13 events for one heating point.
Now i only have 2. One with the condition leading to On and one with the condition leading to Off.

With the coming possibility to put condition inside actions i will have 1 event!!!
It's a new beginning for me. I am now able to create complex scenario.

@Ron: This is from my point of view a major improvement in domotiga. Scripting capability is the heart of a Home AUTOMATION software.

THANKS

[RDNZL]

I have received it, will merge it.

You are right, scripting is important, but DomotiGa started as a tool to log data and master protocols (not that that's the best part, that has it's shortcoming too)
Gambas was perhaps not the best choice when you need scripting, perl is better (but we all know what happened to misterhouse code tree)
But the current eval() solution is not that bad.

[airox]

I don't think scripting should be the heart of a home automation system. It should be possible for my girlfriend to configure actions and events in my system. This shouldn't be done by someone who has scripting knowledge. Scripting can be used for those advanced users of the system.

But it all depends on who will be using the system of course.

[Andy_Burn]

@airox
I agree. Some form of simple user interface, perhaps what Ron started with the webclient, would be excellent. I have the dream (!) of running a small embedded board (efika anyone!) with a touchscreen running a browser only, set into the wall.
However, there should always be the control and scripts etc behind the scene, otherwise wheres the fun!!!!
@hplus/Ron
I have also noticed it is difficult to control my heating, needing timed scripts etc...but it is working!
This is an interesting development, please keep it up

Thanks, Andy

[hplus]

Don't misunderstand me, by script i meant being able to have complex event. No matter how you achieve this (by directly typing a script, by a script wizard, whatever....) this is the point of a home automation system. If you don't have this feature then you have a home monitoring software.

[Andy_Burn]

Yes, thats exactly what I want too Just maybe with a wife friendly interface when its setup! I will probably have a go at this myself.
Complex scripting - being able to build several conditions into events/triggers etc, will make everything more flexible and easier to program/debug...
I have been reading this topic with interest and cant wait to see what you guys come up with. Then I'll probably rewrite half of it to see if I can break it ha ha ha - I like to 'field test'

Andy

[airox]

This is what I do in my software (old pictures unfortunately and not english ):

Adding an event based on 1 trigger:
http://fotoboek.vandulmen.net/domotica/ ... tion_1.png

Assigning any other conditions for this same event where one can use AND, OR, XOR constructions and make it as complex as wanted:
http://fotoboek.vandulmen.net/domotica/ ... tion_2.png

And adding the actions for this event:
http://fotoboek.vandulmen.net/domotica/ ... tion_3.png

This can be done through the web interface. I also allow to export these events to XML:

http://fotoboek.vandulmen.net/domotica/ ... ribing.png

And am capable of importing this XML to the system again. I mostly use the XML definition for defining my events.

Greetings from the netherlands!

[hplus]

I've been working with the multi trigger thing for a while now everything seems ok (except the timecron trigger which is done multiple time)

I've kept on thinking about all this stuff and once again i have some ideas i would like to share before to start coding.

In the event menu i introduce the "timers" concept :
basically the user create using the cron syntax as many timers as he wants. Each timer has a name and a enabled/disabled status and is stored inside the database.
Timers are displayed like the triggers condition action in the event tab. You can add and delete timers.
Once a timer is enabled in goes through a compilation process which simply compute the next date at which this timer will trigger, this date is stored in the memory. Each minute the list of enabled timers date is parsed checking if they should trigger. If yes, they are first recompiled to have their next execution date and a list off all the timers that has fired for this minute is created.
Then a query is done on the event selecting all the even that has one fired timer in their multi trigger list.
Of course for this to work, the multi trigger tab should display a tree with the devices, the globalvar, and the timers.

This simple and elegant solution is a lot more efficient that todays timecron.... ( executing TimeCron each minute on all the redundant timers) By this way half of the work is done by the selective database query.

I think that using the same logic we could include the ir events in the multitrigger. And get rid of the other type of events.

I can code this, but i just want to get sure it will be merged because i don't want to have a personal fork of domotiga.