Events, Triggers and Actions

Post by **RDNZL** » Wed Jan 21, 2009 7:56 pm

Hi,

I have implemented Events in DomotiGa with some nice GUI to manage them, but have been re-thinking it's design.

I now have events, which can have a trigger like DeviceChange -> Value x = or <> "text", and I have actions like SetDevice (X10 lamp on for example)

I'm thinking about adding 2 conditions to the event so.

If Trigger AND/OR condition1 AND/OR condition2 (optional) then run action(s).

triggers can be for example:
devicechange 'Kitchen Motion' value = "Motion"
devicechange 'Bathroom sensor' value3 = "wet"
timenow = 15:00
globalvar change House_mode = "Home"

todo:
manual (click button)
got e-mail

condition can be:
globalvar Dark = True
globalvar House_mode = "away"

action(s) can be:
set device "Kitchen lamp" ON
set device "house ventilation" speed 3
todo:
run other trigger ()
send e-mail
take camera image
play sound
say 'some text'
set globalvar

up to three can be run after each other.

anyone has some thoughts about this?

regards,
Ron.

Post by **Digit** » Mon Feb 02, 2009 1:23 am

I would try to create an event handler that has no limits. You never know when you need that extra trigger or action.
For instance i have an event that has 10 actions: turning off all lights and other things i don't need while i'm asleep. And it's still growing.
Also it is good to implement AND's and OR's on the trigger conditions, so you can create conditions like: (X or Y) and not Z.
And make sure you create your event handler in a way so that it only checks those events that are actually triggered by a change of something.
And to prevent introducing delay, make sure the event handler gets notified about changes ASAP.

Post by **RDNZL** » Mon Feb 02, 2009 2:00 am

Hi Robert,
your info is appreciated!

I agree all should be unlimited, but.. it can slow things down I have noticed, and have started witth a limited set, this event stuff is by far the most complicated part of DomotiGa. Well ok, I have to admit that designing something on paper beforehand will make thing easier, but most of the time I just start typing away...

I have decided that -for the time being- I have Events which can have one Trigger (that determines what kind of Event it is), for example an 'Device Status Change event', or a 'Time Now event', so when a device value changes I fetch all Events with this Trigger type and Device id and process them. It should be better to load these events at program start and event changes only, but now I fetch them from the mysql db everytime, gives a bit of load I found out. What do you do in your program?

Then beside this Trigger an Event can have 2 (for now) extra conditions (and/or) which are basically triggers but these are only processed if a trigger is true.

If these extra conditions (if defined) are true then 1 to 3 actions are executed. I guess you are right about the need for a higher amount of actions, at first I thought about defining an Event of type 'Manual' which can also be called by other Events. But i'm not sure, I guess it's better to have the actions unlimited.

Can you enlighten me about what kind if actions you have defined? (if they are not too personal

Here some windowshots of what is working now. The type/tabs of Triggers/Conditions will be expanded later.

Regards,
Ron.

Post by **Mdamen** » Mon Feb 02, 2009 8:51 am

I'm in the middle of the same proccess for houseagent. Really having a hard time getting my head around this [B)] Any input from you Robert would be appreciated.

--
Maarten Damen

www.maartendamen.com

Post by **Digit** » Mon Feb 02, 2009 11:49 am

Well i'm not the kind that produces lengthy documents before coding either, most of the time i have a general concept in my head and just start...
Let me first tell something about how my HA app works, otherwise you'll probably not understand all the things i write down later on.
I have constructed a way where 95% of all device-common code is contained in a single class and i derive new classes of that on demand.

Code: Select all

clDevice
  |
  |--- X10Module
  |         |
  |         |--- UM7206
  |         |--- SAIX
  |
  |--- PLCBUSModule
  |         |
  |         |--- PLC2027
  |
  |--- VisonicMotionSensor
            |
            |--- NEXTMCW
            |--- K980
            |--- etc..

The mother of all classes, clDevice, takes care of everything common to all devices: type conversions, data collection to SQL, status updates to SQL, error conditions etc. Everything that's not common is handled in derived classes. So for example only the PLCBUSModule class knows how to handle PLCBUS data, how to convert it to normalized data types as numeric, text etc.

The memory of my HA app is in fact a virtual representation of all the devices in my house. For every device (temp, door, motion, actor, counter) i have an object in memory. That object knows what kind of device it represents, where it is located, etc.
And then there are some helper classes, like a virtual device for time, a few data handlers, queues etc.
For every physical interface (TI213, RFXCOM receiver, transmitter, Plugwise) i have an object also.
These objects handle configuration of/communication with the physical interfaces and are responsible for all incoming and outgoing data, validation and 'data routing'.
For incoming data these interface objects determine the address contained in the data and then pass the data packet on to the device-object that is known to represent the device with that address.

Then the device-object does some more validation and calculates the values as defined by the object class (temp sensor object calculates the temp, door sensor object determines open/closed status etc).
These device objects have their current and previous value in memory, as a class-property. So when new data arrives, i have to do very little to determine device value change.
The device object also 'knows' whether it is a event trigger or not. If it is, and a device value change occurs, the event handler will be signaled to check all events that have that specific device with that unique address as trigger.

This way i can keep event processing overhead to a minimum, doing only that what needs to be done.

<blockquote id="quote">quote:<hr height="1" noshade id="quote">Originally posted by RDNZL
 It should be better to load these events at program start and event changes only, but now I fetch them from the mysql db everytime, gives a bit of load I found out. What do you do in your program?
<hr height="1" noshade id="quote"></blockquote id="quote">
You're right, that is what i do, load only once. At startup an object is created for every event listed in the database. And every event-object has a list of trigger-objects and a list of action-objects.
Reducing disk I/O will definetely improve speed!

<blockquote id="quote">quote:<hr height="1" noshade id="quote">Originally posted by RDNZL
 Can you enlighten me about what kind if actions you have defined?<hr height="1" noshade id="quote"></blockquote id="quote">
Currently i have 2 ActionTypes: Actor and SQLStatement.
With the Actor ActionType i can define any type of Actor i have (X10, PLCBUS, Circle etc.)
(and yes, i use event system for periodic database maintenance by triggering SQL statements by the virtual clock-device

)
No need for more ActionTypes yet. Currently i have around 35 EventActions defined.

Hope this is useful.

Post by **Mdamen** » Mon Feb 02, 2009 6:34 pm

Thnx for the info.. although your event structure is not clear to me yet.

Funny to see we use the same idea for devices, I have something like this:

Code: Select all

class Device(SQLObject):
    name = StringCol(length = 32, notNone = True, default = 'Name not set')
    address = StringCol(length = 32, notNone = True, default = '')
    type = ForeignKey("DeviceType")
    interface = ForeignKey("Interface")
    location = ForeignKey("Location")
    value = StringCol(length = 32, notNone = True, default = '')
    lastchange = DateTimeCol(notNone = True, default = datetime.datetime.now())
    extension = ForeignKey("DeviceExtension", default=None)

And then for each device an inhereted class:

Code: Select all

class PlugwiseDevice(DeviceExtension):
    offruis = FloatCol(notNone = False, default = 0)
    offtot = FloatCol(notNone = False, default = 0)
    gaina = FloatCol(notNone = False, default = 0)
    gainb = FloatCol(notNone = False, default = 0)
    watt = FloatCol(notNone = False, default = 0)
    currentlogaddress = IntCol(notNone = False, default = 0)
    lastlogaddress = IntCol(notNone = False, default = 0)

--
Maarten Damen

www.maartendamen.com

Post by **RDNZL** » Mon Feb 02, 2009 9:55 pm

Interesting thread.

Having a default device class with different devicetype classes underneath it is easy to do in Gambas I guess.

But..

How do you guys manage that on the db schema side? You can't have classes there, so you have to discard info, or have all fields possible in the devices table, now I have all devices with the same table layout, with a max of 4 values per device. I now use this:

tableDevices = Main.hDB.Tables.Add("devices")
tableDevices.Fields.Add("id", db.Serial)
tableDevices.Fields.Add("name", db.String, 32)
tableDevices.Fields.Add("address", db.String, 64)
tableDevices.Fields.Add("user", db.Integer)
tableDevices.Fields.Add("module", db.Integer)
tableDevices.Fields.Add("location", db.Integer)
tableDevices.Fields.Add("value", db.String, 32)
tableDevices.Fields.Add("value2", db.String, 32)
tableDevices.Fields.Add("value3", db.String, 32)
tableDevices.Fields.Add("value4", db.String, 32)
tableDevices.Fields.Add("label", db.String, 32)
tableDevices.Fields.Add("label2", db.String, 32)
tableDevices.Fields.Add("label3", db.String, 32)
tableDevices.Fields.Add("label4", db.String, 32)
tableDevices.Fields.Add("onicon", db.String, 32)
tableDevices.Fields.Add("officon", db.String, 32)
tableDevices.Fields.Add("interface", db.Integer)
tableDevices.Fields.Add("firstseen", db.Date)
tableDevices.Fields.Add("lastseen", db.Date)
tableDevices.Fields.Add("enabled", db.Boolean)
tableDevices.Fields.Add("hide", db.Boolean)
tableDevices.Fields.Add("log", db.Boolean)
tableDevices.Fields.Add("groups", db.String, 128)
tableDevices.Fields.Add("graph", db.Boolean)
tableDevices.Fields.Add("batterystatus", db.String, 32)
tableDevices.Fields.Add("tampered", db.Boolean)
tableDevices.Fields.Add("comments", db.String, 0)
tableDevices.Fields.Add("valuerrddsname", db.String, 32)
tableDevices.Fields.Add("value2rrddsname", db.String, 32)
tableDevices.Fields.Add("value3rrddsname", db.String, 32)
tableDevices.Fields.Add("value4rrddsname", db.String, 32)
tableDevices.Fields.Add("valuerrdtype", db.String, 32)
tableDevices.Fields.Add("value2rrdtype", db.String, 32)
tableDevices.Fields.Add("value3rrdtype", db.String, 32)
tableDevices.Fields.Add("value4rrdtype", db.String, 32)
tableDevices.Fields.Add("switchable", db.Boolean)
tableDevices.Fields.Add("dimable", db.Boolean)
tableDevices.Fields.Add("x", db.Integer)
tableDevices.Fields.Add("y", db.Integer)
tableDevices.Fields.Add("floorplan", db.Integer)
tableDevices.Fields.Add("lastchanged", db.Date)
tableDevices.PrimaryKey = ["id"]

Those device classes are very nice, but if device records needs updating (and there are alot updates like lastseen , lastchanged field etc), you write them to the class in memory and the db at the same time? Or you only update the db if you close the program or on regular intervals? (a bit tricky if programs fails)

I even looked at using MySQL's MEMORY(HEAP) db engine instead of the classes in memory. So you can have exact the same db layout in memory, and you only have to talk to another db handle to use it, anyone have ideas about that approach?
I remember that there were some caveats at the time I looked at it, but can remember them.

Regards,
Ron.

Post by **Digit** » Mon Feb 02, 2009 10:35 pm

@Maarten: here some code, maybe this will make it more clear.

It all starts in the device class code:

Code: Select all

  // if known to be a Trigger for an Event, then queue Eventdata now.
  if IsTrigger
  then begin

    if assigned(EventDataHandler)
    then begin

      D:=TDataQueueItem.Create;
      D.Source  := Self.ClassType;
      D.Address := DeviceID;
      D.Data    := ValueDesc;

      EventDataHandler.QueueToThread(D);

    end;
  end;

The EventDataHandler is basically an object that involves a Queue and a Worker Thread.
When the queued item is being processed, the following code is executed:

Code: Select all

  for i:=0 to EventList.Count-1 do
  begin
    Event:=TEvent(EventList.Objects[i]);

    if Event.EventTriggerList.IndexOf(Address) >= 0 then Event.CheckEventTriggers;
  end;

This is what the Event.CheckEventTriggers looks like:

Code: Select all

  AllConditionsAreValid:=true;

  for i:=0 to EventTriggerlist.Count-1 do
  begin
    Trigger:=THCSEventTrigger(EventTriggerList.Objects[i]);

    { TODO : Change the fixed boolean to a variable operator and use of parentheses }
    AllConditionsAreValid:=AllConditionsAreValid and Trigger.ConditionState;

  end;

  // now if all conditions were met, perfrom the actions.

  if AllConditionsAreValid
  then begin
    for i:= 0 to EventActionList.Count-1 do
    begin
      Action:=TEventAction(EventActionList.Objects[i]);

      case Action.ActionType of
      
        ATActor:begin
          D:=TDataQueueItem.Create;
          D.Source:=ClassType;
          D.Address:=Action.Address;
          D.Priority:=75;
          D.Data:=Action.ActionValue;
          ActorDataHandler.QueueToThread(D);
        end;

        ATSQL:begin
          SQLExecutor.Execute(Action.ActionValue);
        end;

      end; // case

    end;

  end;

That's about it

@Ron:
In my case a Device also has 'pins' as i call them: for example a Circle has 3 pins: PowerUsage, PowerTotal, PowerState.

A Device is stored in this table:

Code: Select all

CREATE TABLE [PhysicalDevices](
	[DeviceID] [varchar](10) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
	[TypeID] [varchar](50) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
	[PhysicalAddress] [varchar](20) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
	[Location] [varchar](60) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
	[Available] [bit] NOT NULL,
	[SaveLastReceived] [bit] NULL,
	[LatchTimeout] [int] NULL,
	[LastReceivedInterval] [int] NULL,
	[LastReceived] [datetime] NULL
)

The pins are stored in a separate table:

Code: Select all

CREATE TABLE [LogicalDevices](
	[LogicalDeviceID] [varchar](20) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
	[PhysicalDeviceID] [varchar](10) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
	[PinID] [int] NOT NULL,
	[Description] [varchar](60) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
	[Unit] [varchar](10) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
	[WebDisplay] [bit] NULL CONSTRAINT [DF_lgDevices_WebDisplay]  DEFAULT ((1)),
	[DevicePinInformationTypeID] [varchar](50) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
	[SaveData] [bit] NULL CONSTRAINT [DF_lgDevices_SaveData]  DEFAULT ((0)),
	[SaveEvent] [bit] NULL CONSTRAINT [DF_lgDevices_SaveEvent]  DEFAULT ((0)),
	[SaveStatus] [bit] NULL CONSTRAINT [DF_lgDevices_SaveStatus]  DEFAULT ((1)),
	[DataTimeResolution] [int] NULL,
	[DataHistoryDays] [int] NULL CONSTRAINT [DF_lgDevices_DataHistoryDays]  DEFAULT ((0))
)

So for a Circle i have 4 records: 1 in the first table, 3 in the second.
That will make it much easier to store your classes.
Another way could be adding a blob or memo field and call it 'properties', and do something like this:

During startup a regular expression parser will evaluate the 'properties' and assign them to the right object properties.

And yes, the SQL DB is updated as soon as values change.
This is done even before the Events handling is executed.

I've also worked with in-memory tables for some time, didn't like it.
Now i load the information stored in the DB into the objects at startup and only refresh when for example a new Device has been added.

Post by **Mdamen** » Mon Feb 02, 2009 11:02 pm

@Robert: nice, that's more clear. Could you post your table definitions for events and triggers aswell? Thanks!

@Ron: I use the SQLObject class, which basically is a object relational manager for databases. Good thing is that it is multi database platform ranging from mysql to oracle to sqlite etc. (http://www.sqlobject.org/)

SQLObject has a caching module which I use, from the documentation:

This implements the instance caching in SQLObject. Caching is relatively aggressive. All objects are retained so long as they are in memory, by keeping weak references to objects. We also keep other objects in a cache that doesn't allow them to be garbage collected (unless caching is turned off).

Changes however are directly written to the database.

--
Maarten Damen

www.maartendamen.com

Post by **Digit** » Mon Feb 02, 2009 11:35 pm

Code: Select all

CREATE TABLE [dbo].[EventDefinitions](
	[EventDefinitionID] [varchar](50) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
	[Description] [varchar](50) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
	[Enabled] [bit] NOT NULL CONSTRAINT [DF_EventDefinitions_Enabled]  DEFAULT ((1)),
	[EventType] [int] NULL,
	[Offset] [int] NULL,
	[Date] [datetime] NULL,
	[RetriggerDelay] [int] NULL,
	[Randomize] [bit] NULL,
	[NoLog] [bit] NULL
)

CREATE TABLE [dbo].[EventTriggers](
	[EventDefinitionID] [varchar](50) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
	[Connector] [varchar](50) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
	[TriggerType] [varchar](50) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
	[lgDeviceID] [varchar](50) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
	[Operator] [varchar](50) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
	[TriggerValue] [varchar](50) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
	[Enabled] [bit] NULL
)

CREATE TABLE [dbo].[EventActions](
	[EventDefinitionID] [varchar](50) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
	[EventActionID] [int] NOT NULL,
	[ActionTypeID] [int] NULL,
	[LgDeviceID] [varchar](50) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
	[CommandTypeID] [int] NULL,
	[Comment] [varchar](50) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
	[ActionValue] [varchar](500) COLLATE SQL_Latin1_General_CP1_CI_AS NULL
)

Not all fields are in use, i still have to work on the events, but that will happen when i need it