Software engine for processing RFXmeter values

Pop your questions regarding Home automation software here.....

Software engine for processing RFXmeter values

Postby ludom » Mon May 07, 2012 10:19 am

Dear,

Currently I am rewriting my domoticasystem written in PHP, for the past year I was monitoring my energy usage (water, electricity, gas) that worked tolerable but not perfect!.

I saw that in process of time the electricity countings from my system continuously deviated from the real countings on the meter of my electricity supplier. The countings of the water and gas are reasonably accurate. This make me thinking and searching to the problem in my system. First I checked if the pulse counter on my electricity meter was counting correct, I checked this for hours (with ambient light and without) but the conclusion was that pulse-counter counts accurate.

The problem came from my dom-system, when I programmed v1 of this class I calculated a formula that the system will use to calculate the kWh's, it looks like the following;

My meter has a C=187,5 that will result in 187,5 rotations for each 1kWh. Here for I devised the formula;
1kWh / 187,5 = 0,005333333333333333333333333333333333333 (and go on).

So every pulse from the RFXmeter will say that there is 0,005333333333333333333333333333333333333 kWh of power was consumed. But you can guess the problem, this can't be accurate at all! for the first weeks the system was accurate to the real meter but after 1 year it was way off because of this formula (333333333 1/3).

My current MySQL table `energy_usage` looks like this;
Code: Select all
CREATE TABLE IF NOT EXISTS `energy_usage` (
  `id` int(10) NOT NULL AUTO_INCREMENT,
  `address` varchar(4) NOT NULL DEFAULT '',
  `type` int(1) NOT NULL DEFAULT '0' COMMENT '1 = Elektriciteit, 2 = Gas, 3 = Water',
  `valueStart` decimal(36,30) NOT NULL DEFAULT '0.000000000000000000000000000000',
  `valueTotal` decimal(36,30) NOT NULL DEFAULT '0.000000000000000000000000000000',
  `pulseStart` int(8) NOT NULL DEFAULT '0',
  `pulseTotal` int(8) NOT NULL DEFAULT '0',
  `reads` int(4) NOT NULL DEFAULT '0',
  `dateAdd` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  `dateModified` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB  DEFAULT CHARSET=utf8 AUTO_INCREMENT=8756 ;


One record of data;
Code: Select all
(8754, '01F1', 1, 123816.257333333333333333333330997180, 123816.667999999999999999999997663821, 2245182, 2245259, 57, '2012-05-07 09:00:22', '2012-05-07 09:59:02')


For your information, every hour I create a new record in the Db, every minute (when the RFXmeter send it's readings) I will update the record for that hour, so at the end of the day I will have 24 records each day per energyType, so each day a total of 72 records (electricity, water, gas).

My question is, how can I make this accurate and how do you do it (Bwired, Digit, Airox etc?). Maybe I have only to save the pulses and not the calculated values before I insert it into the the database and calculate the kWh, m3 rate's afterwards (on the views)?

I hope you can help me out with this problem so i can improve this piece of software ;)

Thanks.
Home-brewed HA system based on Linux, PHP5, MySQL and Beanstalkd
User avatar
ludom
Member
Member
 
Posts: 90
Joined: June 2010
Location: Netherlands

Re: Software engine for processing RFXmeter values

Postby Digit » Mon May 07, 2012 12:08 pm

Use the pulses as basis for your calculations, can't get any better than that.
I only store the readings, no calculated values.
If you want to know the total usage for a month, just subtract the first countervalue of the month from the last countervalue of that month and divide the result by C. Or do a SUM of all the pulses per hour-record where the year & month are equal to the month you're interested in.
Whatever you like.
Average error will always be (and stay!) half a rotation = (1000/ 187.5) /2 = 2.6 Wh
User avatar
Digit
Global Moderator
Global Moderator
 
Posts: 3388
Joined: March 2006
Location: Netherlands

Re: Software engine for processing RFXmeter values

Postby ludom » Mon May 07, 2012 12:32 pm

Hi Digit,

Thanks for your reply! I think it's the right way to store only the pulse values, this saves a lot of table fields, zero's and other "meuk" :) I also want to merge all energy_records with the other data records of my system so I can work with one Data table for all kinds of data, certainly when I use only the start and stop pulse this could be a nice combination.

In addition to the data table I will create a new Energy_Usage class that can calculate all kind of usages on basis of the readed pulses.
Home-brewed HA system based on Linux, PHP5, MySQL and Beanstalkd
User avatar
ludom
Member
Member
 
Posts: 90
Joined: June 2010
Location: Netherlands

Re: Software engine for processing RFXmeter values

Postby ludom » Mon May 07, 2012 8:40 pm

Hi Digit,

I reprogrammed my RFXmeter driver now and it's working well! it only logs the startvalue and endvalue from each read!

I have one question left.

The RFXmeter has a count limit of 16.777.215. On this moment I have for example startvalue of 2246836 and endvalue of 2246927 for one hour of energyusage. When the RFXmeter resets to zero the startvalue of the first new record on that moment will be 0, the endvalue for example 60. How can I sum up the total counts on that moment? on this moment I select the startvalue of the first record and the endvalue of the last record and substract these values. But when the meter is resetted the endvalue of the latest record will be lower than the startvalue of the first record. Do i have to save the substraction value between start and end for each record? so I can sum up all substracted values at once? How do you do this?

Thanks!
Home-brewed HA system based on Linux, PHP5, MySQL and Beanstalkd
User avatar
ludom
Member
Member
 
Posts: 90
Joined: June 2010
Location: Netherlands

Re: Software engine for processing RFXmeter values

Postby Digit » Mon May 07, 2012 9:53 pm

I made a automatic rollover detection.
I know it's a 24 bit counter, so I (well, my system does) know when the rollover will take place. When it occurs, I programmaticaly add 0xFFFFFF (16777215) to the countervalue received and store that in the database. So I just keep on counting; that's the best/easiest way I guess.
And the field size in the database is large enough to count all the power I will consume during my whole life this way :wink:
User avatar
Digit
Global Moderator
Global Moderator
 
Posts: 3388
Joined: March 2006
Location: Netherlands

Re: Software engine for processing RFXmeter values

Postby ludom » Mon May 07, 2012 11:29 pm

That's smart thinking :D

Do you save the amount of occurred rollovers somewhere (database or configfile alike), so your system does know in how many multiplies it has to add to the counted value? Or can your system detect this out of the latest value (when it's higher than 16777215).
Home-brewed HA system based on Linux, PHP5, MySQL and Beanstalkd
User avatar
ludom
Member
Member
 
Posts: 90
Joined: June 2010
Location: Netherlands

Re: Software engine for processing RFXmeter values

Postby Digit » Tue May 08, 2012 12:07 am

I have a 'counterbits' field in the database. For RFXMeter the value is 24; for other counters it can be different (for example a 1-Wire counter has a value of 32)
Whenever a rollover occurs, I increment an 'offset' field in the database with a value based on the number of 'counterbits'.
User avatar
Digit
Global Moderator
Global Moderator
 
Posts: 3388
Joined: March 2006
Location: Netherlands

Re: Software engine for processing RFXmeter values

Postby ludom » Tue May 08, 2012 9:15 am

OK, that makes sense. I'll get to work with this!

Thanks for the information and ideas, this will work out for me! The first new 72 data records are added to the DB last night, it looks very nice with only the pulse values :D On the pages where I will have the usage I call my Energy_Usage class for the relevant deviceId and that will calculate the actual energy usage based on a setpoint (the first manual reading of the power meters) and the rate. Works really nice this way.

I wonder what will happens when the data table will have multiple (1.000.000 or more) records, I think this shouldn't be a problem because I created a few indexes and for the energy usage I currently select only the first and last record and some queries that group by month, year, etc which is very fast.
Home-brewed HA system based on Linux, PHP5, MySQL and Beanstalkd
User avatar
ludom
Member
Member
 
Posts: 90
Joined: June 2010
Location: Netherlands


Return to Questions & Discussions Forum

Who is online

Users browsing this forum: No registered users and 1 guest

cron