Thursday 3 April 2014

Controlling the boiler from the internet

With the Pi and Arduino all connected up and with the Arduino already knowing how to turn the boiler on and off, it didn't take long to get the Node server able to listen for boiler updates and signal changes of its own.



As the scheduler isn't (anywhere near) in yet, all we have for now is a greyed out flame icon next to the temperature read out. Clicking that flame icon will toggle the boiler to the alternate state:


All other connected clients will see the flame icon light up on their dashboards at the same time and the boiler fires up.

Just the scheduler to go and we've got the basics of a thermostat!

Wednesday 2 April 2014

Tidying and Linking

A lot of housekeeping today.


  • A separate test bed Arduino for the temperature sensor has been integrated with the boiler controller sketch. It polls the sensor every 10 seconds and keeps the average of the last 10 as the 'current' temp.
  • Calibration of the temperature sensor has been added. Simple as adjusting the reported value by a set amount which is stored by the Pi and (eventually) the user can calibrate the sensor from the web interface.
  • If the boiler is turned off by the system, there's a 5 minute 'cooldown' before a new 'on' signal can be triggered to prevent excessive cycling of the boiler. The boiler has this protection built in anyway but if controlled by the Arduino it's something that can be tracked and fed back to the user in case they're wondering why the boiler hasn't actually fired up.
  • Lots of code tidying and organisation.
  • NodeJS backend now handles any data sent from the Arduino and handles the various 'packets' accordingly. Temperature readings and boiler state transmissions so far (along with any error codes).
  • Same backend has rudimentary authentication for visiting users. Once authenticated, connected users receive updates from the Arduino as the Pi gets them allowing for the current temperature to be updated on the interface every 10 seconds in real time.
Well... its a start...

Tuesday 1 April 2014

Controlling the Arduino from a Pi

The Raspberry Pi is a great little device, just like the Arduino. Whilst the Arduino is a little input/output star the Pi is a little Linux box capable of doing anything (albeit a little slower ;)) than its bigger cousins.

The Pi is therefore a little more suited to hosting a standard web stack, you can run Apache/PHP/MySQL with no trouble at all. I'm taking the NodeJS route however as its nicely asynchronous so we can easily read messages from the Arduino whilst thinking about other things.

With NodeJS/NPM already installed, getting what we need on the Pi to communicate with the Arduino is as simple as...

 npm install serialport  

When plugged into the Pi via its USB socket, the Arduino serial connect you're used to connecting to on your computer is the same one that the Pi can read/write to. Make your Arduino sketch listen for incoming commands on the serial port and provide some feedback via print/println as usual.

A quick-n-dirty test routine later...

   var serialPortModule = require("serialport");  
   var SerialPort = serialPortModule.SerialPort;  
   var serial = new SerialPort("/dev/ttyUSB0",{  
     parser: serialPortModule.parsers.readline("\n")  
   });  
   serial.on("data", function(data)  
   {  
     console.log("Received: " + data);  
   });  
   console.log('Serial Port Started');  

Note, your Arduino may be somewhere else (ie not on /dev/ttyUSB0), unplug the Arduino and ls /dev. Plug the Arduino in, and ls again. Whatever appears is your Arduino. Need to create something a little more robust for the future but for testing this'll do just fine.

We use the readline parser and not the default parser. Try with the default and you'll see why!

Putting the above Javascript into an Node (and coupled with the Arduino writing the boiler state to the serial port every 10 seconds) you'll get something like:

 Serial Port Started
 Schedule loaded
 Received: OFF
 [10 second pause]
 Received: OFF

With a smidge more code (inside a JS object) and we can write a 1 to the serial connection to trigger the Arduino to turn the boiler on, wait 5 minutes (don't really want to toggle your boiler too much!) and turn it back off. You'll need to expand all of this out with code to prevent toggling your boiler on and off too often etc but this is the basics.

   serial.open(function()  
   {  
     console.log('Serial Port Opened!');  
     self.serialConnected = true;  
     setTimeout(function() { self.turnOn(); }, 5000);  
   });

   this.turnOn = function()  
   {  
     console.log("Writing to the serial connection...");  
     serial.write("1", function()  
     {  
       serial.drain(function(err)  
       {  
         if(err)  
         {  
           console.log('Uh oh', err);  
         }  
       });  
     });  
     setTimeout(function() { self.turnOff(); }, 300000);  
   };

   this.turnOff = function()  
   {  
     serial.write("0");
   };

The boiler front panel has an LED that flashes every time it receives communication from the thermostat and the manual states this will flash every 5 minutes when the thermostat is on. The boiler shuts down automatically if it doesn't receive this periodic signal after a while. I've not timed it but it seemed to be around 10 minutes but certainly longer than the 5 minutes it waits between thermostat signals.

So at this point I've also modified the Arduino sketch to keep a record of its required boiler state and to re-transmit that every 5 minutes to mimic the behaviour of the standard thermostat. This way the Pi simply tells the Arduino 'boiler on please' and then leaves it up to the Arduino to keep it alive until told otherwise.

Thursday 27 March 2014

Turning recorded sound into code

So we've got a nice pulse train all recorded. Its time to put this into code so we can get our little Arduino to transmit the same code and make the boiler think its just received a command from the thermostat as usual.


If you highlight a section of the recording in Audacity, it tells you the start point and either the end point or the length of the section as well. Except it doesn't go down to enough decimal places, even when set to milliseconds it said either 0.001 or 0.000 for me. Not overly helpful when that last decimal place is being rounded.

What I did was switch that section to just display samples. Now it shows you the number of samples you've highlighted. If your project rate is 192,000Hz that means you have 192,000 samples per second.

1 second / 192,000 samples = 0.0000052083333 seconds per sample or 5.2083333 microseconds. 
Highlight from the beginning of a state change in Audacity and multiply that value by 5.2083333 to get the value we need.


In this screenshot I'm highlighting an 'on' state (on is negative dB) and seeing the 98 samples listed as length calculates as 98 * 5.2083333 = 510 microseconds. You can get away with things being slightly out, a sample here or there doesn't seem to affect the boiler as its within its error tolerances it would seem. Either that or I got lucky with my code the first time ;)

Work out the time of every state change, on/off all the way along the set of pulses until you hit that 100ms gap at the end.

You'll notice the first two thirds of both the on and off pulses are the same. The thermostat sends an identifying signal before the payload so the boiler knows the signal is coming from it.

My code puts this first section into one array, then seperate arrays for the on and off signals. Whenever it sends a signal it runs the identifier array through a transmit function followed by the appropriate command.

This is my basic 'test' code, paste your timings into the arrays at the top and upload to your Arduino. Open a serial console and send the number 1 to get the system to transmit the on, anything else to turn it off. You should see your boiler react accordingly!

 //define the arrays thatdetermine how long to transmit/wait between pulses  
 //values are in on/off pairs, microseconds (1/1,000,000 of a second)  
 //1000 microseconds = 1 millisecond (ms)  
 //1ms = 1000/second  
   
 //192Hz = 192,000 times a second  
 //so each sample = 5.2083333 us long  
 //96 samples therefore = 500us  
 int comms_start[] = {  
  520,500,  
  520,500,  
  520,500,  
   
 // ... many... many more rows...  
 //containing my boiler ID and knowing my friends
 //someone would find it funny to control my boiler for me ;)  
   
  520,1000,  
  520,500,  
  520,500  
 };  
   
 int comms_on[] = {  
  1000,1000,  
  520,500,  
  520,500,  
  520,500,  
  1000,1000,  
  1000,500,  
  520,500,  
  520,1000,  
  520,500,  
  520,0  
 };  
   
 int comms_off[] = {  
  520,500,  
  520,500,  
  520,500,  
  520,500,  
  520,500,  
  520,500,  
  520,500,  
  1000,500,  
  520,500,  
  520,1000,  
  520,500,  
  520,0  
 };  
   
 #define INT_SIZE sizeof(int);  
   
 //define the digital pin the transmitter data pin is connected to  
 int txPin = 4;  
   
 void setup()  
 {  
  Serial.begin(9600);  
  pinMode(txPin, OUTPUT);  
 }  
   
 void loop()  
 {  
  if(Serial.available() > 0)  
  {  
   //something in the serial buffer  
   int incoming = Serial.read();  
     
   if(incoming == 49) //char code for 1  
   {  
    //turn the heating on  
    Serial.println("ON!");  
    turnHeatingOn();  
   }  
   else  
   {  
    //turn the heating off  
    Serial.println("OFF!");  
    turnHeatingOff();  
   }  
  }  
 }  
   
 void turnHeatingOn()  
 {  
  sendPreambleAndBoilerID();  
  sendOn();  
 }  
   
 void turnHeatingOff()  
 {  
  sendPreambleAndBoilerID();  
  sendOff();  
 }  
   
 void sendPreambleAndBoilerID()  
 {  
  Serial.println("Sending Preamble");  
  sendArray(comms_start, sizeof(comms_start));  
 }  
   
 void sendOn()  
 {  
  Serial.println("Sending On");  
  sendArray(comms_on, sizeof(comms_on));  
 }  
   
 void sendOff()  
 {  
  Serial.println("Sending Off");  
  sendArray(comms_off, sizeof(comms_off));  
 }  
   
 void sendArray(const int start[], int arrSize)  
 {  
  int arrLength = arrSize / INT_SIZE;  
  for(int i = 0; i < arrLength; i = i + 2)  
  {  
   transmit(start[i], start[i+1]);  
  }  
 }  
   
 void transmit(int uS_transmit,int uS_wait)  
 {  
  //turn on transmitter  
  bitWrite(PORTD,txPin, 1);  
    
  //wait the right amount of time  
  delayMicroseconds(uS_transmit);  
    
  //turn off the transmitter  
  bitWrite(PORTD,txPin, 0);  
    
  //and wait out its delay before the next 'on' can occur  
  delayMicroseconds(uS_wait);  
 }  


Proof of concept = done!

Wednesday 26 March 2014

Hacking the signals from a DT10RF

I ranted about the DT10RF here, in my last post but instead of replacing the thermostat with something 'better' I've decided to stick with it and build something myself.

To give you some idea, all you need for a basic controller is an Arduino, a 433Mhz transmitter, a TMP36 temperature sensor, some wires and a little coding skill and you can build something that works to the same functionality as the DT10RF but instead of being £160 you're looking at less than £25 for the parts. No screen, no interface and without hours of coding/design you've still got nothing... but where's the fun in buying something already built? ;)

First step.
Work out what the controller sends to the boiler to make it turn on / off.

Fortunately I'm treading in the footsteps of those before me somewhat and all I have to do is replicate his work. Different model thermostat but same manufacturer and thus expected to be broadly the same (it was :))

This post is pretty much a rehash of his, I have quite literally followed what he did - I'm just providing my own take and pictures on the subject to help others in the future.

Like Steven, I didn't have an oscilloscope so I built the same logic gate analyser he described. As he explains, the idea is that you can use a soundcard with a cable plugged into its mic socket to record the signal. Sound cards only like up to 2v being thrown at them, logic signals (the kind we're reading anyway) are 5v so we have to bring that voltage down.

I started with an old stereo cable and cut it in half. Separated the two cables and their sheaths.


Solder a 18k resistor and length of wire to the sheath on both sides. Then solder an 82k resistor onto the end of the 18k and also attach the 'main' wires of the cable to the mid point between the two resistors.


Attach two more cables to the end of the 82k resistor, wrap it all up and voila, you have a two channel logic analyser.

All you need at this stage is to power the receiver, here I've got it plugged into a breadboard along with an Arduino Nano. The Nano provides 5v (red cable) over to the receiver. The white cable is the antenna, black cables are all ground, green cable is the data cable going back to the Nano (not used yet).



The four cables heading off in the top right are the cables from the logic analyser. Both connected across the data pin on the receiver to ground.

Once you've got that wired up, connect the other side of the stereo cable to the mic socket on your computer and plug in the Arduino to power it.


On the laptop run Audacity (or similar sound recording program) and hit record whilst you turn the heating on or off. As you can see in the picture, you'll (hopefully!) get distinct patterns as the thermostat does its thing.

I did this with the receiver antenna drapped over the thermostat to get a really nice clear signal. It worked from the next room as well but with a lot more noise. When it comes to interpretting what was going on, clearer = better!

Like the older model Steven has, the newer thermostat does the same thing - it pings each command three times with short gaps (roughly 2 seconds) between each attempt. This is because the boiler doesn't feedback when it receives a command - the thermostat can transmit only. It just blindly transmits its signals and keeps its fingers crossed and does so three times to improve the chance of one of those signals making it.

Zooming in (a lot!) on those spikes shows clear pulses:

You're looking for this BEFORE the clear sections on the recording. There's a lot of noise on these cheap receivers when they're doing nothing and automatic gain control takes 100ms or so after its finished receiving to ramp back up into noise making duty. They work fine though (and for £2 off ebay you really can't go wrong...)!

Also recommend setting audacity to something higher than its default of 44Khz (I used 192Khz). The higher you go the more samples it records thus the more accurate your pulses will be. You need to measure the timings of them next...

New boiler... and the beginning of a slippery slope...

Last Friday the plumbers left after spending a week here putting in my new Worcester Bosch Greenstar Heatslave II.

Compared to the old boiler (30 years old and DRINKING oil by the tanker) this thing is quiet, sips at its fuel and as an added bonus even has a working thermostat and controller. Ok... the old one basically was a boiler that, through failures occurring before I bought this house, had been reduced to something that produced heat only when someone physically turned it on and did so by bankrupting the poor owner in the process.

The plumbers gave me a choice of thermostats and I plumped for the nice wireless one. No complicated install and it given the thing cost £160 it should be damn good right?

WRONG.

The DT10RF MkII is useless. I mean it does its job but its only capable of programming the central heating (why not the hot water timings as well? My boiler is not accessible from the main house and to go adjust it physically... defeats the object of having the shiny wireless controller!) and you have to put up with the same heating schedule on the weekends as during the week (without adjusting things anyway). Add to that the missing 'give me heat for an hour' button that I've seen on the most basic controllers and you've got yourself the most expensive piece of junk you've ever seen.



Annoyingly the next model up has the proper individual day programming and only costs £20 more... something I was going to rant at the plumbers for until, as part of my Googling, I came across a fantastic page...

http://www.stevenhale.co.uk/main/2013/08/home-automation-reverse-engineering-a-worcester-bosch-dt10rf-wireless-thermostat/

Steven had issues with signal and with the optimiser feature and set about replacing it with something he built himself.

Now I'm a software developer with a very healthy interest in engineering (built a couple internet enabled cat feeders for example) and seeing the faults in my own thermostat and reading a webpage describing how to build a replacement from scratch... well thats just a big red flag to a bull shaped Roger...

The purpose of this blog...

... is a place for me to collect my thoughts as I go about automating my home.

Hopefully helping others to do the same but also for me somewhere down the line when I'll think 'how did I do that again?'

Welcome!