Personal Weather Station
A weather station based on the Raspberry Pi
6.4 Temperature & Humidity microcontroller software
We will start with the microcontroller test program from section 5.3. The microcontroller test program from section 5.3 does a lot of what we want already. It controls waking up and putting to sleep the XBee transmitter, sending a message to the base station as well as timing a specific interval between sleep periods.
We can start with this program and add reading the temperature and humidity sensor through the I2C port. We will also need to increase the time the XBee transmitter is asleep in order to conserve battery life. In the test program we had the XBee go to sleep for about 16 seconds in order to be able to test sleep mode while not having to wait long periods of time between test messages. We will increase the time between sleep periods to be about 5 minutes.Lets take a closer look at the code for the microcontroller remote temperature and humidity sensor unit. Once we have this circuit working we can modify the code to build additional different sensors only needing to modify the sensor reading routine and the message packet sent to the base station. We will keep this in mind so that we generate a modular program. Here we will examine snippets of source code, the full source code is available in the downloads section. If you are not using a dsPIC30F3013 you will have to port this program to your target microcontroller.
6.4.1 The main program
The main program is shown in figure 6.7. The main program performs the following tasks; initializing the microcontroller, reading the sensor data, sleeping the XBee transmitter, timing the period between sleeps and transmitting the sensor data thru the XBee to the base station.
int main(int argc, char** argv) { char transmitData[PKT_SIZE] = {'N','u','l','l'}; uint8_t sleepCount =0; initDevice(); RCONbits.SWDTEN = 1; //Turn on watchdog timer Sleep(); RCONbits.SWDTEN = 0; //Turn off watchdog timer __delay_ms(100); //Do an initial read and transmit whenever the device is reset/powered up. read_humidity(&transmitData[0]); //Read temp and humidity transmit_to_xbee(&transmitData[0],PKT_SIZE); while(1) { if (sleepCount < NUM_SLEEPS) { Sleep_RQ = 1; //Put the Xbee module asleep RCONbits.SWDTEN = 1; //Turn on watchdog timer Sleep(); //Put the uC to sleep RCONbits.SWDTEN = 0; //Turn off watchdog timer sleepCount++; } else { Sleep_RQ = 0; //Wake up the Xbee module __delay_ms(100); //Wait for the Xbee to wake up before trying to transmit serial data to it read_humidity(&transmitData[0]); //Read temp and humidity transmit_to_xbee(&transmitData[0],PKT_SIZE); sleepCount =0; } } return (EXIT_SUCCESS); }
Figure 6.7: Microcontroller HIH6131 Sensor main program
The first section of the program initializes the device through the function initDevice(). The initialization function should configure all of your IO pins on the microcontroller as well as turn on the UART module and the I2C module. The UART module should be configured to 9600 Baud , 8 data bits , 1 stop bit and no parity. The I2C module should be configured to run at 100 Khz.
You should configure all of the unused pins on the microcontroller to outputs and set them to drive out a logic zero. The reason for this is to minimize power consumption. If you leave the IO pins configured as inputs and left unconnected, they can pick up small amounts of electrical noise and switch randomly sometimes at high frequency. Since you are not using the pins, you won't notice anything wrong with the operation of your program, nor will you do any damage to the microcontroller. However, each of these unconnected inputs will consume small amounts of switching power which will start to add up the more pins you have unconnected.
This unnecessary switching power wastes battery life so it needs to be fixed. There are two ways to fix the unused IO pins so that they don't switch randomly and consume power. First, you can tie all the unused inputs to ground on your PCB, but this would result in a lot of wiring on your PCB which can be time consuming. Secondly, you can set them as outputs instead and set an initial value of logic zero on the pins. This forces a fixed value on the pins and prevents switching noise and power consumption. We also put the XBee module to sleep by setting its SLEEP_RQ pin to logic one.
We also need to use the watchdog timer to control the sleep periods. We configure this by setting the fuses to enable it , and at the same time set the pre-scaler values to control the time period of the watchdog timer.
_FWDT(WDT_OFF & WDTPSA_512 & WDTPSB_16)
The above fuses set the watchdog timer off, which means that we can turn it on and off by software in our program rather than it being enabled all the time. We want to control it in software since we are only going to use it to control the sleep period.
Normally the watchdog timer would reset the entire device , but it behaves a little different when the microcontroller is asleep. When the microcontroller is asleep and the watchdog timer runs out, instead of resetting the device it wakes the device and starts program execution from the instruction right after the sleep() command. If the watchdog timer were to run out while the device is awake, it would force a device reset. For this reason, we will turn the watchdog timer on just before we go to sleep, and let the watchdog timer wake us up when its timer runs out.
We also set the watchdog timer pre-scalers to their maximum values of 512 and 16 . We want the maximum sleep time. The watchdog timer runs off of an internal 128 KHz and is 8 bits wide, therefore it takes about 2ms to normally timeout. If we set both pre-scalers to the maximum values we get:
Watchdog timeout period = 2ms * 512 * 16 = 16.384 Seconds
We will wrap the sleep routine with a counter to count 20 sleep periods. Each time we wake up, we will increment the sleep period counter and if it is not 20, we will go back to sleep immediately without powering the XBee module up or reading the sensor values. This way we can time 5 minutes between transmissions:
Total sleep time = 16.384 * 20 = 327.68 Seconds =~ 5.5 minutes
If the sleep period counter is 20 or greater , that means that we have slept for about 5 minutes. We then wake the XBee module by asserting SLEEP_RQ = 0. We use a delay routine to wait about 100ms for the XBee to fully wake up, then call the routine to read the HIH6131 sensor through I2C.
read_humidity(&transmitData[0]);
This routine will return the temperature and humidity data from the HIH6131 sensor in the array transmitData. We will then pass this array into the transmit function to send the data from our XBee back to the base station.
transmit_to_xbee(&transmitData[0],PKT_SIZE);
After transmitting the data, we then reset the sleep count to zero, put the XBee module back to low power sleep by asserting SLEEP_RQ = 1, and begin counting another low power sleep period of about 5 minutes. Let's take a closer look at the read_humidity() and transmit_to_xbee() functions.
6.4.2 Reading the temperature and humidity function
The function read_humidity() performs the I2C operations that read the temperature and humidity from the HIH6131 sensor and return the data to be transmitted back to the base station in a 4 byte array.
In the read_humidity() function we are essentially implementing the two I2C operations as shown previously in figure 6.2 and figure 6.3. We start by implementing Figure 6.2, the I2C register write to the HIH6131 sensor to initiate a humidity and temperature measurement. We then follow by a delay of 75ms which is almost double the maximum time (37ms) the HIH6131 takes to complete a measurement according to the data sheet. Following that we do a register read of the HIH6131 sensor to get the 4 bytes of data. The first two bytes are the sensor status and the humidity value, the second two bytes are the temperature data. We will store them in the transmit data array for transmission to the base station using the function transmit_to_xbee().
The I2C functions are specific to the dsPIC30F family of microcontrollers and other microcontrollers will require different code to operate their I2C modules, but the concept is the same. Implement the sequence shown in figure 6.2 delay more than 37ms , then implement the data read shown in figure 6.3 and finally return that data in a 4 byte array for the transmit function.
In the case of the dsPIC30F series, we write a bit telling the microcontroller's I2C module to do a particular operation, then we poll that bit to see when it is cleared. The I2C module's hardware will clear the particular operation's bit when the operation is complete. Once this happens, we can then set the next control bit to generate the next I2C operation and wait for that to be done too.
For example, to generate the first bit in figure 6.2 which is the start bit, we would use the following code:
I2CCONbits.SEN = 1; //Send start bit while (I2CCONbits.SEN == 1){Nop();}; //wait for start bit to be done
Once we set SEN to 1 , the I2C module will perform the necessary I2C protocol on the I2C pins to generate the start bit, once that is done, the I2C module will clear the SEN bit, which our software is waiting for.
Once the start bit is complete our software goes on to the next stage which is to write to the sensor using the sensor address of 0x27 plus the read/write bit set to 0 to indicate a write, this gives us a I2C data value of 0x4E (see figure 6.2).
I2CTRN = 0x4E; //Addr 0x27 Read =0 while (I2CSTATbits.TRSTAT == 1) {Nop();}; //Wait for transmit to be done
In this case we load the I2C transmit register with our data (0x4E). This automatically sets the TRSTAT bit to 1, which indicates a transmission is in progress. The hardware I2C module will clear this bit once all the data has been shifted out of the I2C module to the sensor and the sensor has responded with an ACK bit.
Next we generate the final stop bit to end the I2C transmission.
I2CCONbits.PEN = 1; //Send stop bit while (I2CCONbits.PEN == 1) {Nop();}; //Wait for stop to be done
We follow the same method of waiting until the I2C module clears the stop bit to indicate it has finished sending the stop bit before proceeding. At this point we have successfully generated the I2C register write to the sensor to initiate a measurement request. After waiting for about 75ms using the delay_ms() routine, we can now read back the HIH6131 sensor data.
Reading back the sensor data follows much the same routine as initiating a measurement request except we send the sensor address (0x27) plus the read/write bit set to one to indicate a register read of the sensor. This gives us a value of 0x4F as the I2C data to the sensor.
After we have sent the read request data (0x4F) to the sensor, we then need to turn on the receive path in the I2C module and wait for the data from the hardware.
I2CCONbits.RCEN = 1; //Turn on I2C receive while (I2CSTATbits.RBF == 0) {Nop();}; //Wait for the data tmp0 = I2CRCV;
The RBF flag is I2C read buffer full. We wait until this is set to 1 to indicate that the I2C read buffer is full of data. We then read the I2C receive buffer register I2CRCV into a temporary byte tmp0. We then need to send an ACK back to the sensor.
I2CCONbits.ACKDT =0; //Send Ack I2CCONbits.ACKEN = 1; while (I2CCONbits.ACKEN == 1) {Nop();}; //wait for ACK to be done.
We follow the same steps to read the remaining bytes out of the sensor into temporary variables. The only note is that last byte read from the sensor requires a NACK to be sent instead of a ACK, which is done by setting ACKDT =1 to tell the I2C module to send a NACK rather than a ACK (ACKDT =0)
After sending the final stop bit, we return all of our temporary variables into the transmitData array in the same order as we received them from the sensor.
It is important to exactly implement the sensor protocols shown in figure 6.2 and figure 6.3. Failure to do so may result in the sensor not responding or returning incorrect data. Make sure to note what data the microcontroller needs to send over I2C and what the microcontroller expects returned from the sensor. In figure 6.2 and figure 6.3, bits in green are generated by the microcontroller and bits in blue are generated by the sensor and read by the microcontroller.
6.4.3 Transmitting data to XBee function
The transmit_to_xbee() function is the function that uses the UART to send data to the XBee module for transmission back to the base station. The function takes in the 4 bytes of data we put in the transmitData array using the read_humidity() function and the size of the packet to transmit, which in this case is 4 bytes. Since our remote XBee is set into AT or transparent mode, we only need to push those 4 bytes directly out through the UART into the XBee which will take care of transmitting the data back to the base station. We have already configured the UART to 9600,8,N,1 in the initDevice() function so we just need to make use of the microcontroller's UART routines to push the data into the XBee.
If you are using a different microcontroller, you will need to port this function to use your particular microcontroller's UART.
The first section of this function simply pushes the data into the UART.
while (i<txSize) { if (U2STAbits.UTXBF == 0 && CTSn == 0) { U2TXREG = *(transmitData+i); i++; } }
We push data into the UART transmit fifo until it is full (UTXBF==1). We must also make sure that the XBee is ready to receive data by checking that its active low clear to send is zero. (CTSn == 0) Once we push all of our data into the UART transmit buffer fifo (U2TXREG), we need to wait until the transmission completes.
while (U2STAbits.TRMT == 0) { Nop(); //Wait until transmission is complete. }
Since the U2TXREG is a fifo, this means that even though we have pushed all of our data into it, the UART may still be shifting data out to the XBee. If we don't wait until this is complete by observing the TRMT flag, we will return from the transmit_to_xbee() function while the UART is still busy, then our main loop will put the XBee to sleep. This will result in some data being lost, as the XBee will go to sleep while we are still pushing data into it. It is important that we wait until the hardware UART in the microcontroller tells us that the data is completely transmitted before we return from the transmit_to_xbee() function. As an added margin of safety, we wait 100ms after the UART is finished before returning to ensure that the XBee is finished transmitting before the main loop puts the XBee back to sleep.
6.5 Raspberry Pi Perl script
Now that the humidity and temperature data is successfully sent through the XBee to the receiving XBee at the Raspberry Pi base station, we need to capture that data on the Raspberry Pi and log it to a file for future processing.
Since our XBee on the Raspberry Pi is connected to the serial port and is configured in API mode, we expect to receive an API packet over the serial port that looks something like figure 6.8.
Figure 6.8: Received API packet from remote HIH6131 sensor
We need to capture this packet from the serial port, process it and convert the raw values into actual temperature and humidity values that we log to a file. We will also include a time stamp along with the recorded temperature and humidity values.
We will create a Perl script similar to the at_serial.pl script that we created in section 5.4 to receive test messages from our XBee test setup. This Perl script will run in the background on the Raspberry Pi and will be responsible for receiving messages from the remote sensor XBee's and logging the data to a file.
This Perl script will be a little more complicated than the simple at_serial.pl test program. We will need to handle receiving data from multiple remote sensors, decoding the data and logging it to a file. We will also need to think about what to do if a sensor goes offline for a short period of time. We don't want to be logging stale data. The Perl script also needs to be able to adapt to adding additional sensors as well as the fact that the sensors are asynchronous in nature.
Each sensor sends data about every five minutes, however that time period is relative to when the sensor first powered up. So from the Raspberry Pi's Perl script point of view, data from any sensor can come in at any time and in any order. We have to expect a packet at any time and be able to look at the packet to determine what sensor it came from so that we can apply the correct equations to convert the data into the appropriate sensor readings.
The full source code to receive_remote_sensors.pl is located in the downloads sections. We we start it and run it in the background on the Raspberry Pi:
receive_remote_sensors.pl &
Let's take a closer look at the program and how it is structured.
6.5.1 Main loop overview
The main loop of receive_remote_sensors.pl performs the following functions:
- Initializes the serial port and the log file.
- Listens for a packet from a remote XBee on the serial port
- Upon receiving a new packet decodes it.
- Writes current sensor data to the log file every 5 minutes.
A flowchart of the main program loop is shown in figure 6.9.
Figure 6.9: Flow chart of receive_remote_sensors.pl Perl script.
The routine get_packet() waits on receiving data from the serial port but it also periodically times out and returns control to the main loop even though there is no new packet received. We need get_packet() to time out so that the main loop can take care of log file processing and time keeping. We also want get_packet() to time out so that our program does not hang in the event we stop receiving sensor data for some reason. The routine get_packet() returns the received data into a global array called dataPacket.
If get_packet() returns due to a new packet being received, we check the address of the received packet by looking at the address value located in dataPacket[4..11] . The array dataPacket contains the received packet in the same format as figure 6.8. Here dataPacket[0] contains the start delimiter, 0x7E, and dataPacket[1..2] contains the packet length and so on. We then compare that address to our known sensor addresses to determine which packet decoding routine to call based on the type of sensor at that address. So far, we only have a single sensor module which we will decode using the routine decode_HIH6131().
Make sure to change the sensor addresses in your program to match the address of your XBee module. Each XBee module has a unique address which acts like a 64 bit serial number. You can read this value out of the XBee module using X-CTU, or in many cases, the serial number is printed on the bottom of the XBee module.
Next we want to update the log file with all of the current sensor readings every 5 minutes. If the current minute is a multiple of 5 and the log file has not been written in the last 5 minutes, we dump all the current sensor values to the log file.
One important issue that comes up is how to deal with stale data. A sensor can go offline for a short period of time due to electrical noise, or for an extended period of time in the event of a depleted battery. If a sensor goes offline for an extended period of time, we want to detect that it is offline and mark the data for that sensor in the log file as invalid. Remember that each sensor reports in about every five minutes but the actual time that a sensor reports in depends on when it was initially powered up.
For example, let's assume that sensor A was powered up at exactly 5:00 , and sensor B was powered up at 5:03. In this case, sensor A will report in at 5:00, 5:05, 5:10 etc. However, sensor B will report in at 5:03,5:08 and 5:13. As you can see each sensor reports in every 5 minutes, but not necessarily at an even multiple of 5 minutes. So our base station has no way of knowing when a particular sensor will report in other than each sensor should report back in 5 minute intervals.
We can tolerate one or two missing sensor readings, but if the sensor goes offline for too long we want to identify that in the log file. We can do that by incrementing a stale data flag for each sensor every time we write the log file, which is at 5 minute intervals. If we get a new sensor reading, we clear the stale data flag for that sensor. If the stale data flag for a particular sensor reaches the value 3, that indicates that we went through 3 log file cycles with no update from the sensor which is a total dead time of 15 minutes. In this situation, we null out the corresponding sensor value in the log file.
Let's take a closer look at the routine that waits for the packet and the routine that decodes the packet data and returns the sensor's converted values.
6.5.2 The get_packet() routine
Figure 6.10 shows the source code for the get_packet() routine.
sub get_packet { $breakout =0; $index=0; $startofpacket=0; $maxindex = 128; while($breakout ==0) { my ($count,$byte)=$port->read(1); $hexbyte =unpack("H2",$byte); if ($count != 0){ $dataPacket[$index] = $hexbyte; if ($hexbyte =~ /7e/) { $startofpacket =1; } if ($startofpacket == 1) { $index++; } if ($index == 3) { $maxindex = (hex($hexbyte)+4); } if ($index == $maxindex) { $newpacket =1; $breakout=1; } } else { $timeout--; if ($startofpacket != 1 && $timeout == 0) {$breakout =1;} } } }
Figure 6.10: The get_packet() routine.
The get_packet() routine sits in a loop waiting for characters to come in on the serial port. We breakout of the loop and return from the get_packet() routine if either a full packet is received, or a timeout occurs.
We attempt to read from the serial port using:
my ($count,$byte)=$port->read(1);
The function which is part of the library Device::Serial, normally blocks until a character is received, however we have configured it to return after 1000ms if no characters are present on the serial port. We did this at the beginning of the Perl script with the setting:
$port->read_const_time(1000);
When $port->read(1) returns, it returns the character received in $byte and the number of bytes read from the serial port in the variable $count. Since we configured the serial port to timeout every 1 second, and to read only 1 byte at a time, $count will be either 1 to indicate a byte was read, or 0 to indicate no byte was read and a timeout occurred.
The byte that is returned is the raw hex value from the serial port. Since Perl proceses strings, we need to convert this raw value into its string format. We use the function unpack() to convert from the raw hex value into a string that contains the hex number so that the rest of the Perl script can process the value.
$hexbyte =unpack("H2",$byte);
The above line of code will take the raw value from the serial port in the variable $byte, and convert it to a two character hex string (H2) , and return it into the variable $hexbyte.
Next we check the value of $count returned from the serial port read. If $count is 0, this indicates that no data was returned from the serial port, so in that case, we skip processing the data and instead decrement the timeout variable ($timeout--) . $timeout was set to 30 before we called get_packet() . Since the serial port read times out once per second, the function get_packet() will timeout every 30 seconds if no packet arrives. We also check to make sure that a packet is not currently in the process of being received before timing out. We don't want to have the get_packet() routine return due to a timeout when it is in the middle of receiving a valid packet.
If $count is 1, it indicates a valid byte was received, converted to a string and is currently residing in $hexbyte. We then put that value into the array dataPacket at the current index location $index.
$dataPacket[$index] = $hexbyte;
We initialized the $index variable to 0 when get_packet() was called, and we only start incrementing the index when we detect a start delimiter byte (0x7e). We check for the start delimiter so that we don't accidentally start receiving a packet that for whatever reason is already in the middle of being transmitted when we start reading the serial port. By looking for the start delimiter (0x7e) we can guarantee that the packet is received from the start and it loaded into the array dataPacket in the correct order. The decoding routines expect that a packet will be correctly loaded into the array dataPacket according to the ordering in figure 6.8.
We know that when $index reaches 3, we have just read the packet length field in the XBee receive packet (refer to figure 6.8 and remember that $index points to the next location to be filled, not the current location) . We take the packet length value and convert it to a decimal number to calculate the max index value:
$maxindex = (hex($hexbyte)+4);
The hex() function takes a hex string and returns the decimal equivalent. We need to add 4 to the length value because the length value in a XBee receive packet does not include the first 3 bytes of the packet (start delimiter and two bytes for the length value) , and $index always points to the next location. Therefore we need to add the 3 extra bytes at the beginning of the packet plus 1 to compute the maximum index value.
By computing and using the length value found in the XBee receive packet, the get_packet() routine can adapt automatically to varying packet sizes that can arrive from different sensors. This means that we can send different amounts of data from different sensors and the get_packet() routine can receive any of them. A pressure sensor may only require to send two bytes, while our humidity and temperature sensor sends four bytes.
We continue reading bytes from the serial port and putting them into the array dataPacket until we reach the maximum index value. Once this happens, we set $newpacket=1 to indicated we received a new packet, breakout of the loop and return from the function.
The new packet is now stored in the global array dataPacket and the flag $newpacket is set to indicate to the main program that a new packet is available for processing. The $newpacket global variable will be cleared by the main program after the packet has been processed.
6.5.3 The decode_HIH6131() routine
Figure 6.11 shows the simple decoding routing for the HIH6130 sensor. This routine is called by the main program when a new data packet arrives and the new data packet's 64 bit address matches the address of our HIH6131 sensor XBee module. Since the main program loop handles address decoding, we can assume that when this routine is called, the array dataPacket contains our HIH6131 sensor data.
sub decode_HIH6131 { $humidity = ((hex("$dataPacket[15]"."$dataPacket[16]"))/16383)*100; $temperature = ((hex("$dataPacket[17]"."$dataPacket[18]")/65532)*165)-40; $humidity = sprintf("%.1f",$humidity); $temperature = sprintf("%.1f",$temperature); $humidity = "$humidity"."\%"; $temperature = "$temperature"."C"; }
Figure 6.11 The decode_HIH6131() routine
Our HIH6131 sensor data is contained in the XBee receive packet shown in figure 6.8. The humidity value would therefore be stored in dataPacket[15], dataPacket[16] and the temperature data is stored in dataPacket[17], dataPacket[18]
We use the hex() function to convert the hex values into decimal values and then plug them into the HIH6131 humidity and temperature formulas as show in figure 6.1 and 6.2 respectively.
The one issue here is our calculations will return a fair bit of precision in their results. For example the temperature value might end up being 21.33567 degrees Celsius. Later on this will present another problem when we try to graph the data. Having too many decimal places will show too much fluctuation on our graphs when in reality the temperature may be relatively constant. If we leave too many decimal points, our graphs will look noisy and detract from being able to watch trends of temperature and humidity. We will investigate this more in section 7 when we start working on the post processing and graphing of our weather station sensor data.
We will round off the temperature and humidity values to 1 decimal point using sprintf(). Sprintf allows us to round off the data to one decimal point by using the format specifier "%.1f" This means format the variable as a floating point number with only 1 decimal place.
Next we add units to each of the temperature and humidity variables by appending a C for Celsius and a % for relative humidity. The variables $temperature and $humidity now contain a floating point number with 1 decimal place and the associated unit. The main program loop will handle printing these values to the log file.
