Real-World Range Testing By Christopher Hofmeister August, 2011

Real-World Range Testing By Christopher Hofmeister August, 2011 Introduction Scope This paper outlines the procedure for a successful RF range test that provides quantitative data on how the RF link performs in its intended environment. This enables informed decisions about network architecture. The paper also explains subtleties that might be overlooked during a RF range test that can affect performance. Although these principles apply to any radio, we give special attention to features of the 802.15.4 standard that can affect range test results. The design of a RF range test is broken down, and specific elements are explained in a realworld range test using LS Research s RF modules. So Many Variables One of the most frequently asked questions about a given radio is How far will this radio transmit? Unfortunately, it is also one of the most difficult questions to answer, as so many variables apply. Besides obvious variables like transmit power and receive sensitivity, a host of other factors come into play. Without careful attention to understanding and controlling as many of these variables as possible, the results of an RF range test will leave more questions than answers. For example, consider this common scenario. An engineer evaluating a radio puts the radio in a spot, walks some distance from it while monitoring an LED that indicates a received packet, and hopes to find a magical line on the ground at which the radio works on one side and doesn t work on the other. In real life the link will become marginal at some point and continue to work intermittently, perhaps even improving at certain points, forcing the engineer to make a judgment call on how far the radio worked. A week later the same test performed by the same or a different individual yields drastically different results. What happened? As we attempt to reconcile the results, questions come up. Even if the test was done outdoors, many factors apply. What antennas were used? How were they oriented? How far off the ground were they? How were the radios powered? If batteries were used, what was their voltage? Was the same version of firmware used? What was the over the air data rate? How many bytes were in the RF packet? What was the RF environment like during each test? How was it determined that the link was bad? Were RF acks and retries used? What obstructions Page 1 of 16

were in the path of the radios? The answers to these questions reveal that the trials had little in common, and the results are inconclusive. Add to this the complexities of an office or industrial environment. The RF environment is constantly changing as laptop computers move about and cordless phones or even microwave ovens are operated, affecting the 2.4GHz band in particular. Doors open and close, office equipment is turned on and off, equipment is moved, and so on. Also, the building s physical construction certainly has an effect; an addition may make it appear that an outside wall of concrete, stucco or metal siding is an interior wall. These situations illustrate why it is so hard to answer the question How far will this radio transmit? Clearly, as many variables as possible need to be controlled, and those that cannot be controlled at least need to be understood. When radio manufacturers list a range distance, be sure to ask: How were those results obtained? Is this the environment in which my radios will operate? What was the performance of radios at this distance? Good Range Test Design Specifying Range Most manufacturers specify range in an optimum situation, specifically line of sight outdoors. Besides putting their best foot forward, this enables them to repeatedly reproduce the test conditions and obtain similar results. This does not mean, however, that users of the radio can obtain the same results in a typical environment. Before running the range test, determine the criteria for specifying how far the radio transmits. This is especially important when trying to compare different radios. Some may specify this as the farthest point at which 99% of packets are successfully transmitted. Others may plot out the average receive sensitivity over distance to determine range. A combination of both metrics could also be employed. Write Everything Down It should go without saying, but it s easy to take shortcuts under the premise that we will remember. In reality, hours turn into days, and days into weeks, and we can t recall the details. So take the time to write everything down. You could use the example at the end of this paper to create a form that ensures all important details are noted. Collect Quantitative Data A good range test will collect useful information, such as the number of packets transmitted, the number of packets received, RSSI, LQI, the amount of noise on the channel, and other test conditions such as timestamps on the packets, the number of bytes in the packet, and power Page 2 of 16

supply conditions. Such information should be logged into a PC to enable analysis of the data collected. Understand the Difference Between LQI and RSSI RSSI is an acronym for Received Signal Strength Indicator. LQI is an acronym for Link Quality Indicator. Some use the terms interchangeably, but they are different. RSSI is an absolute value that can define the magnitude of the received signal, and it can be converted into units of dbm. LQI is a relative number between 0 and 255 that represents the quality of the received signal. Since there are no official rules on how it is calculated, every stack vendor usually calculates LQI differently. It is generally a function of RSSI and possibly other indicators such as the quality of the modulation on the received signal. When range testing a radio, it is generally good to track both RSSI and LQI and to understand how the LQI is being calculated. A strong RSSI but weak LQI (assuming LQI is factoring in the quality of the modulation) would indicate the presence of RF noise or interference. Understand Acknowledgment Requests and Retries Many radio standards today, such as 802.25.4, employ a RF acknowledgment/retry mechanism. When they are mentioned, it is important to understand that they can exist in the MAC layer and the application layer of software. In 802.15.4, acks/retries can be enabled on a packet bypacket basis in the MAC. If a message is sent with MAC layer ack, the recipient of the message will send a very small RF acknowledgment back to the sender, confirming receipt. If the sender does not receive this ack, it will resend (retry) the message again, up to three times. If you are doing the math, the means that the same message could be transmitted up to four times: the original message and up to three retries. Figure 1: MAC layer acks/retries It is often assumed that if a message is retried, it was not heard by the recipient. In practice, it is not uncommon for the acknowledgment to go unheard by the originator, prompting the originator to resend the message. This means that it s possible for the recipient to hear the same message multiple times. Page 3 of 16

RF Message with MAC Ack Request 3 RF Message with MAC Ack Request 2 RF Message with MAC Ack Request 1 ORIGINATOR RECEIPIENT Ack 1 Ack 2 Ack 3 Figure 2 It is important to consider that using MAC layer acks/retries may create a false sense of security regarding RF link quality. For example, if the ack/retry mechanism results in each message being sent two or three times, it may appear that the link is very good when in reality it is on the fringes. A clue that this is happening could be gathered from statistics kept on each side of the link. If the originator side shows 100 packets were sent out and the recipient shows 300 messages were received, this is a sure sign that the network relied heavily on the ack/retry mechanism. Although not the focus of this paper, application layer acks/retries have more practical value in hopping networks or in networks where extremely high reliability is required. The MAC layer acks/retries only guarantee that the message made it to the next hop not the final recipient. Also, even in single hop systems, this method does not guarantee that the message made it to the application. For example, a node could receive an RF message, send the MAC layer ack, and then try to allocate memory to send the message to a PC or display but run out of memory. The message originator would have no knowledge of this problem. Page 4 of 16

RF ALARM Message with MAC Ack Request ORIGINATOR RECEIPIENT UART Alarm Msg Attempt NO MEMORY MAC Ack Figure 3: Pitfall of MAC only acks LS Research Range Test LS Research has developed a protocol for evaluating range of its radio modules, and the principles contained therein can be used to conduct a successful range test. One part of the process involves using a simple form, like the one at the end of this paper, to write down important details about the test, including details that are easy to overlook. The second part of the process combines the firmware on the radio module being tested with PC software to display and record the results. The range test is also designed to achieve the following goals: Obtain quantitative range test results ( Error Rate, RSSI, LQI, Background RF Noise). Establish the ability to restart data collection remotely that is, look at performance in various locations without having to walk back to a PC to restart the test. Enable a single user to perform the entire test. Provide a means to log the results. Record as many variables (such as battery voltage) as possible. Use only one PC. Materials for Range Test The range test in this paper was performed with two of the ModFLEX series radios, the ProFLEX01 (2.4 GHz DSSS) and SiFLEX02 (900 MHz DSSS). Development kits for the radios are commercially available. The module firmware and PC software (ModFLEX Test Tool Suite) are available for download from the LS Research website at www.lsr.com. Page 5 of 16

Over the Air Structure Necessary parts of range test results are transmitted over the air to allow either end of the system to collect data. Each time the master transmits a packet, it increments its packet ID. If the application ack is set to 1, the slave device will transmit a packet back to the master, making it a round trip test. The following example shows what these packets would look like. Master to The packet ID indicates how many packets have been transmitted by the master. The power supply voltage of the master is sent to the slave. If the slave receives the packet, these can be determined: 1 packet was transmitted, 1 packet was received (100% success); and RSSI, LQI, and battery voltages. information is 0 as this information needs to be filled in from the slave. MAC Header Type Id Application Ack Lqi Rssi Rx Pkt Count First Rx'd Pkt Id Voltage Master Voltage Payload CRC 0x70 515 1 0 0 0 0 0 3.23 1 10 Table 1: First master to slave packet to Master The slave appends information into the packet: RSSI and LQI of the master to slave packet; battery voltage. If the master receives the packet, these can be determined: 1 packet was transmitted, 1 packet was received (100% success); RSSI, LQI, and battery voltages of itself and the slave. MAC Header Type Id Application Ack Lqi Rssi Rx Pkt Count First Rx'd Pkt Id Voltage Master Voltage Payload CRC 0x70 515 1 100 60dBm 516 0 2.92 3.23 1 10 Table 2: First slave to master packet Missed s If the slave misses several packets, the statistics will correct themselves as soon as another packet is received. For example, if the slave does not hear packets 90 99 but does hear packet 100, the over the air message would look like the one in Table 3. It would be determined that the master sent 100 messages but the slave heard only 90. MAC Header Type Id Application Ack Lqi Rssi Rx Pkt Count First Rx'd Pkt Id Voltage Master Voltage Payload CRC 0x70 100 1 12 90dBm 90 1 3.3 3.2 1 10 Table 3 Page 6 of 16

Restarting Test in the Field Smarts are built into the range test application to start/restart the test if the packet ID of the received message is less than the packet ID of the previous message. This would allow a user to restart the packet error rate calculation at distance intervals without having to reset both ends of the test. Table 4 provides an example of how this would look in the over the air packet. MAC Header Type Id Application Ack Lqi Rssi Rx Pkt Count First Rx'd Pkt Id Voltage Master Voltage Payload CRC 0x70 100 1 0 0 0 0 0 3.2 1 10 0x70 1 1 0 0 0 0 0 3.2 1 10 Table 4 RF Acknowledgments and Retries In the LSR range test, application layer acknowledgments can be turned on or off. Turning them off makes the test one dimensional; turning them on creates a round trip test. Application layer acks/retries can be used in conjunction with MAC layer acks/retries. MAC Header Type Id Application Ack Lqi Rssi Rx Pkt Count First Rx'd Pkt Id Voltage Master Voltage Payload CRC 0x70 1 1 0 0 0 0 0 3.2 1 10 Table 5 Figure 4: Application acks enabled MAC Header Type Id Application Ack Lqi Rssi Rx Pkt Count First Rx'd Pkt Id Voltage Master Voltage Payload CRC 0x70 1 0 0 0 0 0 0 3.2 1 10 Table 6 Page 7 of 16

Figure 5: Application acks disabled Host Serial Message Structure Applicable information from the received range test RF packet and additional information is formatted into a serial message and passed on to the PC. The ModFLEX Test Tool Suite uses this information to display and log the results of the range test. More information on the serial message is available in the respective modules Host Protocol Guide at the LS Research website: www.lsr.com. Figure 6: Range test serial message Page 8 of 16

Figure 7: Range test results in the ModFLEX Test Tool Suite Page 9 of 16

Sample Range Test Form for SiFLEX02 Testing Date of test June 1, 2010 Name of person performing test Christopher Hofmeister, LS Research Location Star Lake, Vilas County, Wisconsin. See Figure 10: Map of test area. Radio tested SiFLEX02 (900 MHz DSSS) Antenna(s) tested Wire antenna, +2dBi dipole (LSR P/N 001 0002) Data rate 40kbs Firmware version 2.0 5 29 2010 Hardware version 2.0 Channel tested 5 Output power +24dBm ED results 0 See Figure 14: SiFLEX02 energy scan results MAC level retries Off Application level retries On making this a round trip test in which the results of each side are recorded. Power sources The slave device was powered by two AA batteries. The batteries were verified to be 2.9V or greater at the beginning of the test. Antenna orientation Height of devices off ground Description of test The master device was powered by the USB of a laptop computer. Both the wire and dipole antennas were kept in the vertical position. The slave device was placed on an outbuilding on the edge of the lake, 10' above water level. See Figure 12: device location. The master device was 9' above water level. The tester stood with his arm extended vertically and held the master device in his hand, always pointing toward the slave device. The master device was carried in a boat and connected to a laptop, which allowed viewing and recording the results. Increments of 0.25 mile were marked on a handheld GPS device the length of the lake (1.5 miles). Every 0.25 miles, the boat was stopped, and 500 packets were exchanged between the devices. The goal was to plot RSSI vs. distance in an open air, line of sight environment. See Figure 11: View across lake (1.5 miles) looking toward slave device. Page 10 of 16

SiFLEX02 Results Results were graphed for RSSI vs. distance on each antenna type and data rate. The results from each end of the test were graphed: what the slave saw from the master (MS) and what the master saw from the slave (SM). At 40kbs, the SiFLEX02 with the +2dBi dipole antenna had adequate link margin to the end of the lake at 1.5 miles. If the data were extrapolated, it is reasonable to assume the link margin would be adequate to at least 2.0 miles. At 40kbs, the SiFLEX02 with the wire antenna had adequate link margin to the end of the lake at 1.5 miles. If the data were extrapolated, it is reasonable to assume the link margin would be adequate to at least 1.75 miles. Figure 8: RSSI vs. distance for wire antenna and +2dBi dipole antenna Page 11 of 16

Sample Range Test Form for ProFLEX01 Testing Date of Test June 1, 2010 Name of person performing test Christopher Hofmeister, LS Research Location Star Lake, Vilas County, Wisconsin. See Figure 10: Map of test area. Radio tested ProFLEX01 (2.4 GHz DSSS) Antenna(s) tested F antenna, +1.2dBi dipole (LSR P/N 001 0001) Data rate 250kbs per 802.15.4 standard Firmware version 2.0 5 29 2010 Hardware version 2.0 Channel tested 14 Output power +20dBm ED results 0 See Figure 13: ProFLEX01 energy scan results. MAC level retries Off Application level retries On making this a round trip test in which the results of each side are recorded. Power sources The slave device was powered by two AA batteries. The batteries were verified to be 2.9V or greater at the beginning of the test. Antenna orientation Height of devices off ground Description of test The master device was powered by the USB of a laptop computer. The dipole antennas were kept in the vertical position. The F antennas were kept in the horizontal position (pcb plane parallel to the earth). The slave device was placed on an outbuilding on the edge of the lake, 10' above water level. See Figure 12: device location. The master device was 9' above water level. The tester stood with his arm extended vertically and held the master device in his hand, always pointing toward the slave device. The master device was carried in a boat and connected to a laptop, which allowed viewing and recording the results. Increments of 0.25 mile were marked on a handheld GPS device the length of the lake (1.5 miles). Every 0.25 miles, the boat was stopped, and 500 packets were exchanged between the devices. The goal was to plot RSSI vs. distance in an open air, line of sight environment. See Figure 11: View across lake (1.5 miles) looking toward slave device. Page 12 of 16

ProFLEX01 Results Figure 9 shows RSSI vs. distance on each antenna type. The graph shows results from each end of the test: What the slave saw from the master (MS) and what the master saw from the slave (SM). The ProFLEX01 unit showed adequate link margin to about 1 mile. Figure 9: RSSI vs. distance for F antenna and +2dBi dipole antenna Page 13 of 16

Appendix: Supporting Documentation Figure 10: Map of test area Figure 11: View across lake (1.5 miles) looking toward slave device Page 14 of 16

Figure 12: device location Page 15 of 16

Figure 13: ProFLEX01 energy scan results Figure 14: SiFLEX02 energy scan results Christopher Hofmeister, a senior software engineer at LS Research, has spent more than 15 years in the low power wireless industry. He has been involved in hardware and firmware design as well as testing of low power RF transmitters, receivers and transceivers for both engineering validation and production testing. Page 16 of 16