Inter-Device Synchronous Control Technology for IoT Systems Using Wireless LAN Modules TOHZAKA Yuji SAKAMOTO Takafumi DOI Yusuke Accompanying the expansion of the Internet of Things (IoT), interconnections between multiple devices and their cooperative operation via wireless LAN modules have become essential to achieve more efficient functioning of IoT systems. Timing synchronization through wireless communication has therefore become an issue of vital importance for various IoT systems. Toshiba has developed a wireless synchronous control system that can establish synchronization between multiple devices with a high degree of accuracy using existing wireless LAN modules. The timing signal for cooperative operations is provided by the wireless LAN module attached to each of the devices, in which all additional functions are implemented as software. Experiments on a system using wireless LAN modules that implement a set of algorithms to compensate for synchronization errors due to jitter caused by other tasks running on the CPU have confirmed that it achieves a synchronization accuracy within approximately ±2 µs even in the worst-case jitter scenario. It is expected that IoT systems incorporating this wireless synchronous control system will be able to be constructed at a cost equivalent to those employing a conventional wireless LAN system without any need for additional hardware. 1. Introduction IoT has evolved owing to the convergence of multiple technologies. Using communication technology, IoT is expected to coordinate the operations of various devices so that they work as a single unit. The potential applications of IoT are wide-ranging and may include electric power systems that effectively coordinate multiple inverters, digital signage systems that synchronously display video on multiple monitors, and industrial systems that consist of many robots working in a coordinated manner. To realize these applications, a synchronous control technology is necessary to operate multiple devices simultaneously or at a constant interval. Constituent devices should also provide a wireless communication capability to simplify their floor planning and installation, and high communication speeds are needed for them to share control information and content data. For wireless communication, a wireless LAN compliant with IEEE 82.11 ( )(1) is convenient. This paper describes the synchronous control technology developed by Toshiba for coordinating multiple devices on a wireless LAN with high accuracy. 2. Synchronous control technology using wireless LAN 2.1 Existing issues IoT needs low-cost synchronous control solutions with practically sufficient accuracy. The required accuracy depends on applications. For example, digital signage needs to perform synchronization with an accuracy of 8.3 ms (1/12 s). However, in wireless LAN and other wireless communication devices, automatic transmission retry and wait control frequently occurs on the media access control (MAC) layer ( * 1) in the event of transmission failures due to noise and network collisions. This makes the time taken to complete a transmission indeterminate. For example, the Network Time Protocol (NTP), a communication-based synchronous control technology, exhibits a synchronization error of a few tens of milliseconds. This error is intolerable for multi-monitor digital signage systems since it makes video frames out of sync across multiple monitors (2). An alternative solution is to use a wireless synchronization system with dedicated hardware, which can provide an accuracy of within 1 μs, However, such a system is expensive. ( * 1) Functional layer that provides control over the use of a communication medium 17
Timing signal Synchronization using the newly developed technique Wireless LAN AP Wireless LAN STA 1 STA M Synchronization via wireless LAN Slave device 1 Slave device M Figure 1. Overview of newly developed wireless synchronous control system. The slave devices share the same timing signal generated by the master device via wireless LAN. 2.2 Overview of the wireless synchronous control system Figure 1 shows the organization of the newly developed wireless synchronous control system (4). This system consists of one master device and multiple slave devices. The master device generates a periodic timing signal that serves as a reference for the synchronization of the entire system and transmits it to the slave devices wirelessly. The master device is connected to a wireless LAN access point (hereinafter AP ) and provides a timing signal to the AP. Each slave device is connected to a wireless LAN station (hereinafter STA ). The STAs provide a signal that is synchronized with the timing signal applied to the AP. All the slave devices, in turn, receive this signal from the corresponding STA to keep in sync with one another and the master device. Our synchronous control system utilizes the IEEE 82.11 ( ) Timing Synchronization Function (TSF) that comes standard with wireless LAN chips. The TSF-synchronized clock () counts up every 1 μs. The STAs adjust their own clocks using time information that is transmitted from the AP periodically (typically every 1 ms). Since the is used for communication control, it is synchronized at a high accuracy without being affected by retry and wait control on the MAC layer. In our synchronous control system, the AP uses the to estimate the parameters of the applied timing signal and informs the STAs of them (Figure 2). The STAs, in turn, regenerate a timing signal according to the to adjust the timing. This technology can be implemented as software for the wireless LAN chip in our wireless LAN modules. Therefore, practical wireless synchronous control systems that accept a synchronization reference signal from the external device can be created without adding new hardware. 2.3 Challenges and solutions for software implementation A wireless LAN chip consists of a CPU for running software, a nonvolatile memory for storing software, a pulse generator for generating a timing signal, general-purpose input/output (I/O) circuitry and various peripherals (Figure 3). The CPU prioritizes the execution of a communication program and gives lower priority levels to useradded software tasks including a synchronous control program. Consequently, the CPU never performs synchronous control while communication is in progress, causing random delays (jitters) in synchronous control. Jitters degrade synchronous control accuracy. To resolve this issue, it is necessary to reduce the effect of jitters in estimating the parameters of the timing signal applied to the AP from the master device and in regenerating timing signals with STAs. Generate a timing signal. Wireless LAN AP Wireless LAN STA 1. Estimate signal parameters based on the. s (synchronized as standard) 2. Transmit. 4. Regenerate a timing signal 3. Receive. 5. Adjust timing. based on the. Slave device Operate according to the timing signal. Figure 2. Algorithm of newly developed wireless synchronous control system. The AP estimates the parameters of a timing signal from the master device using a and passes them to STAs, which, in turn, regenerate timing signals in sync with the to adjust the timing. 18
Lower priority (AP) CPU Higher priority Nonvolatile memory Communication program Synchronous control program Jitters General-purpose I/O Pulse generator IEEE 82.11 ( ) low-layer processor (STA) Figure 3. Configuration of wireless LAN chip. Since programs running on the CPU get out of sync due to the effect of jitters, a workaround is necessary. 2.3.1 Estimation and transmission of timing signal parameters at AP Figure 4 shows a functional overview of the AP incorporating the newly developed technology. The AP estimates the period,, and the offset,, of the applied timing signal, based on the. Here, denotes a difference in time among signal reference points (rising edges of a pulse signal in Figure 4). As stated above, the timing at which the signal parameter estimation task starts is affected by jitters. A solution for this issue is as follows. As shown in Figure 4, the AP raises an interrupt to the CPU on the rising edge of the timing signal applied to a general-purpose I/O shown in Figure 3 and has the interrupt task read the. Since the rising edge of the timing signal, jitters have been giving random delays to the. To reduce this effect and obtain more accurately, the interrupt task calculates an average of the intervals between multiple values of the. Next, the interrupt task uses this estimated to calculate with multiple values of the. This way, the interrupt task can compare the effects of jitters on each of values. The effect of jitters can be alleviated by using the minimum value as the result of estimation. The AP periodically performs and estimation. When either or deviates by over the initial or a certain threshold, the AP informs STAs of these parameters via typical data communication. 2.3.2 Regeneration and adjustment of timing signal at STA On reception of the and parameters, each STA regenerates a timing signal as shown in Figure 2. More specifically, the STA sets the value into the pulse generator and begins to generate a periodic pulse signal at an appropriate time. Figure 5 shows an outline of this operation. An appropriate time means the time of the next and subsequent rising edges calculated from the parameters the STA received and the current value of the. Signal reference point Time that is a multiple Timing signal of the estimated (1) (2) (4) Wireless LAN AP Read the. Estimation of (1) (2) (2) (4) Average Estimate Jitters Estimation of (1) (2) Min (4) Estimate Figure 4. Estimation of timing signal parameters at AP. γ is the average of the intervals of multiple values. is used to reduce the effect of jitters by selecting the minimum value from values. 19
Wireless LAN STA Receive parameters. The timing signal gradually drifts from the. Slave device Pulse signal Regenerate Adjust Generated timing signal Input signal to the slave device CPU The STA adjusts the timing signal only when jitters are small. Busy Idle Large delay Small delay Correct rise time Read by a task Pulse signal Skip the trigger for the pulse generator. Trigger the pulse generator. Figure 5. Regeneration and adjustment of timing signal at STA. The STA regenerates a timing signal based on the by using parameters received from the AP. The effect of jitters is reduced by determining whether to start tasks according to the magnitude of jitters. Since the of an STA is synchronous with that of the AP, the timing signal generated by the STA s pulse generator gradually shifts over time from the beginning of pulse output, relative to the of the STA. Therefore, the timing signal must be adjusted periodically. As shown in Figure 5, the CPU gives a low priority to the timing adjustment task as is the case with the parameter estimation task. Thus, the timing adjustment task is also affected by CPU jitters. To reduce this effect, the timing adjustment task reads the and calculates a difference in time between correct and actual rising edges. The STA terminates and restarts the output of the pulse generator to adjust the timing of the pulse signal only when this difference is smaller than a predefined threshold (i.e., when the effect of jitters is small). If the effect of jitters is large, the STA does not adjust the pulse generator. The STA invokes the timing adjustment task periodically and adjusts the timing if the effect of jitters drops below the threshold. 3. Evaluation By way of evaluation, we measured the performance of the newly developed wireless synchronous control system. Figure 6 shows the setup of synchronization accuracy measurement. The setup consisted of one IEEE 82.11n ( ) -compliant AP and three STAs (1 to 3). Instead of a master device, a signal generator was used to apply a pulse signal with a period of 2 ms (which is equivalent to a 5 Hz timing signal) to the AP. The outputs from the signal generator and the three STAs were observed with an oscilloscope to measure synchronization errors, i.e., the differences in time between their rising edges. Because the new system is implemented as software as described above, the communication workload on the Signal generator (master device) AP Timing signal Oscilloscope STA3 (with minimum communication load) TCP data transmission STA2 (with maximum communication load; transmitter) AP is in a line of sight to STA, and its distance is less than 1 m. The 2.4 GHz band is used in the office environment. STA1 (with maximum communication load; receiver) Figure 6. Synchronization accuracy measurement system. Measurements were taken, placing a communication load that is supposed to be the maximum in the actual use environment on some slave devices in order to emulate a situation in which the synchronization accuracy will become lowest. CPU has impacts on synchronization accuracy. In order to compare such impacts and measure synchronization errors, Transmission Control Protocol (TCP) data were sent from STA2 to STA1 in transmission speed measurement mode. The purpose of this was to artificially emulate a situation in which a system has both wireless devices with the maximum communication load (STA1 and STA2) and a wireless device with the minimum communication load (STA3). 2
Frequency (1 3 times) 8 6 4 2 STA3 77 μs on average STA2 1 μs on average STA1 86 μs on average 21 μs max. 2 15 1-5 5 1 15 2 25 Synchronization error relative to the output of the master device (input to the AP) (μs) Figure 7. Results of synchronization accuracy measurements. As a result of evaluation, it was confirmed that the maximum synchronization error is approx. 2 μs. Figure 7 shows a histogram of the measurement results synchronization errors of the STA outputs relative to the output of the master device (i.e., the input to the AP). STA3 with a low communication load has negative synchronization errors because the timing signal regeneration and adjustment tasks were started earlier in all the STAs (i.e., the task start times were corrected in the negative direction). The amount of correction was determined so as to minimize the maximum synchronization error, based on the results of prior measurement taken with no correction. The evaluation results show that the maximum synchronization error between the timing signal applied to the AP and the timing signals regenerated by the STAs is approximately 2 μs. 4. Conclusion In this paper, we have presented a wireless synchronous control technology for coordinating the operations of many IoT devices. By implementing the function developed for our wireless LAN module as software and evaluating its operation, the effectiveness of this technology was confirmed. The wireless synchronous control technology has various applications. For example, a frame rate of 12 frames per second (fps) may be assumed for digital signage systems, four times higher than the currently most widely used frame rate of 3 fps. At 12 fps, a synchronization accuracy of 8.3 ms is considered to be a design target. In view of this assumption, the newly developed technology can generate a timing signal for synchronous control with a sufficient accuracy (approximately 2 μs). Other expected applications include the AC synchronization of electric power inverters running in parallel, and coordination control of industrial robots. We will continue with our research to further improve the accuracy and stability and increase the system size with the aim of putting the technology into practical use. References (1) IEEE Std 82.11 TM : 212. IEEE Standard for Information technology Telecommunications and information exchange between systems Local and metropolitan area networks Specific requirements Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications. (2) Tomizawa. T. et al. 214. A Study on Clock Synchronization Method in WLAN System (1) (in Japanese). The Institute of Electronics, Information and Communication Engineers (IEICE) General Conference 214. Niigata, Japan, March 214. Amagai. A. 213. Sub-microsecond synchronization of data acquisition systems in LAN/Wi-Fi mixed network (in Japanese). IEICE Technical Report. 112 (46): 9 14. (4) Tohzaka, Y. et al. 214. A Practical and Cost-Effective Cyclic Pulse Synchronization System using IEEE 82.11 Wireless LAN. IEEE 25th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC). Washington DC, USA, September 214, IEEE: 261 265. IEEE and 82 are trademarks or registered trademarks of the Institute of Electrical and Electronics Engineers, Inc. in the U.S. and other countries. TOHZAKA Yuji Network System Laboratory, Corporate Research & Development Center. He is engaged in the research and development of wireless resource control and wireless networking applications. SAKAMOTO Takafumi Senior Research Scientist. Network System Laboratory, Corporate Research & Development Center. He is engaged in the research and development of wireless communication systems. DOI Yusuke, Ph.D. Senior Research Scientist. Network System Laboratory, Corporate Research & Development Center. He is engaged in the research and development of distributed systems and network technologies. Note: This is basically a full English translation of TOSHIBA REVIEW Vol. 7, No. 1, 215, published by Toshiba Corporation. 21