Welcome to the Symposium, I am [name and title] from Keysight technologies.

Transcription

1 The evolution of tactical and public safety radio technologies presents new challenges in radio testing. For example; combination of new and legacy standards in a radio requires a wide range of test configurations, measurement capabilities, and modulation-analysis tools. Also, crucial business drivers demand simplified system integration, easy migration of test IP, increased throughput, and lower cost of tests. This paper will review these challenges and show how to unlock the future of radio test through hardware and software configurations enabling the excellence you expect from your radios Hello Everyone, Welcome to the Symposium, I am [name and title] from Keysight technologies. For today s presentation, we will discuss several case studies around radio test measurement challenges to highlight some of the key pain points that most R&D and Production engineers face when they design and manufacture radios. I will share the main takeaways from these case studies, and then demonstrate what Keysight s radio testing solutions can do to address these challenges. 1

2 Here is the Agenda for today s presentation: We will start with a high level introduction to Radio test applications We will then summarize radio test measurement challenges that often arise in the R&D and production phase Then we will share with you several case studies that we conducted around the topics of test IP migration from R&D to manufacturing, Measurement speed and how measurement uncertainty can have a significant impact on your overall cost of test. Lastly we will wrap up with a summary and Q&A session. 2

3 As I mentioned earlier, the purpose of this presentation is mainly to focus on test challenges. We will not cover the various radio formats and technologies. However, it is important to start with a brief introduction of the different kinds of radios and how they are fragmented and deployed between three main subsegments: MilCom (Military Communication), Avionics and LMR which is includes Public Safety and commercial radio applications. Now from a design point of view, whether it is for MilCom, public safety or avionics, we find a common radio architecture and functionality. This ensures secure voice and data communication. From a test point of view, there are several attributes that are specific to each radio, such as the modulation format (Analog or digital), the frequency band and the encryption algorithm. 3

4 Let s take a look at how these different radios are designed, manufactured and tested. The building blocks shown here are separate instruments in a traditional test configuration. The dotted block in this slide represents the radio test main core, which is a combination of SA, SS, BER tester and audio tester. In some cases, some functionality such as over the air testing may require a one-box-tester as shown in the upper right. These components can also be delivered as modules or completely addressed in a one-box-tester Typically in a R&D and production whether they manufacture Radios for public safety or tactical military uses engineers combine all of this equipment into a bench or rack system with test executive automation software to test these radios. Even though the diagram presented here fits many radios test needs, the various types of radios in the market as well as the technology dynamic, creates multiple challenges. That s why, the purpose of this presentation is to review the most common challenges and explore solutions that address them. 4

5 Let s start with Radio Test Challenges. 5

6 Radio Test Challenges mainly fall in two areas: Business Challenges and Technology Dynamics With the tremendous evolution in Radio Technologies whether it is military communication, public safety or commercial radios the business conditions continue to put pressure on driving down the cost of test for radio manufacturers. At the same time keeping up with technology demands, is driving the radio standards to require wider bandwidth and higher frequencies. All of these requirements have a direct impact on the creation of new digital radio technologies, which inevitably increases the demand for test equipment. Most of the R&D and production engineers that we have spoke to, expressed the needs of an optimal engineering solution that could reduce both, the cost of test and the time spent on development, while still ensuring a high quality product. 6

7 Now, let s dive a little deeper into the business challenges, specifically around the cost of test. Cost of test can be significantly impacted by test development time. It traditionally takes up to 4 engineer-months to transfer radio test IP from R&D to Manufacturing. Radio manufacturers typically use different test equipment for each phase of their product life cycle. Therefore, this variation in test equipment, eventually leads to differences in measurement science, HW architecture and API for test automation; resulting in long test development times and delayed revenue. 7

8 The second impact on cost of test, is measurement correlation and inconsistent measurement results that typically occur when transitioning from R&D to production. Measurement correlation issues are also explained by the different instruments that are typically used for testing in these two stages (R&D and production). To overcome these issues, R&D and production engineers often need to apply an offset value to make the measurements correlate since they cannot be resolved. Obviously, this solution is not ideal because it could lead to production failures, no fault found in the field, driving higher warranty cost and reducing profitability. 8

9 Therefore, most of our customer end up in the status quo in which they are trying to find ways to balance test development time and test optimization. They often face a tradeoff: First option is the traditional method, where radio manufacturer choose one unique instruments for each phase of the lifecycle. However, the cost of this approach is increased by long test development time and measurement correlation issue. For the second option, you might be tempted to use the same instrument for the entire lifecycle. However, with this option you will sacrifice long term revenue. So, unfortunately, both option comes at a high cost. That said, we re are going to explore this tradeoff in this presentation, and demonstrate what Keysight can offer to optimize both test development time and faster measurement while at the same time maintaining good measurement consistency to ensure a high quality product. 9

10 To demonstrate what we just described, let s look at some practical case studies 10

11 For all of our case studies, we will be performing an APCO P25 C4FM digital modulation Quality measurement. APCO P25 is an LMR standard that is mainly used by police and fire departments. The table on the left summarizes the ideal P25 C4FM digital symbol representation. Each symbol is associated with an ideal frequency deviation. The overall ideal frequency deviation is 1.8 KHz. In the second display, we are presenting the measurement of a P25 C4FM transmitter Modulation Quality. This measurement has been made with the N9064A X-Series Application for digital Demodulation measurement. In this display, you can see everything you need to make a qualitative measurement of the transmitter modulator performance. In the upper left, you can see the measured FSK constellation. From this plot, you can see the demodulated symbol for the P25 C4FM and that this signal is using the 4 FSK modulation format. On the upper right, you can see the results of the FSK error versus symbol. If you are not familiar with FSK error, it stands for Frequency Shift Keying error. The FSK error provides a convenient signal number metric and insight into the overall modulator balance and quality. 11

12 On the lower right, you can see modulation quality metric where we display the FSK error, magnitude error, carrier offset, frequency deviation and the demodulated symbols. On the lower left, you can see the spectrum display which occupy 12.5 KHz BW Overall, this display offers you great insight into the transmitter modulation quality for an APCO P25 C4FM radio. Note to the speaker: Often the digital modulation performance is not tested directly, and tune and test is performed using Analog FM 11

13 Now let s have a closer look at how this modulation quality measurement might be automated in R&D to measure the frequency Deviation of an APCO P25 radio. In the upper left picture is an example of a typical R&D setup to make this measurement. The radio will transmit the signal to the analyzer, which is controlled by the PC via LAN. To automate this test, we used Keysight s command expert, which is a free Software tool that Keysight developed to help improve the productivity of SW development and system integration. In addition, it allows you to build and execute, in an automated fashion, instrument command sequences. So in this example, we are basically initiating a connection to the LXI Signal Analyzer via LAN, then sending SCPI commands to configure the analyzer and measure the modulation quality performance for the transmitter. Now let s take a closer look at the execution of this sequence. 12

14 In this test, we measured a Frequency deviation = 1.806KHz which is 6 Hz of error relative to the ideal frequency deviation. The FSK error for this UUT was approximately 0.6%. This is an extremely good result for this radio transmitter 13

15 Now, for our case studies, we will be migrating this test sequence from R&D to production using two different scenarios. 1- In the first scenario, we will migrate the Modulation Quality test sequence that we created in the previous slide from a Keysight solution to a Non Keysight solution. This will demonstrate the impact on test development time/ and measurement correlation when you migrate to different HW architecture, measurement science and API. 2- In the second scenario, we will migrate the test IP from Keysight LXI platform to a Keysight PXI platform. However, in this configuration we will have different hardware architecture but same measurement science and API. 14

16 Here are the results of the first scenario. To no surprise, with the non Keysight solution, in the production phase, Command expert failed to execute the test sequences that had been created in the R&D phase due to the SCPI commands of the Non Keysight one box tester that are not compatible with the Keysight solution. So, the question is, how does a production engineer solve this issue? 15

17 To work around the SCPI compatibility issues, Production engineers typically end up creating new test IP for production without leveraging or re-using the test IP that has already been written in the R&D phase. This process leads to an increase in test development time. Now, let s take a closer look at the measurement results for this specific test and see how they correlate. 16

18 Shown here are the results of the Modulation Quality measurements in both phases: R&D and production. For this measurement comparison, the darts (shown in red and purple) represent the clear differences between the two measurements. The dark green circle represents the ideal target to measure which is 1.8 KHz. The red circle represent the failure region. Luckily, for this UUT the Frequency Deviation measurement test, passed in both configurations. But, as you can see, there is a big difference between the two measurements. In this case, it s really difficult to debug this difference, and provide a decisive answer on why the measurement results vary as there are three variables (different HW architecture, measurement science, and unknown measurement uncertainty and calibration) that could contribute to these differences. However, it is important to highlight that the combination of higher systematic error and large measurement uncertainty in your instruments can result in incorrect test results and higher warranty cost. Wouldn t it be nice if somehow we could reduce both the test development time and reduce the systematic errors at once?? 17

19 That s what we are going to explore. In this scenario, we will be migrating the same test IP from Keysight LXI architecture to Keysight PXI architecture. The PXI Signal Analyzer has been mainly designed to reduce the footprint, increase measurement speed while maintaining measurement accuracy. The only difference here is that we will be applying the same measurement science on both architectures 18

20 Shown here, are the results of the second scenario where we are migrating from Keysight LXI SA to Keysight PXI SA. As you can see, the transition from R&D to production is much easier and more cost effective than the first scenario. With only one a minor change, the production engineer was able to re-use the same test IP that had been developed in the R&D phase. As shown in row 1 of the two sequences, this change resides in how production engineer remote control each instrument via the IP address, In this scenario, having the same measurement science across these different platforms and different phases resulted an extreme smooth migration of test IP from R&D to production. But again, let s take a closer look at the measurement results. 19

21 Here are the measurement results in R&D and production. The left screen shot represents the modulation quality measurement of the transmitter in the R&D phase. And the right one is the same measurement in the production phase. As you can see the results are very consistent across these different platforms. The difference between the two FSK error measurement is extremely low, it is approximately 0.02%. And 0.4 Hz difference in the frequency deviation. 20

22 As a recap, test equipment that uses the same measurement science and have a quantitative measurement uncertainty enabled us to make more accurate, repeatable and consistent measurements results. This ultimately reduces the impact on your overall productivity, and operational costs. 21

23 The next case study is on measurement speed. 22

24 In this case study, we will be exploring the PXI performance versus LXI performance in a production line environment. The first benchmark setup (on the left) is made to compare the measurement speed between Keysight PXI signal analyzer versus a Non Keysight LXI OBT. And the second setup (on the right) is made to compare the measurement speed performance between Keysight PXI Analyzer versus Keysight LXI signal Analyzer. 23

25 For both benchmark test setups, we will be performing a speed analysis to: Measure how long it will take the instruments to load the APCO P25 measurement preset and measure the modulation quality of the transmitter. To make sure the same conditions are applied in both benchmarks, we developed a speed analysis software to control the instruments. This software application mainly uses SCPI commands over LAN to communicate and track the test speed for each instrument. 24

26 Here are the results of the cycle time and measurement speed performance for the Keysight PXI Analyzer relative to the non Keysight OBT. For the Non Keysight OBT, it takes 50 seconds to load the APCO P25 application. It is important to mention that during this time the end user will not be able to make any measurements as the OBT is busy loading the instrument state for P25. However, the Keysight PXI Analyzer was 50 times faster as it only takes 1 second to load the application, and less than 400 msec to execute a modulation quality measurement. Performing this accurate measurement at this speed present a great opportunity in production line environement. 25

27 Now let s take a look at the performance between Keysight LXI vs Keysight PXI analyzers. The cycle time and measurement speed performance for the Keysight PXI and LXI Signal analyzer are summarized here. Also, in this configuration the PXI signal analyzer is still slightly faster than the Keysight LXI signal analyzer. It is important to note that regardless of whether you are in a R&D or production phase, the measurement speed may not be the only criteria for your measurement needs. However, these different HW architectures offer you additional options to help you optimize the potential tradeoffs between measurement speed, footprint, performance, precision and accuracy. 26

28 So the question you may be asking is: why is the PXI platform is faster? Shown here is a block diagram of the M9018A PXI chassis that shows the PCIe interconnections. This diagram shows the chassis with multiple x8 links routed to PCIe switches within the chassis. The M9018A chassis is unique since its PCIe switches can be reconfigured to support either system slot 2 link or 4 link configurations. This block diagram shows the 2 link 4GB/sec each. Which enable higher data transfer between the embedded controller and the PXI modules that are shared through the PXIe backplane. Also, you ll notice the PCIe connections to the individual slots support both x4 and x8 PCIe links, depending on the slot. The slots with the x8 links will provide the highest data BW. Finally, see the FPGA SMBus controller, which connects as a PCIe endpoint. The SMBus provides utility communications within the chassis itself, and it is used for reporting fan speed, voltages and temperatures. 27

29 To take advantage of the PXI architecture and enable faster measurements, we added a layer to our software architecture called the Resource Manager. The Resource Manager enables the user to quickly switch between direct driver access to the PXI VSA hardware and the modular X-apps analysis software. The benefit is that your software or test executive, can directly access the PXI VSA drivers to get the raw data, in a very fast way, and then switch seamlessly to the standards-based x-apps software application. That is what I am going to demonstrate in the next slide 28

30 Here is an example of a transmitter power measurement. In this example, we re measuring the total transmit channel power using both the LXI and PXI Signal analyzer. The display on the left represents the measurement made by the LXI Signal analyzer. This measurement was made using the X-series application. On the right side, is a C# example program to perform the power measurement using the direct IVI driver API. In this example, we are calling the ReadPower API that is implemented in the IVI driver. This function communicate directly with the HW to measure the power in each channel. Let s see the result of the execution of this code. 29

31 Here are the measurement results. Again, The measurement results are extremely consistent across these different platforms. Even though we used different API, the difference between the two measurements is extremely low, it is approximately about 0.04 db. However by taking advantage of the Resource manager, the PXI signal analyzer was able to do the same measurement with approximately 8 times faster than the LXI signal analyzer. and this also could be used as flexible option to improve your throughput. 30

32 Now that we covered, the test IP migration and measurement speed, let s review how measurement uncertainty can have a big impact on your overall cost of test. 31

33 First, what is measurement uncertainty and why does it matter? Shown here is two-dimension representation to describe the definition of measurement uncertainty The X axis represent the measurement precision of your instruments. In other terms, how repeatable are your measurements The Y axis represent how accurate are your measurements. Or how close you are to the true value Measurement Uncertainty is the combination of precision and accuracy of your test system. Also, it could be defined as the dispersion of value around a measured parameter. For example, if you are trying to measure 0 dbm power, and if the measurement uncertainty of your instrument is +/- 0.2 db, the actual power being measured could be anywhere between -0.2 dbm and +0.2 dbm That s why, knowing and quantifying the Measurement Uncertainty of your test equipment and your overall test system can have big impact on your yield, productivities and the test results of your products. That s what we will demonstrate in this case study 32

34 Let s look at an example of the impact of measurement uncertainty. What you see here is a bell curve of the pass/fail metrics of a product being tested in R&D or manufacturing The tolerance for acceptable pass range for the product being tested is +/- 2 db for a specific measurement. So any measurement within -5 dbm +/- 2 db range passes the limits. Now the measurement uncertainty for the test equipment being used for testing a specific parameter is shown to be +/- 1 db. In order to ensure that the products that pass the test are truly within the tolerance, the test limits must be reduced by the amount of the measurement uncertainty; that is, 1 db on either side. The section marked as yield are the products that have been measured within acceptable performance limit. However, for the products whose measurement results fall in an area of measurement uncertainty, there is no way to verify and be confident of their performance, which sometimes leads to false fail products. Therefore, this negatively impacts your yield rate resulting in reduced productivity and increase the amount of false fail product. 33

35 To elaborate this point more, this is a two-dimensional representation of the measurement error distribution versus the possible true values. Areas with lighter color are more densely populated, indicate the Passed results which are measured (or observed) in specification. The areas marked as Failed indicate those results where the Measured value and true value are out of specification. However, whenever a binary Pass/Fail decision is made in the presence of measurement uncertainty there are two additional possible states. False Passes are those results which are measured in-specification, but the true result is out of specification. False Fails are those results which measured out of specification, but the true value is actually in-specification. Typically, this causes the device to be repaired unnecessarily or even thrown away, incurring additional costs in either case. What you would do to decrease the number of False Passes? To reduce the amount of the product that are in the False Pass region, you might be tempted to apply guard banding 34

36 Good news! - The number of false passes decreases Bad news! a lower incidence of False Passes comes at a price - See how the number of False Fails has noticeably increased 34

37 We just said that whenever you employ Guard-banding to reduce false failures, it comes at a cost of increasing False Failures! Wouldn t it be nice if somehow we could reduce both at once? Turns out you can, by reducing the Measurement uncertainty in your test system. See how both False Passes and False Failures decrease when the measurement uncertainty is decreased. Now, I think you may ask how do I reduce measurement uncertainty? Well, in our Signal Analyzer instruments, we implemented a self calibration routine to perform an automatic alignments that can run between measurement acquisitions. The instrument s software determines when alignments are needed to be performed to maintain warranted operation and reduce the risk of increasing your measurement uncertainty. Also, to ensure the integrity of your measurements, despite unavoidable test equipment drift, calibrate and quantify the measurement uncertainty of your test equipment periodically and have the peace of mind/confidence that your equipment continues to operate to its factory-shipped specifications giving you 35

38 accurate, reliable, and repeatable measurements. 35

39 As a recap, quantifying the Measurement Uncertainty and adding it to your test limits further minimizes the risk of passing devices that are outside of their specifications, while on the other hand, it increases the number of good devices that fail the test. This provide you confidence that your devices are performing to their specifications and reducing impact on your throughput, productivity, and operational costs. 36

40 Now, let s summarize the key takeaways from today s presentation. 37

41 We discussed in this presentation how your cost of test could be negatively impacted by long test development time and the measurement correlation issue that occur during designing and manufacturing new radios. We also demonstrated what Keysight can offer, to help you deliver the excellence you expect from your radio. With un-matched measurement performance across the lifecycle for your radio test, starting from R&D to design validation and production, you could choose the best instrument that meet your measurement needs while at the same time maintaining a great measurement consistency across the lifecycle, and minimizing your test IP integration efforts. This would positively contribute to reduced operation costs, reduced test development time, reduced test equipment inventory, and increased test speed. Also, we discussed the importance of measurement uncertainty of your equipment and how it impacts your yield. So, keeping your test equipment calibrated ensures integrity of your measurements and gives you increased confidence in your test results. While you are free to choose a calibration vendor or different supplier of your choice for your test equipment, make sure to ask your supplier about the uncertainty and measurement consistency of their products. Make sure you're 38

42 getting what you need from your test equipment to achieve your business goals. 38

43 If you d like to learn more about Keysight s Radio Test Solution, you can find a number of tools that are available on our website to help you understand and configure your solution. 39

44 40