The past year has seen a flurry of activity related to bringing the next generation of broadband wireless technology, commonly referred to as Mobile WiMAX, to the market. Based on the IEEE 802.16e standard for local and metropolitan area networks, released in 2005, Mobile WiMAX promises wireless data connections of 10 to 15 megabits per second in a typical cellular radius, far exceeding the capabilities of today’s 2.5-3G wireless networks. The WiMAX Forum™ is an industry organization that was formed to promote the development of devices meeting the 802.16 standards and ensure that all components of the wireless network operate together correctly. Only devices passing a range of certification tests for conformance, interoperability and performance will receive the “WiMAX Forum Certified” designation. With hundreds of companies involved in all aspects of development for products, services and infrastructure, including industry giants like Sprint and Intel, there is considerable emphasis on ensuring that WiMAX technology provides an exceptional user experience from day one. Meeting this goal requires not only that WiMAX devices follow the 802.16e standard and work seamlessly with each other, but also that they provide a minimum level of performance across the entire network.

While basic “can you hear me now” type testing of a device on a wireless network may provide some level of confidence in its performance, it’s not a very scientific approach to determining the radiated RF performance of a wireless device. In recent years, segments of the wireless industry have adopted laboratory test techniques for determining the over-the-air transmit and receive characteristics of wireless devices in simulated usage cases. In the United States, wireless carriers have come to rely on this performance data for evaluation of mobile phones prior to allowing them on their networks. In an industry where a 2 dB loss in radiated performance can result in the need for 25 percent more base stations (based on 40 dB per decade path loss as described in Lee and Hata Models), the potential cost savings of performing this type of laboratory testing is significant. ETS-Lindgren recognized that the WiMAX industry would require a similar level of confidence in their products, and has spent the past year and a half developing the capability for performing these tests on 802.16e devices, as well as providing the principal contributions to the WiMAX Forum Radiated Performance Tests (RPT) for Subscriber and Mobile Stations.1 In August of 2007 the company was the first to demonstrate fully automated total radiated power (TRP) testing of a WiMAX device, followed shortly thereafter by fully automated testing of total isotropic sensitivity (TIS). On January 10, 2008, ETS-Lindgren announced the installation of the first WiMAX RPT lab at AT4 wireless in Herndon, VA.2

Introduction to Radiated Performance Testing

The basic concepts of radiated performance testing (RPT) are relatively simple. The goal is to determine the “edge-of-link” performance of the device to determine the limits under which a wireless connection is likely to be maintained. Imagine carrying on a voice conversation with someone as you get further and further apart. Eventually you would be yelling at them while straining to hear what they were yelling back. The conversation would break down when one member of the conversation could not hear the other well enough to understand what was said. The link could break down either because the speaker could not yell loud enough or because the listener was hard of hearing. Added to that, any noise around the listener, including their own attempts to talk back to the speaker, would reduce the chances of their hearing what was said.

The same basic idea applies to a wireless link. Once a mobile device is unable to communicate with a base station, whether because of insufficient transmitter or receiver performance, the link breaks down. In this case, two standard metrics are typically used to represent the performance of a wireless device. The total radiated power (TRP) is the “talk” metric, and represents the average transmit performance of the device in any random direction. Conversely, the total isotropic sensitivity (TIS) is the “listen” metric and represents the average receiver sensitivity from any given direction. Similar to hearing or speech, the actual performance in any given direction varies, and the methodologies used to determine these quantities are capable of evaluating that directional behavior in addition to determining the overall average performance. Instead of measuring the combined behavior of the transmitter and receiver out in the field, each quantity can be evaluated separately in a laboratory environment.

While the specifics of TRP and TIS measurement methods are covered in detail elsewhere,3–5 the methods used today evolved from passive antenna pattern measurement techniques, where the directional performance (gain) of an antenna is evaluated from all directions around the antenna. To do so requires an isolated free-space environment, typically produced by an RF absorber-lined shielded room known as a fully anechoic chamber. This ensures that only energy radiated by the device or transmitted directly to the device is measured. Within the chamber, the device to be tested is moved relative to a measurement antenna, either by moving the device, the measurement antenna, or both, in two orthogonal axes in order to cover the surface of a sphere around the device (think of latitude and longitude lines on the surface of the earth). A dual-polarized measurement antenna is typically used to determine the resultant field vector magnitude at each point on the spherical surface no matter what its orientation. To determine TRP, the device is made to broadcast at full power and the radiated power is measured in each direction. For TIS, the signal level transmitted to the device is reduced until a specified number of errors are introduced into the data stream. The resultant power level is recorded for each direction around the device.

With active wireless devices, the goal of TRP and TIS testing is to evaluate the radiated performance of the radio through the integrated antenna(s) including the impact of objects typically found in the vicinity of the radio during use. This includes the platform in which the radio is installed (such as a cell phone or notebook PC) and possibly the operator (head, hands, etc.) or tabletop in proximity to the device. All of these can affect the transmit or receive radiation pattern of the device and thus the resulting radiated performance. In addition, electronic noise sources such as CPU clocks, displays, etc. can couple through the antenna causing desensitization of the receiver that cannot be determined through conducted tests.6

Testing WiMAX Mobile Stations

Developing the capability for radiated performance testing of any emerging technology is no mean feat. Ideally the devices to be tested would be in a configuration identical to that which the end user would obtain for use on a network. Given that WiMAX technology is still in development it’s not possible to go down to the local electronics store to purchase a device to test. Obtaining prototype devices requires considerable cooperation between major developers in the industry. Even then, devices are rarely available in a configuration suitable for RPT. Most R&D testing, including that for protocol and radio conformance testing within the WiMAX Forum, is done in a conducted environment. Signals are routed directly to and from the radio without ever going through an antenna. In addition, mechanisms typically exist for getting information out of the WiMAX radio, usually through a digital cable connection.

In determining radiated performance, any cables attached to the device can significantly affect the radiation pattern of the device. Thus, it’s critical that all of the required performance information be obtainable over the wireless interface. Currently this is complicated further due to the lack of a standardized wireless test interface in the 802.16e protocol.

Once a suitable WiMAX mobile device is available, the test development effort is far from over. In order to operate the device as it would on a real network, a base station emulator (BSE) is required to provide the other end of the link and produce the expected network behavior, as well as to perform various test functions. Of course for a new technology it takes time for this type of test equipment to become available to the market. Although several test equipment vendors are developing BSE solutions, as of this writing the Agilent E6651A Mobile WiMAX Test Set is the only BSE that has been successfully used to perform RPT tests. The early availability of this unit and Agilent’s willingness to support the RPT development effort has been invaluable in the progress made to date.

Commercially available base station emulators are typically designed for conducted testing and thus are not expected to generate or respond to the range of signal levels usually seen on a real wireless network. A typical radio conformance test is intended to verify that the subscriber or mobile station device under test (DUT) meets a very specific set of requirements. The associated cable losses and expected signal levels are thus well known, such that the BSE does not have to adapt to a wide range of signals. However, once an antenna is attached to the radio, this is no longer the case. The signal level in any random direction can vary over a wide dynamic range, depending on whether the measurement antenna is directed towards a peak or a null in the radiation pattern. In addition, the path losses for over-the-air radiated propagation are considerably larger than that of the typical RF cable used in conducted tests. In order to overcome these limitations, external signal conditioning circuitry is typically required to extend the dynamic range of the BSE transmitter and receiver in order to perform radiated testing. This usually consists of a number of amplifiers, attenuators and high-speed switches. The TDD nature of the WiMAX signal adds a level of complexity to the signal conditioning, since the signal has to be separated into its uplink and downlink components and amplified separately, then recombined into a bi-directional signal, all the while isolating the input of the opposite amplifier from the high power output of a given link direction. Doing so requires a control signal from the BSE to determine the direction of the radiated signal.

Often, a separate broadband power measurement device, such as a spectrum analyzer or IQ analyzer, is required to measure uplink power from the DUT. Since these devices can have considerable dynamic range, this is usually simpler than trying to measure using an integrated communications tester that combines the BSE with other measurement functions. The signal conditioning required to maintain a connection to the BSE may often interfere with the accuracy of an integrated measurement, and a broadband power measurement is typically faster and more forgiving of low signal levels than is a VSA where the signal must first be detected and demodulated in order to determine the power of a packet.

Of course all of these signals must be routed between the anechoic chamber and the test system, requiring an RF switch matrix in order to automate the system. A positioning system and controller as well as a controlling PC running test automation software is also required. Figure 1 shows a system diagram for a typical WiMAX RPT test system.

Even given all of the pieces listed here, progress is often hampered by interoperability issues between the DUT and the BSE. The WiMAX requirements have been changing rapidly and developers and manufacturers are in a race to keep up with a moving target. Thus, it can be difficult to find compatible revisions to be able to test.

Measuring TRP of a WiMAX Mobile Station

To determine TRP, the wireless mobile station must be connected to the BSE and made to transmit uplink packets at full power to simulate usage at the edge of the link. This is not always as easy as it sounds since power control requirements are still in a state of flux and implementations vary. Some devices have manual power control overrides, but that always introduces the risk of a test mode that does not mirror real world behavior.

The 802.16e standard does provide an uplink padding mechanism that allows the BSE to tell the device to completely fill the uplink data channel with random data. However, in many device implementations, prolonged periods of uplink padding will result in a loss of connection. Presumably the device is unable to perform other network functions while sending this “useless” data and eventually gives up the connection.

Once the DUT is made to generate suitable uplink traffic, TRP is determined by making broadband power measurements of the uplink data bursts at each orientation and polarization around the DUT. This is typically done using an IQ analyzer in time domain mode or a spectrum analyzer with a wide resolution bandwidth. Figure 2 illustrates a typical WiMAX frame measured using an RMS detector. It’s important to be able to detect valid uplink bursts and only measure the average RMS power of the uplink data burst, as shown in Figure 3.

Figure 4 illustrates some of the first ever fully automated WiMAX TRP data measured on a PCMCIA card device installed in the middle of the right side of a notebook PC. The pattern clearly shows the shadowing effect caused by the display and base of the notebook, as well as the ripple caused by the constructive and destructive interference between the main lobe of the antenna and its image reflected from the display. In this configuration, the device generated a TRP of 26.3 dBm, while in another smaller laptop with the PCMCIA slot on the opposite side, the TRP was 24.1 dBm, illustrating just how much impact the platform can have on radiated performance. For research purposes, this data was sampled every 5°, which is considerably more than is necessary to obtain a valid TRP result. A theta dependent phi optimization was used to reduce the number of points measured near theta = 0° and 180° where the surface points are closer together. The test time at this sample density was on the order of 40 to 45 minutes per frequency, where an equivalent TRP result could be obtained in about 10 minutes with a coarser sampling grid.

Measuring TIS of a WiMAX Mobile Station

Measuring TIS involves repeated searches for sensitivity by lowering the transmit power of the BSE until a target packet error rate (PER) is reached. This simulates the case where the user is at the limits of the connection. Since the shape of the pattern varies as a function of orientation and frequency, the power level required to cause the target PER will vary and thus a search algorithm must be used to find the sensitivity level at each data point. Because the sensitivity measurement is referenced to the output power of the BSE, it requires that the output power of the BSE be calibrated for traceable measurements, and have enough dynamic range to cover the best sensitivity levels while still being able to maintain the connection through the deepest pattern nulls. Given the amount of time involved, it’s impractical to perform the long PER measurements typically used to verify that a radio exceeds the sensitivity levels specified in 802.16. A bit error rate (BER) measurement is also not possible since there is no PHY layer loopback mode defined for WiMAX radios. In order to minimize the time required to search for sensitivity, statistical techniques are used to quickly determine a pass-fail criteria against a large target PER (10%), and a maximum packet count of 1000 frames is sufficient to have reasonable confidence in the results. The larger relative PER helps to narrow the range of power where the target value might be found, due to the steepness of the PER vs. power curve; since the sensitivity search results are integrated across the surface of a sphere to determine TIS, the effective number of packets evaluated for determining overall sensitivity is considerably larger than that for one search.

When performing a sensitivity measurement, it’s important that the uplink signal is not degraded in order to ensure that the PER is solely due to the signal level on the downlink. Thus, just adding an attenuator between the BSE and the DUT is not a suitable way to measure sensitivity. Instead, separate uplink and downlink paths are typically used to ensure that the uplink has a lower path loss (see Figure 1).

Currently, the only way to perform a PER measurement of a WiMAX DUT as described is to use features of the TCP/IP protocol stack. The common method is to use an ICMP Ping/Echo request and count the number of echoes received. An alternate method involves sending UDP packets to an unused socket on the DUT and counting the ICMP “Destination Unreachable” responses. The disadvantage of these methods is that they require a full implementation of the protocol stack. This usually happens later in the design lifecycle of a product, once all drivers, etc. are complete. However, since a large part of what the TIS test evaluates is the impact of the platform on the receiver performance, it is desirable to be able to determine TIS early in the hardware development stage in order to resolve any RF issues that may require hardware modifications long before the device is near completion. This need should be addressed as devices are developed that are compliant with the “Wave 2” WiMAX certification requirements. These devices will be required to support Hybrid Automatic Repeat Requests (HARQ), which provide an error detection and acknowledgment (ACK/NACK) scheme within the 802.16 architecture that can be used for determining PER. By setting the retry count to zero, the DUT will indicate whether or not it received a valid packet, but the protocol will not try to recover the packet in following frames. Since this mechanism exists as part of the radio protocol itself, an embedded radio that is capable of making a wireless connection can potentially be installed in a platform (for example, a notebook PC) and tested without having to be able to access the device from the operating system.

Figure 5 illustrates the first ever fully automated WiMAX TIS data measured on a PCMCIA card device installed in the middle of the right side of a notebook PC. Due to the required test time, the data was sampled at a 15° angular resolution using the same theta dependent phi optimization as for the TRP test. Difficulties maintaining connections make estimating a typical test time difficult, but under ideal circumstances, a test time of five to seven hours could be expected for this sampling resolution. Sampling at 30° would likely still provide a suitable TIS result for this device, reducing typical expected test time to less than two hours. In this configuration, the TIS of the device was determined to be –90.7 dBm.

Intermediate Channel Sensitivity

One other piece of the RPT puzzle is what happens in between the frequencies where TRP and TIS have been determined. Due to the time required to perform these tests, it is impractical to repeat them at every supported channel of the device. In general, measuring full radiation patterns at the low, middle and highest frequency in each band is sufficient to determine if there are any variations in pattern that might affect the behavior of the device. However, in the case of receiver sensitivity, where a narrow-band interference source from the platform could have a devastating effect on only a narrow range of channels, a method is needed to measure sensitivity performance across the entire band. While it’s not necessary to test every 5 to 10 MHz wide channel at a 250 kHz spacing, at a minimum, one must be able to test a continuum of channels across the band to ensure that every possible interference frequency has been tested. This is where the intermediate channel sensitivity (ICS) comes in. Rather than performing TIS at every frequency, a much shorter normalization process is used. The assumption is that the pattern remains relatively constant, so that by re-measuring the sensitivity at one position on the TIS pattern for each desired frequency, the relative difference in sensitivity can be used to determine the corresponding TIS for each frequency. The resulting values have a slightly larger uncertainty than a full TIS scan, due to the reduced number of PER measurements used. As long as the DUT configuration is not altered between the reference and intermediate channel measurements, however, the result is sufficient for its intended purpose.

This last requirement tends to be difficult with the current state of devices. Most are configured to only connect to a single frequency without additional user intervention or to scan a very small range of frequencies. The number of intermediate frequencies to be tested across a band is larger than most frequency list tables implemented to support carriers with a given amount of bandwidth, and having the device scan every possible channel in the band is not a very efficient solution either. Since channel handoff functionality in WiMAX is currently still in the development stages, there is no way for the BSE to tell the DUT to switch to another channel. Instead the BSE can only change its signal to a target frequency and wait for the device to scan the band, find the signal, and register on the new channel. With as many as 1600 frequencies across a band, scanning all channels would be a slow process. Thus, to make this test practical, device firmware will need to be able to allow scanning a defined list of frequencies (up to about 90 for the worst case channel bandwidth/band combination) in order to allow the ICS measurement to be performed without user interaction with the DUT.

What About MIMO?

This question seems to come up every time there’s a discussion of radiated performance testing. WiMAX devices today are designed with a one transmit, two receive antenna configuration. In current implementations the two receive antennas typically provide some form of receive diversity, but by the time these devices are deployed, the intention is to offer downlink Multiple Input, Multiple Output (MIMO) for increased bandwidth and throughput over the current Single Input, Single Output (SISO) implementation. So how is MIMO tested in the above configuration?

The short answer is, it isn’t. Remember that the initial purpose of these tests is to determine edge of link performance as a fundamental indication of radio performance. MIMO does not work at the edge of the link, since it requires a considerable signal-to-noise ratio (SNR) to be able to resolve the multiple data streams. Thus, when asking “What about MIMO?,” the more important question is, “What do you want to know about MIMO?” Is it maximum possible throughput, or average throughput under a range of conditions? Or is the question, “What is the impact of the antennas and platform on ideal MIMO performance?” Or maybe just, “How well do the diversity antennas respond to a faded channel?” Each of these questions would have a different answer and require a considerably different test system implementation in order to measure them. However, until the wireless industry as a whole decides what radiated performance testing of MIMO means, and can then judge the value of such testing, the question will, for the time being, remain unanswered.

This does bring us to one of the last hurdles in RPT testing, which is fundamental to RPTs roots in passive antenna testing. The one underlying assumption when performing TRP or TIS measurements is that the radiation pattern does not change as it is being measured. However, with two receive antennas and a diversity algorithm applied to the two receivers, the device can react to the signal that it receives and change the relationship between the two antennas, thereby altering the radiation pattern and apparent performance of the device. While this “diversity gain” is actually a useful quantity to determine, it is not really the point of the TIS test. Moreover, the test itself makes the assumption that the device does not change, especially between the measurements of the two orthogonal polarizations of the measurement antenna. The assumption is that these two measurements are the components of one static field vector, but if the device can change its behavior between the measurements, that assumption is no longer valid. The reported result can actually be better than what is possible in real life. Worse yet, device orientation within the test system can alter the result, proving that this interaction between the test system and the device could completely invalidate the results of the test if it was allowed to occur.7,8

In order to avoid this possibility, it is necessary to test the receive performance of each antenna separately, or to disable any diversity algorithms such that the relationship between antennas remains fixed throughout the TIS test. This is not a function typically provided by WiMAX devices and will have to be added in order to ensure the integrity of RPT tests. Unfortunately, it’s not as simple as just disconnecting each antenna to test one at a time, since most devices use one of the antennas for uplink. Disconnecting the uplink antenna in order to test the second downlink antenna would remove the uplink from the system, preventing the connection to the BSE.

Conclusion

Wireless consumers are about to see a revolution in their mobile wireless experience in the form of WiMAX Forum Certified devices operating on a WiMAX network. By developing technology based on a non-proprietary international standard, and creating a certification program to cover conformance, interoperability and performance, the WiMAX Forum is poised to help companies deliver an international broadband wireless access network unlike any seen to date. The technical expertise that has gone into improving existing cellular technologies over the past decade is now being applied to emerging WiMAX devices to ensure that they meet demanding performance requirements at launch.

At the time of this writing, the WiMAX Forum™ Radiated Performance Tests (RPT) for Subscriber and Mobile Stations is in the final stages of balloting so that manufacturers may be prepared for RPT certification testing that will be required later this summer. In addition, because RPT provides final stage “user experience” test metrics, the Forum has also chosen to use RPT testing to help reduce overall test requirements. Manufacturers incorporating pre-certified “compliant portion” radio modules in their platforms can avoid repetition of many of the conformance and interoperability tests by just performing RPT to verify that the integrity and performance of the device is maintained. Through this approach, not only will overall test time be reduced, but also device manufacturers will be able to better understand the radiated performance of their products and learn to improve them throughout the product lifecycle.

References

1. WiMAX Forum™ Radiated Performance Tests (RPT) for Subscriber and Mobile Stations, V0.1.0 Draft, WiMAX Forum, Beaverton, OR, December 2007.

2. “Agilent Technologies, ETS-Lindgren Test Equipment Selected by AT4 wireless for Use in First Test Facility for WiMAX™ Radiated Performance Test,” www.forbes. com, January 10, 2008.

3. M.D. Foegelle, “Spherical Pattern Measurement Techniques for Low Directivity Antennas,” 24th Proceedings of the Antenna Measurement Techniques Association (AMTA 2002), Cleveland, OH, pp. 175–180.

4. M.D. Foegelle, “Antenna Pattern Measurement: Concepts and Techniques,” Compliance Engineering, 2002 Annual Reference Guide, Vol. XIX, No. 3, pp. 22–33.

5. M.D. Foegelle, “Antenna Pattern Measurement: Theory and Equations,” Compliance Engineering, 2002 Annual Reference Guide, Vol. XIX, No. 3, pp. 34–43.

6. M.D. Foegelle, “Analysis of Interaction Factors for Active Wireless Devices,” 29th Proceedings of the Antenna Measurement Techniques Association (AMTA 2007), St. Louis, MO, A07-0014.

7. M.D. Foegelle, “Over-the-Air Performance Testing of Wireless Devices with Multiple Antennas,” RF Design, February 2006, pp. 44–52.

8. M.D. Foegelle, “OTA Performance Testing of Wireless Devices with Multiple Antennas,” 28th Proceedings of the Antenna Measurement Techniques Association (AMTA 2006), Austin, TX, A06-0524.

Michael D. Foegelle is director of technology development at ETS-Lindgren, Cedar Park, TX, and is the editor and principal contributor for the WiMAX Forum™ Radiated Performance Tests (RPT) for Subscriber and Mobile Stations test plan. He received his PhD degree in physics from the University of Texas at Austin, where he performed theoretical and experimental research in both condensed matter physics and electromagnetic compatibility (EMC). In 1994 he began working for EMCO in Austin, TX (now ETS-Lindgren). There he has been integral to the development of products, software and test methods for wireless, RF and EMC testing. He has been involved in numerous national and international standards committees on EMC and wireless, including the ANSI ASC C63 working groups, the CTIA Certification Program Working Group on over-the-air performance testing of wireless devices, the IEEE 802.11 Task Group T for wireless performance prediction of 802.11 devices, the Wi-Fi Alliance Wi-Fi Mobile Convergence Group, the CTIA/Wi-Fi Alliance Converged Wireless Group and the WiMAX Forum’s Radiate Performance Test working group. He is co-chair of the CTIA’s Converged Devices ad-hoc group and has served as vice-chair of the Wi-Fi Alliance’s Wi-Fi/Mobile Convergence group. He has authored or co-authored numerous papers in the areas of electromagnetics, EMC, wireless performance testing and condensed matter physics, and is dedicated to advancing the state-of-the-art in radiated RF testing of emerging wireless technologies.