Sierra Wireless USBConnect Shockwave

By however you've seen all the ads pitching wireless companies' new 4G mobile broadband services and devices. Now beyond all the buzzwords and hype, which companies can reliably provide straightway-generation speed?

Wireless data speeds have soared: Since this time last year, the major wireless carriers, as a group, have increased their average download speeds for laptop-modem users by more than threefold, an apparent result of their urgent transition from 3G to 4G network research. Over laptop modems, the Big Four carriers nevertheless have a collective average download speed of approximately 3.5 megabits per second in our 13 testing cities, versus a near 1-mbps average download speed in those cities in the beginning of 2010, a remarkable improvement.

In our previous wireless-network performance studies, we measured the "reliability" of the data service, expressed as the percentage of tests in which we could obtain a good connection. Now our test results show that network service has improved to the point where it's rare to find an unusable signal or no signal at all. So we have retired our reliability measurement -- another testament to the dramatic improvements of the past year.

Verizon's 4G LTE (Long Term Evolution, latest standard in the mobile network technology) is for real: Verizon's 4G LTE (Long Term Evolution, latest standard in the mobile network technology) service, which is but in 38 U.S. markets, was widely available in 12 of our 13 testing cities. Our laptop-modem tests on Verizon clocked speeds that were far faster than those on competing 4G networks in the same tests. Verizon's network had an average download speed of approximately 6.5 mbps and an average upload speed of 5.0 mbps.

One important caveat: A relatively small number of Verizon clients currently use this new network. While our testing period, Verizon offered only two laptop-modem models that worked on the network, and none of the company's smartphones could take advantage of the new 4G speeds. The performance of Verizon's network could degrade as more people -- and devices -- connect to it.

Downside to Verizon's 4G success

And there's a downside to Verizon's 4G success. During the new 4G LTE network is lightning-fast, our smartphone-based tests suggest that the 3G CDMA network that most Verizon smartphone clients use today may in fact be getting slower. The connection speeds we measured on our Verizon testing smartphone stayed the same or decreased in 10 of our testing cities since last year. And for the moment, those CDMA phones are all that's available to Verizon Wireless clients.

T-Mobile smartphones are fastest: Verizon may have the fastest network for laptops, now in our tests T-Mobile had the speediest results for smartphones. The T-Mobile HTC G2 we used for testing produced a 13-city average download speed of nearly 2.3 mbps; that's about 52 per cent faster than the second-fastest phone, Sprint's HTC EVO 4G, which had an average download speed of 1.5 mbps.

T-Mobile as well impressed in our laptop-modem tests. Though only half as fast as Verizon's, T-Mobile's download speeds averaged nearly 3 mbps in our tests -- more than a threefold increase from the carrier's near 0.9-mbps average download speed in our January 2010 survey. With these laptop- and smartphone-based results, T-Mobile is proving to be a worthy challenger to its much-larger competitors.

And AT&T's speed gains didn't translate so then to our smartphone-based tests: The average download speeds we measured on our Apple iPhone 4 increased only 15 per cent over the speeds we measured on the same device in early 2010. Nevertheless, AT&T intends to launch its own 4G LTE network later this year, a move that might tip the balance of the 4G speed race in its favor once again.

Sprint needs more 4G: In the cities where Sprint offers its 4G WiMax service, clients saw large speed increases over the past year. Sprint's average download speeds grew 170 per cent to 2.1 mbps in our tests this year; the result would have been even better had the WiMax service been more consistently available throughout our test locations. Nevertheless in cities just as New Orleans, Phoenix, and San Diego, where Sprint on the whole relies on its 3G CDMA network for data service, download speeds have fallen, and remain so then below the 1 mbps mark.

In our study we tested both with representative smartphones and with a laptop employing a USB modem suggested by the carrier. The laptop-based testing, which uses the Ixia industry-standard testing software, provides more precise metrics than smartphone testing does. The laptop results are a good measure of the maximum performance possible on a network and are a satisfactory predictor of the speeds that the network will likely deliver to smartphones in a year or so.

We use Ookla, an FCC-approved Web-based speed test, to measure data rates on smartphones. Those results aren't as precise for a number of reasons: we must use different smartphones on different networks, and the results necessarily reflect the limitations of the smartphone's radio chipset, processor, and battery, and the test itself comes with a somewhat higher margin of error.

The charts below list the cities in the leftmost column; moving rightward across the chart, you can see the speed averages and network latency times for each of the four wireless networks. Speeds are expressed in megabits per second. Latency is represented in milliseconds. We recorded download and upload speeds and latency times while our laptop-modem tests, and download and upload speeds in our smartphone tests.

Our testing method is designed to approximate the experience of a real laptop-modem or smartphone user on any given day in their city. PCWorld's testing partner, Novarum, tested in each of our 13 cities while the first six weeks of 2011. At each of our 20 testing locations in each city, we took a "snapshot" of the performance of each wireless service, testing for upload speed, download speed, and network latency.

The country

Because we couldn't test every city in the country, we chose 13 cities that are broadly representative of midsize and large wireless markets in terms of size and topography: Baltimore, Boston, Chicago, Denver, New Orleans, New York, Orlando, Phoenix, Portland, San Diego, San Francisco, San Jose, and Seattle. Because wireless signal quality depends to a large extent on variables just as network load, distance from the nearest cell tower, weather, and time of day, our results can't be used to predict exact performance in a specific area. In other words, they illustrate the relative performance of wireless service in a given city on a given day. Each speed number has a margin of error of plus or minus five per cent.

"Verizon's new LTE service smokes," says Novarum CTO Ken Biba, who helped test the network. The speeds tell the story: Verizon's 13-city average download speed for laptop modems is approximately 6.4 mbps, more than double the average download speed of our study's second-place finisher, T-Mobile.

And that average includes Verizon's result in Portland, the only city in our study that has no LTE service but. Excluding Portland and looking at the performance of the LTE network only, Verizon's average download speed jumps to nearly 7 mbps. Only in Orlando did the network average less than 5 mbps, coming in at approximately 4 mbps.

Upload speeds were such as impressive. Overall, Verizon's upload speeds averaged in broad outline 5 mbps in our 13 testing cities; average upload speeds reached near 9 mbps in San Diego and San Jose. LTE networks differ from older 3G networks in that they are designed to be symmetric -- in other words, the pipe going from the client device up to the network is as wide as the pipe going down to the client. In many of our 260 testing locations, the Verizon network delivered upload speeds that were faster than its download speeds. San Diego's average upload speed was faster than its average download speed.

Such fast upload speeds can make bidirectional apps like videoconferencing, online gaming, and, later, mobile Voice over IP work far more smoothly and look and sound better. In these apps, the data you send from your device is such as important as the data you receive.

The network

Such apps as well depend on nearly-instantaneous response from the network, with minimal delay. To illustrate, in real-time VoIP (Voice over Internet Protocol) calls, network delay is mostly the cause of "lag" and echo. To have a natural-sounding VoIP (Voice over Internet Protocol) conversation, you need network latency of less than 150 milliseconds, and LTE proved better at assuring that than other networks in our tests. In our 12 testing cities where Verizon's LTE service is available, latency times averaged just 114 milliseconds, significantly shorter than latency times in the HSPA+ and WiMax networks we tested.

Verizon's LTE network gives us a nice look at the future of wireless service, however only a minority of the operator's clients are using the network at this stage. Verizon currently sells only two models of USB modems that can tap the network, and the company isn't saying how many modems it has sold. New LTE phones aren't likely to arrive until this summer. So Verizon's LTE network currently handles nowhere nearly the number of devices it will have to support hereafter.

"Verizon's new 4G network is a screamer, however that's in some cases because there's hardly anyone using it but," says Craig Moffett, a senior analyst for Sanford C. Bernstein & Co.

Verizon has been assuring skeptics that its network will remain such as fast when loaded up with devices. "We're very comfortable with the speeds we have said all along that our clients should expect: on average, 2 to 5 mbps on the uplink and 5 to 10 on the down," says Verizon Wireless spokesperson Thomas Pica. "That's on a fully loaded network.''

Moffett accepts that claim: "Even as [the network] begins to get loaded with the first smartphones this summer, it will probably keep the crown; as usual, theirs is the network to beat."

Still, at present, Verizon's smartphone subscribers rely on the company's 3G CDMA network. And that network, as demonstrated in our tests, in fact became slower over the past year.

In our January 2010 survey of 3G service, we measured average download speeds of around 1 mbps in nearly all of our testing cities on our Motorola Droid smartphone. In those same cities this year, we saw very similar performance on our Droid 2 smartphone -- again, most speed results were grouped around the 1-mbps mark, now the 13-city average download speed was seven per cent lower than last year's, at 1.008 mbps.

We found furthermore evidence of a stagnant CDMA network in laptop-modem tests in Portland, where the Verizon LTE service is not available. We found an average download speed of 0.8 mbps in Portland last year, and clocked an average speed of only 0.55 mbps this year. This, clearly, is lousy news for Verizon smartphone users, including those who recently bought the new Verizon iPhone.

Did Verizon build its impressive LTE network at the expense of furthermore upgrades to its 3G CDMA network? Are the majority of Verizon subscribers paying the price for the blazing speeds enjoyed by just a few? Verizon chose not to comment on these questions.

T-Mobile began to brand its HSPA+ network service and phones as "4G" this year. Its ad campaign promoting the offering -- you know, pretty girl, polka dots, poking fun at AT&T -- has been hard to avoid. Nevertheless our test results show that the carrier has been spending its money on far more than ad campaigns.

In short, T-Mobile's network is fast -- far speedier and more reliable than it was just a year ago -- and is in actual fact pumping out speeds that are competitive with the 4G services of the other providers. T-Mobile scored the fastest download speeds in our smartphone tests, and took a respectable second place behind Verizon Wireless in our laptop-modem tests.

T-Mobile more than tripled its download speeds in our smartphone tests since last year. In our smartphone tests using the T-Mobile HTC G2, we measured a 13-city average download speed of 2.3 mbps. T-Mobile's 13-city average a year ago was 0.72 mbps. In Denver and Seattle, our T-Mobile phone averaged download speeds of more than 3 mbps. We were able to achieve a connection speed of more than 2 mbps in 52 per cent of our tests.

Upload speeds as well rose dramatically from last year, improving from a 0.134-mbps average last year to nearly 1 mbps this year. The T-Mobile network produced average upload speeds above the 1-mbps mark in five of our testing cities: Baltimore, Boston, New York, Orlando, and Seattle.

T-Mobile as well scored very so then, and improved considerably, in our laptop-modem tests. The network averaged nearly 3 mbps for downloads, with average results nearing the 4-mbps mark in New York, Orlando, and Seattle. Overall, T-Mobile's download speed in our 13 testing cities grew 226 per cent from last year's 0.87-mbps average speed. Latency times averaged 173 milliseconds, not high enough to disrupt services like HD streaming video, now enough to degrade VoIP call quality slightly.

T-Mobile's competitors say that the HSPA+ innovation it uses is not as a matter of fact 4G as T-Mobile claims. That may be technically true, however T-Mobile has proven that through systematic software enhancements it can deliver speeds that are competitive with the 4G networks of its rivals. Given the nearly-term upgrade path of HSPA+ research, T-Mobile will likely be able to continue doing so for the at once few years.

Following T-Mobile's lead, AT&T began branding its wireless broadband service and phones as "4G" this year. And, like T-Mobile, AT&T's HSPA+ service is at any rate delivering 4G-like speeds. In our laptop-modem tests, the service produced an average download speed of 2.5 mbps in our 13 testing cities.

AT&T tells clients to expect download speeds of "up to roughly 6 mbps" in "key markets just as Chicago, Houston, and Charlotte [North Carolina]." Even though we didn't see many 6-mbps scores in our laptop-modem tests, the network did hit download speeds of more than 2 mbps most of the time. To tell the truth, AT&T showed average speeds of approximately 2 mbps or greater in all of the 13 cities in which we tested. The network produced its fastest average download speeds in Chicago and San Francisco.

AT&T's HSPA+ network produced latency times that were very similar to T-Mobile's. We measured an average delay of 169 milliseconds across 13 cities; we saw the highest average latency scores in San Diego and San Jose.

Consequently AT&T finished third in both our laptop and smartphone performance tests. In our laptop-modem results, AT&T trailed T-Mobile only slightly, however showed then less than half the download speed of Verizon LTE.

AT&T's slowing growth was moreover apparent in our smartphone tests. In our early-2010 study, we measured a 13-city average download speed of nearly 1.3 mbps on our AT&T iPhone 4, an improvement of 54 per cent over the previous year. In this year's tests using the same phone, that number moved up to 1.5 mbps, an improvement of only 15 per cent.

Some cities were better than others for AT&T smartphones: Chicago saw an average speed of 2.5 mbps during San Diego averaged only 0.8 mbps. Upload speeds improved dramatically, now, as our AT&T smartphone averaged 0.2 mbps in our 2010 tests and improved to just about 1 mbps this year.

AT&T believes that its new 4G smartphones and other devices will better utilize the speed of its network. "AT&T has introduced two 4G phones -- the Motorola Atrix and the HTC Inspire -- and has announced plans for about 20 4G devices this year," the company says in an e-mail. "Regarding network speed, thorough and expansive testing has concluded time and time again that AT&T operates the nation's fastest mobile broadband network."

AT&T's speed increases over the past two years can be attributed to software upgrades and infrastructure improvements. The operator completed a networkwide upgrade to HSPA 7.2 research in late 2009, at the time announced before this year that it had finished another upgrade to HSPA+ research, which it says allows for maximum theoretical download speeds up to 14.4 mbps. AT&T as well has been investing large amounts of capital in fiber-optic lines for the movement of cellular data to and from the core of its network.

Sprint says no such slowdown has occurred. "The 3G speed results you saw do not match what we see, and what the independent third party testing our network has reported," says Sprint spokesperson Stephanie Vinge-Walsh. "We haven't seen any significant degradation in 3G from last year to this year; our 3G speeds remain in the same range and at the same high dependability levels."

Sprint's 13-city average download speed of in broad outline 2.1 mbps represents a mix of CDMA and WiMax -- 3G and 4G -- connection speeds. Overall, we recorded throughput speeds of more than 2 mbps in about half of our tests. In the majority of our test cities where WiMax was available, we noted a in broad outline 50-50 chance of connecting to the service. There were exceptions: In Baltimore, Boston, and Chicago, the laptop-modem speed results reflected that the 4G network was available throughout the cities, with a few exceptions.

Of its 4G WiMax service, Sprint says users should expect average download speeds of between 3 mbps and 6 mbps, with peaks of more than 10 mbps. Our tests left us skeptical of Sprint's claim. We never saw a speed higher than 7 mbps, and we reached speeds of 6 mbps or more in only 5 of our 260 testing locations. The WiMax network produced a fair number of speeds within the 3-to-6-mbps window, however not consistently.

Sprint's CDMA and WiMax networks, combined, produced the worst average latency score in our tests, at 214 milliseconds. Such network delay can begin to degrade the smooth operation of real-time applications like video chatting and VoIP calling.

The same disparities in Sprint's 3G

The same disparities in Sprint's 3G and 4G networks showed up in our smartphone tests. In locations where WiMax coverage was spotty or nonexistent, average download scores were so then below the 1-mbps mark. In cities where we could regularly connect with the WiMax network, we saw download-speed averages of 2 mbps or greater.

Despite its overall speed gains, Sprint's service ranked last in both download speeds and upload speed in this year's laptop-modem tests. Had Sprint's WiMax network been widely available in all of our testing cities, the results would have been much different. The 4G network isn't slow, it's just not in enough places.

"Coverage has always been their Achilles' heel in 4G, and financial problems at WiMax partner Clearwire have slowed down their 4G network expansion near to a stop," says Sanford C. Bernstein's Moffett. "A year ago, they were first to market; nevertheless they're at real risk of falling behind."

An important transition from 3G to 4G is pursuant to this agreement way and will continue raising the bar for fast mobile broadband. If speeds continue increasing at the rate they have been over the past year, 3G data service will in the near future become just an unpleasant memory. Our tests show, permanently, that the 4G wireless service the carriers but offer -- if it's available in your neighborhood -- is already significantly faster than 3G service.

What will that mean?

What will that mean? The 4G service will very likely speed up your consumption of Web-based content, and smooth the operation of services just as streaming video. As a matter of fact, 4G speeds are likely to let you do things with your mobile device that you simply couldn't do with a 3G connection, applications just as video chatting, online gaming, and VoIP calling. 4G is the first incarnation of wireless broadband that might when all is said and done free people from the desktop, allowing us to manage our online lives whenever and wherever we want.

In each city, we tested from 20 locations situated in a grid over the center of the city. These locations are approximately 2 miles apart, allowing us to measure service levels among and between numerous cell towers. At each testing location, we subjected the networks to industry-standard stress-testing using laptops, and we put the networks even though Internet-based testing using smartphones.

Our laptop-modem tests use a direct TCP connection to the network to test the network's capacity -- in other words, the speed and performance that the network is capable of delivering to subscribers. To connect the laptop to the various networks, we used the fastest USB modem available, as recommended by the carriers themselves. We used the LG VL600 4G USB modem to test Verizon, the ZTE WebConnect Rocket 2.0 USB modem to test T-Mobile, the Sierra Wireless 250U AirCard to test Sprint, and the Sierra Wireless USBConnect Shockwave to test AT&T. Using the Ixia Chariot 4.2 testing tool on our laptop PC, we tested both the speed and the latency of the network.

To measure upload speed, Chariot sends a number of files from the Chariot client on the laptop to the local and distant network servers, again timing each transfer while a 1-minute period. We report the average of all of these transfers, both from the local and distant server, at each location as the average for that location.

The speed tests

During the speed tests, the Ixia testing software as well measures latency, or the time it takes for a packet to move from the client laptop to the network servers and back again. This metric, expressed in milliseconds, can reveal delays or bottlenecks in the flow of data through the network, and can foretell how so then real-time applications just as voice calling and video chatting -- which require near instantaneous packet transfer to work smoothly -- will work on the service being tested.

Our smartphone tests, which we run from the same locations as our laptop-modem tests, approximate the real-world connection between specific smartphones and specific networks. For the tests, we used AT&T's Apple iPhone 4, Sprint's HTC EVO 4G, T-Mobile's HTC G2, and Verizon's Motorola Droid 2.

On each phone we run the FCC-approved mobile-broadband performance test from Ookla. The test sends a large file back and forth between the smartphone and a network server, and at that time measures the speeds at which the data transfers. We perform three upload tests and three download tests at each testing location.

We tested all 13 cities while January 2011 and February 2011, using the same locations, methodology, and personnel we used to test those cities in our January 2010 tests. Maintaining a consistent methodology allows us to compare the performance of the networks over time and to look for evolutionary changes.