Gigabit Ethernet Races Along

By OLIVER RIST

As you might expect, we merry geeks at InternetWeek have an inherent weak spot in our hearts for bandwidth-widening technologies, especially Gigabit Ethernet, or GbE. And since we've made such a habit out of following and testing top industry GbE products, you may be wondering if there have been any noteworthy changes since our last roundup (InternetWeek, May 4).

You bet.

Because it's really just in its infancy, GbE is evolving at a breakneck pace. Products that were deemed winners only six months ago may have missed key technical or standards development and were left in the dust this time. Since these switches still don't come cheap, it's important to stay on top of the news if you're planning a migration.

Key developments include not only a fully ratified Gigabit Ethernet specification, but a spec covering quality of service (QoS) as well. The IEEE's 802.1p specification lays down some bare-bones provisions for QoS management over GbE, but many in the industry feel it's not enough-especially the vendors that participated in our test. As you might expect, most of our contenders chose to implement 802.1p, but tacked on proprietary options for QoS as well. This is worth careful consideration, since it can be a mess trying to figure out whether today's proprietary QoS can be upgraded to tomorrow's interoperable, spec-compliant QoS.

The other development you've probably been reading about is the recent passing of the 1000Base-T standard. While much has been made about the ability to run GbE over Category 5 or Category 6 copper cables, none of our contenders supported this specification at test time. (After all, the spec was only passed in July.) However, even if the spec had been passed earlier than that, we're not sure we would have made a big deal about testing it anyway.

GbE is definitely still a backbone technology. We recommend it only for networks with hundreds of high-performance desktop clients and a clear desktop-to-data-center switch migration from standard, switched 10Base to Fast Ethernet and then finally GbE at the core.

As such, there is little immediate need for copper cabling. GbE to the desktop is still about as rare as a snowy day in Honolulu, and designing your network core around standard building cable is madness. When GbE stretches its tendrils outside the data center, we'll start testing it over copper. Until then, multimode fiber is our GbE pipe of choice.

Who Played Along
Seven vendors heeded our Hawaiian conch testing call: Cabletron, Compaq, Extreme Networks Inc., Foundry Networks Inc., Intel, Lucent Technologies and 3Com. Though we invited many more, most of those who didn't attend missed out only because they couldn't get back to us in time before all our test slots filled up.

We were disappointed that Packet Engines Inc. didn't snag one of our slots in time, but the company partially made up for it by supplying a few of its G-NIC Gigabit Ethernet network interface cards for use in our core-connected servers. Phobos Corp. promised us some of their NICs but wasn't able to deliver them in time. Instead, we mixed things up with additional GbE NICs from Alteon Networks Inc., Intel and Compaq.

How We Put It Together
If you've been following our Hawaiian switch tests, you'll know that we tend to test these switch solutions a little differently from the rest of the industry. Well, actually, that's no longer true. So many of you have written us favorable letters commending this testing approach that it seems to have rubbed off on our sister publications at CMP Media. But remember, you saw it here first.

Our idea was to run a real-world solution-something on which you can actually base a purchasing decision. How did we do that? Well, we didn't bang a bunch of 64-byte packets through every switch port and call it a day. Instead we allowed each vendor a maximum of flexibility in designing a solution to meet our test bed's proposed needs and then we noted how it ran using a real-world traffic model we got from the University of Hawaii's IT department.

That means our test bed included different packet sizes and traffic patterns that spanned multiple subnets and so, of course, it included VLANs. Tied to this, each vendor was asked to supply either a hardware-based, full-motion video solution or to hook into the system Optivision Inc. had supplied our lab. The reason our test bed included this was because the University of Hawaii is developing distance learning and also experimenting with recording lectures for later playback by students, as well as general archiving.

For testing purposes, we used the video stream to demonstrate QoS. All seven vendors claimed the ability to handle QoS, especially when we told them it meant only being able to maintain a steady video stream while we oversubscribed the backbone link with other traffic. We oversubscribed it using Ganymede Software Inc.'s Chariot application traffic simulator. We also used Netcom Systems Inc.'s SmartBits as well. We gave the vendors absolutely no guidelines on how to implement QoS. Each vendor was allowed to configure its own QoS method. Steady video was all we wanted to see.

Frankly, because of the hoopla surrounding QoS on GbE at the moment, we considered this test as the meat of our benchmarking effort. As such, we were amazed at the rate of failure. Now, let's first define failure. It didn't mean that the video stream died completely once the switch port was oversubscribed, although that sometimes did happen. It meant that we saw digital artifacts that either marred or interrupted video play. In practice, you could probably get away with marred video, but if we saw it during our test, the vendor's switch didn't pass our QoS challenge.

What was the general cause of failure? Actually, it varied considerably. Some vendors delivered a clear explanation of why their switches failed, while others just scratched their heads.

An interesting side note is that, despite all the industry screaming about Layer 3 QoS, only two vendors actually attempted it. Everyone else chose to stay with Layer 2, typically video port to video port. It's also important to note that this oversubscription test was not based on head-of-line blocking (although we did run that as a separate test as well).

During this lab test, each vendor's engineers were able to configure as they would the multiple ports located on both core switches. They were then able to oversubscribe the backbone trunk that linked those core switches.

The State Of Gigabit Ethernet
QoS is the most important issue in the GbE industry at the moment and will probably remain so for some time. It was certainly the largest point of variance among our seven contenders' products. Performance rate, for example, is pretty flat across the entire GbE product segment. Well, that is if you can call 30 million to 40 million packets per second flat.

But the speed of performance isn't what each switch is rated by. It's whether it can maintain that rate when carrying a varied traffic load and advanced management provisions, all the while talking to different vendors' NICs. That turned out to be a tall order for all but a few of our competitors.

Price is still a nonissue. Whether you choose a chassis-based, multislot switch or a fixed-configuration stackable unit, GbE switching still costs several thousand dollars per port.

While this closely resembles the price of most backbone-oriented switch technologies such as ATM or FDDI, raw switch price isn't where GbE tickles the sweet spot. GbE means easier integration with existing Ethernet networks and an even easier extension of fast bandwidth out to departments-or even to desktops, should that be required. That saves you time and headaches, both significant cost factors in IT departments.

Although still not perfect, that aspect of GbE can certainly be described as improving. With the ratification of important standards, switch vendors have an easier basis for interoperability and new features. While we expect to see even more improvements in future models-in terms of silicon boosts to switch speeds as well as more refined switching codes-all seven contenders reviewed here represent a powerful backbone boost.

Cabletron
Cabletron continues to impress us, both with its unending enthusiasm to forsake its blissful New Hampshire utopia in favor of our Honolulu testing lab, as well as its incessant ability to just plain work. Had things run smoothly for Cabletron, this would be the least interesting review of the bunch. But things didn't run smoothly and what resulted was an impressive tale of perseverance on the part of Cabletron's engineers.

Cabletron's solution to our University of Hawaii GbE test bed consisted of one SmartSwitch Router 8000 and another SmartSwitch Router 8600, both of which are multislot, Layer 3 switch chassis, with the 8600 carrying the larger frame. Cabletron also demonstrated its SmartSwitch Router 2000 on its solution's edge. The 2000 is a fixed-configuration, downsized version of the 8000 family.

The trouble Cabletron ran into during our test was with Federal Express, which apparently decided to play soccer with the original shipment of SmartSwitch Routers. Caught off-guard with even its backup devices damaged, Cabletron still didn't back out of the test. Frankly, most vendors would have. Instead, the company pulled untested units straight off the assembly line and practically had them beamed to Honolulu.

With no time for hot-staging, these new devices were dropped straight into our test bed and ran with absolutely no problems. In fact, Cabletron set an all-time speed record for completing the test suite from setup to breakdown: four hours. Besides the rock-solid functionality, that kind of faith in its products is rare for switch companies these days.

On a features note, both the 8000 and the 2000 are chock-full of the latest and greatest. The multislot 8000 can be had in a medium-sized rack-mount 8000 version or in the taller, beefier 8600. Both sets support Layer 3 and Layer 4 switching, according to Cabletron, with the 8000 capable of a 16-Gbps, nonblocking fabric and the 8600 running at 32 Gbps. The 8000 carries up to 56 10/100 ports and up to 14-GbE ports, while the 8600 ups that to 120 10/100 ports and 30-GbE ports.

Although our performance tests bore out Cabletron's high-speed claims, the company didn't show us anything regarding Layer 3 or Layer 4 traffic management or QoS. Instead, the company chose to set up a relatively quick Layer 2 solution that passed our QoS test with flying colors. This is not a negative, it's just indicative of the current state of the market: Lots of talk about Layer 3 but, when push comes to shove, Layer 2 is still your safest bet. Admittedly, Cabletron has shown Layer 3 and Layer 4 in our other testing situations; we're just griping now because they didn't run it here.

The SmartSwitch Router 2000 is a cute little box that holds an 8-Gbps, nonblocking fabric connected to 16 10/100 ports (fixed), with an option of another 16 10/100 or four GbE ports. Basically, this is just a scaled-down version of the 8000 offering almost the same feature set, with the exception of redundant power supplies.

Cabletron didn't compete in our long-link test, mainly because it did not have time to run down its long-link blades after the initial shipping fiasco. (Not to mention that the solution would have been third-party anyway.) The company did drop some interesting hints about port trunking, however. Cabletron apparently is working to implement a standard for intervendor switch-port trunking at the Gigabit level.

Though it doesn't beat some of its competitors' prices, Cabletron's stunning performance in the face of adversity is definitely worth the extra money in our eyes.

Compaq
Compaq had results similar to Cabletron's, with the difference being that they encountered little FedEx shipping difficulty and also managed to show Layer 3 functionality-mainly on a dare. Although the company hit a snag during configuration, all of that was smoothed out of the way once its engineers figured out that video traffic should be inhabiting its own VLAN.

Compaq's GbE solution revolved around two 5422 Enterprise Gigabit Ethernet Switches (larger cousins to its 5411) to comprise our network core, while edge duties were performed by a single 5411 and an SW 5450. The 5411 held eight 10/100 ports and two GbE ports, though it can hold up to three GbE ports. The 5450 is a massive Fast Ethernet switch capable of carrying up to 48 10/100 ports with dual GbE uplinks. All these switches are either rack-mounted or stackable.

Interestingly enough, Compaq also brought along samples of its newest NC3131 NIC, which is, in essence, a 10/100 server NIC based around a 64-bit PCI connection. But Compaq also showed us new daughtercards that easily snap into the NC3131, adding either two more 10/100 ports or two GbE ports. The card is fully integrated with Compaq's server management software, as demonstrated on the Compaq ProLiant 6000 we happened to have in the lab. It also supports Adaptive Load Balancing and Cisco's Fast EtherChannel, which allows theoretical throughput rates up to 800 Mbps using only Fast Ethernet connections.

Getting back to its switching solution, the only disappointing part of Compaq's show revolved around its trumpeting of its new "application-aware networking" initiative. This is typical of the vague terminology many vendors are using to describe tentative forays into Layer 3 and Layer 4 traffic management and routing. Just to be spiteful, we decided to call Compaq's bluff.

After its press and marketing folks had finished their presentation, we wandered back into the lab to check up on the engineers' progress. Lo and behold, they had configured the QoS portion of the test to run at Layer 2, despite Compaq's "application-aware" initiative.

After the company had completed its test at Layer 2 (passing with no problems), we snidely inquired where all the Layer 3 and Layer 4 technology was. Although both Compaq's marketing and engineering staff was quick to point out that the Layer 4 stuff wasn't yet available, they did reassert Layer 3 abilities. And then they showed it to us, becoming the second (and last) vendor to do so during our roundup. Red-faced, we ate our crow in silence.

Compaq was even nice enough to give us a sample of its Compaq Networking Management Software (CNMS), which is bundled with all Compaq switches and allows a Windows 95/98/NT-based graphical management interface into the switch. The only advantage of CNMS over the built-in, Web-based management interface, however, is that you can manage multiple Compaq switches via CNMS-not yet so via the Web.

Extreme Networks
The new addition to Extreme's already incredibly popular GbE switching family is the massive BlackDiamond 6800. Where its Summit line has been comprised solely of stackable, fixed-configuration switches, the BlackDiamond is an imposing, chassis-based monster with a 64-Gbps, nonblocking switch fabric that supports up to 256 10/100 ports or 48 1000Base-SX/LX ports, or some mixture in between.

Extreme employed its BlackDiamond as both backbone and edge device on one side of our test core. On the other side sat a combination of its Summit4 and Summit1 switches that we already had in the lab. The Summit4 played core switch, while the Summit1 handled edge duties. Additionally, Extreme brought along its new Summit GbX, which is basically a separate long-link device that allows connected switches to extend transmission distance between and 50 and 62 miles, full-duplex over single-mode fiber.

While those specs did make us arch our eyebrows in appreciation, Extreme's troubles began shortly thereafter. For one, because the company didn't send proper interconnection hardware, we were unable to connect two BlackDiamonds together to form a true backbone. The BlackDiamond and Summit4-Summit1 combination described above was functional for test purposes, but didn't mirror what would be employed in real life.

From there, the company ran into a horde of configuration problems that boiled down to the BlackDiamond being both a hardware and software beta model. Though it turned out we also had to replace a faulty Packet Engines GbE NIC in our ProLiant 6500. Once updated to production code, things ran acceptably-although Extreme had to disable load balancing to make things run.

QoS was another problem. Extreme configured things at Layer 2, but we found that this is configurable only in increments of five percent of the total bandwidth. Frankly, that's a little rough to suit our tastes and will probably wind up costing you in wasted bandwidth. Extreme definitely needs to implement more granular control in this regard. While this scheme protected the video, it forced the BlackDiamond to drop so much of the other traffic that Chariot 2.1 could no longer maintain its connection.

Extreme finally got through the QoS test by doing away with a combination port-based and VLAN-based priority scheme and concentrating solely on port-based, though video also was taking up its own VLAN. This allowed it to finish the QoS test, but without a passing grade, as we still saw significant artifacts. On a positive note here, however, when we ran our Netcom Bit Blaster test, Extreme managed to fully protect the video stream even though the Netcom traffic was running at 100 percent wire speed.

Finally, Extreme also demonstrated its Summit GbX long-link capability, which we tested via 62 miles of coiled, single-mode fiber. True to its word, Extreme maintained connection the entire distance, which was impressive enough to make up for some of the problems we had seen.

While the BlackDiamond didn't manage a stellar showing in this review, it did manage to come through most of our tests with a little coddling. Once Extreme works through its beta troubles and irons out its QoS capabilities, this will be a powerful gigabit contender.

Foundry Networks
Foundry is a company on the move. Working feverishly, the company has established itself as a GbE powerhouse in a relatively short time, coincidentally making itself an acquisition target for many larger switch manufacturers that may be lacking GbE product representation. In answer to our test bed's cry for help, Foundry delivered a combination of its BigIron 8000 and Big-Iron 4000 multislot chassis switches to form the backbone and its FastIron 10/100 switches to perform edge duties.

The BigIron 4000 is an impressive box, capable of carrying up to 32 GbE ports attached to a 64-Gbps switching fabric. Even more impressive, its BigIron 8000 sibling doubles all those rated specs. It's rated to support all the latest GbE standards, including a rated long-link capacity of 70km. However, we were only able to verify 50km because our fiber coils ran in increments of 25km.

The BigIron can be managed either through an internal Web interface or via a graphical management application dubbed "IronView." The representing Foundry engineer, however, preferred his command-line interface, which looked very similar to Cisco's.

The FastIron workgroup switch is your basic fixed-config, stackable 10/100 switch with dual GbE uplink ports. Because it supports multiline trunking, this could give a Layer 2 FastIron stack a wide, 2-Gbps duplex pipe to the core.

Foundry also sells a TurboIron/8 product, which is similar to the FastIron box. The exception here is that it provides 100Base-FX connectivity in sort of a "downlink" configuration, while being mainly designed to stack GbE ports on top of each other. It can stack eight GbE ports to a TurboIron box. Both the FastIron and BigIron families are rated as Layer 3 Routing switches.

Unfortunately, when we arrived at the all-important QoS test, Foundry was unable to make Layer 3 IP policy-based QoS work. Instead, we wound up defaulting down to Layer 2 port-based QoS. This worked except for a few digital artifacts on both ends of our video stream. Although you can get away with video artifacts of this type in real life, Foundry did, technically, fail our QoS test because of them.

Foundry had no trouble with the rest of our test suite, except for a mysterious routing problem that seemed to solve itself overnight. It turnedout it was a faulty fiber cable the company had brought along to augment our lab's limited supply. Although we were unimpressed with Foundry's QoS implementation, the company's switches were rock-solid in all other areas. The capacity specs on both the BigIron 4000 and BigIron 8000 are awesome.

Also important to note is their pricing (see page 56), which looks deceptively high. While the amount listed reflects all of the hardware that Foundry brought as their total solution, please be aware that these folks brought along far more equipment than they had to and more than any other vendor did. Thus, the high price. That aside, if Foundry works through its QoS issues, their switches will be mighty contenders in this arena.

Story Continues Here