Multi-Homing Provides As Many Internet Routes As You Have ISPs - Technology Information

Computer Technology Review, July, 2001 by Dave Trowbridge

This article is the second in a two-part series.

Multi-homing connects a site to the Internet through more than one link, whether an Ethernet, T-1, T-3, or other pipe. Sites choose multiple connections for three basic reasons: redundancy, load balancing, and performance tuning, listed here in the order of management difficulty. A site may be multi-homed to a single ISP, which achieves only link redundancy. To attain ISP redundancy as well as the ability to load balance and tune performance a site must connect to multiple ISPs; of course, the connections to each of those may be multi-homed as well.

To qualify for multi-homing to multiple ISPs, a site will generally be required to obtain its own Autonomous System (AS) number and apply for at least a /24 block of its own P addresses (256 addresses, or what used to be called a Class C address block), rather than using addresses owned by the ISPs it's connecting to. The complications that can ensue from trying to announce addresses owned by one ISP through another are beyond the purview of this article.

The Ins And Outs Of Multi-Homing

While a site with its own AS and set of IP addresses may announce routes to attempt to control the way traffic reaches the site, this aspect of BGP management is somewhat arcane, and subject to interference or denial by the practices of the ISPs supplying the connectivity to the Internet. Their filtering policies can effectively mask a site's routes by aggregating them into a larger address block, and usually do to avoid the strain on the global routing tables. An aspect of BGP called communities--a kind of subgrouping within an AS--can give sites more control over how their route announcements are distributed and acted upon, and many ISPs offer communities for this purpose.

Nonetheless, the degree of autonomy available for sites wishing to announce their own routes to balance incoming traffic is limited, and with web sites, at least, traffic is so asymmetric (often 20:1 in favor of outgoing traffic due to the nature of HTTP), that such balancing is of limited benefit. Some excellent information on route announcements with BGP (aimed at ISPs) can be found at http://avi.freedman.net.

The options for controlling the routes taken by outbound traffic are limited too, of course, since multi-homed sites can only see the routes announced to them by the border routers assigned to them by each ISP they multi-home to: a single route for each destination reachable through a given ISP. Nonetheless, this gives a multi-homed site as many different paths through the Internet to a given destination (visitor, customer, business partner, employee) as it has ISPs. This ensures that should one ISP go down, the others will pick up the slack (assuming the links to them have sufficient capacity for the diverted traffic), giving a multi-homed site full redundancy. The fail-over is automatic with BGP or one of the hot standby protocols (HSRP and VRRP), so this redundancy comes at very little cost.

Leave The Driving To Us

The other benefits of multi-homing--load balancing and performance tuning--require more skill and management time, and many sites opt to rely on the fact that it's rare for a well-managed ISP with good connectivity (lots of peering and/or transit arrangements with backbones) to fall completely. For instance, XOR (www.xor.com) is a provider of customized e-business solutions that is itself multi-homed via T-3 links to five different Tier One (national or global) backbones as well as InterNAPs routing service (discussed below), and offers its customers redundant connections in the data center. Ned McClain, director of infrastructure engineering, notes that the company's engineers adjust the BGP tables daily to maintain optimal connectivity to the Internet for customers whose applications they have developed and host. "Many multi-homed sites will opt to balance for cost," he says. "But we're focused on routing around slow spots on the Internet, which requires routing and BGP expertise many customers don't have o r can't afford."

Likewise, e^deltacom (www.edeltacom.com), a hosting and managed services provider in the Southeast, offers BGP management as a managed service to customers in its 367,000 square foot data center. "We peer to four major backbones at three points around our regional packet-over-SONET ring," says Dave McGirt, vice president of engineering and CTO of the company. "Each customer in our data center has redundant Ethernet links to the Internet, and we manage their routing for them using the Keynote Global 50 benchmark and our own tools to monitor their connectivity to points around the world."

The tools used by companies like e^deltacom are generally home-grown scripts based on trace-route and ping (ICMP-based applications that use TCP fundamentals like the Time-To-Live variable in packets to discover routes and the condition of network elements) to probe the Internet, usually in conjunction with synthetic traffic-based probes from companies like Keynote. "We use Keynote and scripts that trigger every 60 seconds to gain a broad overview of Internet conditions," says Kevin Martin, founder and CEO of Pair Networks (www.pair.com), a hosting provider that maintains 4 DS-3 and OC-3 link to the five backbones it is multi-homed to. He notes that Pair generally makes BGP changes to respond to problems as needed, and then goes through the BGP tables in depth about every three months to tune performance more closely based on what they've seen during that period.