guinix TechNote: DSL in Butte

Now we're smokin'!

Introduction

We recently (January 2003) obtained Qwest DSL service in Butte, Montana. Overall our experience has been very favorable. We recommend it without reservation to any client or business in the area.

Background

After living in West Africa for many years, we were craving bandwidth.

Our online experiences in Guinea were limited to a single weekly session, painfully slow over a limited wireless hookup. We could at least do emails, and just barely enough browsing to be cruelly tantalizing.

So, when we first moved into Butte in October 2002, we were eager to get online with one of the high-speed "broadband" connections we had heard so much about. Especially since we had been reading promotional materials regarding the "fiber hotels" in the Butte Uptown district, we imagined all the amazing blazing possibilities that would be available to us.

Alas, we soon found out that the possibilities in Butte were, if anything, quite limited. Qwest was not yet offering DSL; no cable broadband available; and the "fiber hotels" were evidently someone's opium dream. All we found were a rogue bunch of bumbling wireless wannabees, still trying to figure out what they were doing.

Just to get going, then, we got a modem and the ubiquitous $19.95/month dial-up account. What a buzz kill!

Every week we continued to explore options for broadband. We found at least one DSL provider that did offer services in Butte, and they assured us our line was qualified. They proceeded to send us a contract document for the service: 4 pages of double-columned, very small print legalese.

Come on! The lease on our house is only one page!

Slogging through the small print, we found the contract terms were onerous: 2 year commitment, with a $500 early termination penalty; install fee of $200, --unless it was more; router cost of $329; speed change fee of $100; unspecified taxes and fees ("5.877% FUSF"??); etc. And all this for only 192k of bandwidth?

In a word, ridiculous. Yet many other ISPs were similar, both DSL and the wireless jokers.

Meanwhile, the Qwest website continued to say Butte would be online with DSL service in December 2002. So, after the new year and crossing our fingers, we renewed our inquiries.

On January 8th, we called Qwest. Not only was service available, but our line was qualified up to 4 megabit.

Good news! We immediately ordered the service through our dialup provider, OneWest.net. Within two days we had received our DSL router. Service was up and we were smokin' as of January 14th, less than a week after our order.

Yes, we had heard many of the horror stories about Qwest around the US, the long delays for installation and the unresponsive service. But for us these problems were simply not true. The service was prompt and trouble-free.

And what was the price of this joy? Setup fee of only $50, with no contract or early termination penalty. The router is provided on a rental basis of only $5 per month. It comes with an easy self-installation package that includes all the cables and line filters you will need for a typical setup in a home or small office.

We opted for the Qwest "Deluxe DSL", with a service level of 640kps download, 256kbps upload. Qwest charges $31.95/month for this service. A basic service level of 256k/256k is also available for $21.95/month, and higher level services are available, too. An ISP is required for any service level, and you have a choice of several here. We are with OneWest.net, for another $19.95/month.

Recap of monthly fees:

Note: it kind of cheeses me off that we need to have an ISP in this scenario at all, since we are providing all of our own services (DNS, email, web hosting, etc.) But you can't get around it. Qwest just provides the copper, the ISP provides the IP address and route to the backbone. The blow is softened by the fact that the OneWest.net support staff are super at follow-up and friendly at customer service.

How well does it work? We are consistently achieving downstream rates of 522 kilobits/sec, that is, better than 64K/sec. A 1Mbyte file will download in about 15 seconds. Big ugly pages like CNN load in a snap. We installed FreeBSD 5.0 over FTP, and are now able to hog down distribution ISOs at will. Burn, baby, burn!

At last, the broadband promised land!

Summary

Upsides:

• prompt installation
• modest installation cost
• uses existing phone line; separate line not required
• no external antenna or expensive customer premise equipment
• no contract period or termination penalties
• reasonable monthly rates
• includes static IP address
• excellent customer service from OneWest.net
• reliable
• fast

Downsides:

• Actiontec router is kindof cheezey

Technical

As mentioned above, Qwest currently provides the DSL router on a rental basis of $5/month. This equipment is an "Actiontec 1520". The physical unit is about the size of a thick paperback in a plastic housing. It combines the functions of a DSL modem, rudimentary firewall/gateway router, 4-port hub, and PCMCIA slot for optional wireless network card.

As far as "customer premise equipment" (CPE) goes, the Actiontec is definitely cheap, consumer-grade equipment. The installation and setup is, of course, oriented toward the novice Windoze user. And don't expect any customer support from your ISP for anything more technical than the basic setup. You'll need to bring your own smarts to the party.

The unit does function adequately, however, as a network gateway for a mixed network, including Linux, BSD and/or Mac OS systems.

In our case, we do not do Windoze, yet were able to configure the router completely through its built-in http interface from a FreeBSD browser. Out of the box, the unit is configured with a LAN-side IP address of 192.168.0.1; at least initially, then, you will need to configure one LAN host to be on a 192.168.0.0/24 address to access the router. Then simply http://192.168.0.1/ in, say, Netscrape, to reach the router's configuration interface. The router's LAN-side IP address may then be changed if desired through the "Advanced Setup", "LAN IP Address" configuration screen.

NOTE: unfortunately, the Actiontec router does not work at all with a text-mode browser, such as lynx. Major suction! And although the unit also includes a rudimentary telnet interface, it is completely undocumented.

Out of the box, the router is also configured as a DHCP server. While this may be suitable for its intended audience as a plug-and-pray network appliance, most Linux/BSD users will rather: a) configure their hosts statically; or b) run their own dhcpd server, with logging and fine-grained MAC address control. The DHCP feature may be disabled through the "Advanced Setup", "DHCP Server" configuration screen.

Our preference would be to have the public IP address on our own dual-homed OpenBSD box, and do all routing/packet filtering from there. Unfortunately, this is a no go. Although the Actiontec does have a "transparent bridging" mode, this setup is not compatible with the PPP over ATM (PPPoA) used by the ISP. (I spent a lot of time trying to make this work. If anybody figures out otherwise, please let me know.)

So, the public IP stays on the external side of the router. The "firewall" features of the router do include network address translation (NAT), so all hosts on the local network may then route transparently through the Actiontec to public Internet addresses.

The Actiontec also offers a "port forwarding" feature, allowing you to provide services from your own servers--such as smtp, dns and http--to incoming connections. Yet this feature is completely non-functional if any of the built-in "firewall" settings are enabled! Totally bizarre!

As for what is called the "firewall", the Actiontec provides 3 pre-programmed levels of filtering above NAT. It is otherwise completely unconfigurable, and offers no logging. As a result, it is best to consider the built-in firewall useless if you are providing any of your own services.

To work around these limitation, and for better security, we put up an OpenBSD box to serve as an internal gateway on the router. This box is dual-homed 192.168.0.254 and 10.0.1.254. All of our other local hosts are on the 10.0.1.0/24 network. The OpenBSD box is then the gateway for the local network, and the Actiontec is the external gateway for the OpenBSD box. We set port forwarding on the Actiontec into the OpenBSD box for smtp, dns, and http services. We then packet filter on the OpenBSD box for extra security. If we were running our services on a box other than the OpenBSD gateway, we would set up a DMZ server on yet another separate network, and do a secondary redirect from the OpenBSD box to the DMZ, rather than port forward directly from the Actiontec.

[A diagram here would help, wouldn't it?]

Ok, let's review. We have our "external gateway" network on 192.168.0.x, and our "internal gateway" network on 10.0.1.x. The Actiontec router will NAT outgoing packets from either network, so that all outgoing packets will appear to be coming from our single public IP address.

Now, consider a return packet coming back in that was set up by NAT on its way out. If the local destination address is on the 192.168.0.x network, no problem, since that's on the same network as the internal side of the router. But what about a packet destined for the 10.0.1.x network? How does the Actiontec know where to send it?

Here's where you finally get your money's worth from reading this article: you need to use the Actiontec "Advanced Setup" menus to set up a static route within the router.

Simply go to the "Static Routing" menu and enter the following:

Reboot the Actiontec and--presto!--the router now knows to forward packets destined for the 10.0.1.x network to our internal gateway, the OpenBSD box on 192.168.0.254.

[Another way to do all this, of course, is to NAT on the OpenBSD gateway first, so that all packets from the 10.0.1.x network will appear to be coming from the 192.168.0.254 address. The Actiontec router will then not need to know anything about the 10.0.1.x network at all, as it will only see outbound packets as coming from the 192.168.0.254 address, and NAT them again on the way out. You pays your money and takes your choice!]

All this may seem like a lot of redundant routing and filtering, but it does work. Maybe it even provides some security benefits. The great unknown, though, is the security of the Actiontec itself. If it can be hacked, your network can be screwed. We would feel much better if this were an invisible bridge to the solid OpenBSD system we know so well and trust completely.

Alternatives to the Actiontec router are available, but limited. For Linux/BSD systems, the only internal PCI card with PPPoA drivers is the Sangoma S518, at a cost of around $220. This would allow getting the public IP on your own host. For external DSL routers, the best alternative is probably the Cisco 678, no longer manufactured, but frequently available on eBay for around $100. Since the payback period for these alternatives is long compared to the $5/month rental for the Actiontec, and since it works, we will continue to stick with the Actiontec for now.

Update: Cisco 678

We recently (February 2003) obtained a used Cisco 678 DSL router through eBay, and have replaced the Actiontec 1520 described above.

Don't ask why. As discussed above, there probably isn't any good reason to do this. In the end, I guess it just seems like the Cisco does things more in line with the way I think they should be done.

The Cisco we received was locked with a password that the seller had forgotten. So the first task was to reset the device and clear the password. The procedure is not difficult, but not trivial either. The best course is get a Cisco with a known or open password.

Then we upgraded the operating system of the device to CBOS version 2.4.6. (Here, for expedience, we used the upgrade package published by Qwest, designed to be installed from a Windoze platform.)

With the device now ready for configuration, we used the "minicom" serial communications program on the OpenBSD gateway host to open a terminal session on the Cisco. Connect the pale blue "management cable" between the serial port and the management socket of the Cisco. Then, to invoke minicom without sending unnecessary initialization strings, use:

# minicom -o

A commented configuration session follows:

# configuration for Cisco 678
# wcm, 2003.02.15 - 2003.02.15
# ===

### start from scratch:
enable
set nvram erase
write
reboot

### set internal ip address:
enable
set interface eth0 address 192.168.0.1
set interface eth0 mask 255.255.255.0

### set PPPoA:
set ppp wan0-0 dns 0.0.0.0
set ppp wan0-0 ipcp 0.0.0.0
set ppp wan0-0 login ****
set ppp wan0-0 password ****

### set DMT parameters:
set interface wan0-0 close
set interface wan0-0 vpi 0
set interface wan0-0 vci 32
set interface wan0-0 open

### set static routes:
set route add ip 192.168.0.0 mask 255.255.255.0 \
  gateway 192.168.0.254
set route add ip 10.0.1.0 mask 255.255.255.0 \
  gateway 192.168.0.254

### set NAT and port forwarding:
set nat enable
set nat entry add 192.168.0.254 25 0.0.0.0 25 tcp
set nat entry add 192.168.0.254 53 0.0.0.0 53 udp
set nat entry add 192.168.0.254 53 0.0.0.0 53 tcp
set nat entry add 192.168.0.254 80 0.0.0.0 80 tcp

### set management/access interfaces:
set telnet enabled
set telnet remote 192.168.0.254
### code red protection:
set web disabled
set web port 8787
set web remote 255.255.255.255

### set passwords:
set password exec ****
set password enable ****

### save settings and reboot:
write
reboot

Note that we had to set explicit static routes to both of our local networks. Also note that the Cisco uses the "nat" commands to accomplish port forwarding for the services (smtp, dns, and http) we'll make available to the public. (To open all ports to the gateway host, we could use a single set nat entry add 192.168.0.254 instead.)

The Cisco is also capable of packet filtering according to explicit rules. These are configured with set filter commands. For our own network, we will do the additional packet filtering on the OpenBSD host instead.

Here is the packet filtering configuration on the OpenBSD gateway (using OpenBSD 3.2):

# /etc/pf.conf
# packet filter configuration
# openbsd gateway "nimba.guinix.com"
# wcm, 2003.02.17 - 2003.02.17
# ===
## xl0: "external" interface (toward cisco) 192.168.0.254
## xl1: "internal" interface (toward lan) 10.0.1.254

## macros for our interfaces:
ext_if = "xl0"
ext_ip = "192.168.0.254"
int_if = "xl1"
int_ip = "10.0.1.254"

## === scrub ===

## normalize all incoming traffic:
scrub in on $ext_if all fragment reassemble


## === nat ===

##
## XXX the cisco does nat on outbound packets
##
## set up ftp-proxy for our lan:
rdr on $int_if proto tcp from any to any port 21 -> 127.0.0.1 port 8081

### note: the above requires this entry in /etc/inetd.conf:
### 127.0.0.1:8081 stream tcp nowait root /usr/libexec/ftp-proxy ftp-proxy


## === filters ===
## unless matching rule is "quick", the _last_ matching rule wins

## unimpede loopback:
pass in  quick on lo0 all
pass out quick on lo0 all

## block and log all by default:
block             out log on $ext_if all
block             in  log on $ext_if all
block return-rst  out log on $ext_if proto tcp all
block return-rst  in  log on $ext_if proto tcp all
block return-icmp out log on $ext_if proto udp all
block return-icmp in  log on $ext_if proto udp all

## block and log any incoming without return route:
block in log from no-route to any

## block and log outgoing not from our ip!
## (ie, some kind of spoof or misconfiguration)
## XXX this would only apply if NAT on this host???
###block out log quick on $ext_if from ! $ext_ip to any

## block incoming broadcast packets:
block in quick on $ext_if from any to 255.255.255.255

## block and log incoming from reserved address space:
## XXX except 192.168.0.0/24, the cisco router ??
block in log quick on $ext_if from \
  { 10.0.0.0/8, 172.16.0.0/12, 255.255.255.255/32 } to any 

## icmp
## pass out/in certain icmp queries and keep state (ping):
pass out on $ext_if inet proto icmp all icmp-type 8 code 0 keep state
pass in  on $ext_if inet proto icmp all icmp-type 8 code 0 keep state

## udp
## pass out all udp connections and keep state:
pass out on $ext_if proto udp all keep state
## pass in certain udp connections (dns) and keep state:
pass in on $ext_if proto udp from any to any port 53 keep state

## tcp
## pass out all tcp connections and keep state:
pass out on $ext_if proto tcp all modulate state
## pass in certain tcp connections (smtp, dns, http) and keep state:
## XXX how to handle auth/identd (113) here and cisco??
pass in on $ext_if proto tcp from any to any \
  port {25, 53, 80, 113} keep state

## pass in data mode connections for ftp-proxy running on this host:
pass in on $ext_if proto tcp from any to $ext_if \
  port >= 41952 keep state

### that's all, folks!

This configuration provides a useful measure of layered security and fine-grained access control. Our services are then selected and configured to provide additional security: the Apache webserver runs in a chroot jail; email and nameserving are handled by the security-proven qmail and djbdns packages from Dan Bernstein.

The Cisco 678 performs at least as well as the Actiontec, and my impression is that it performs perceptibly better when making a number of connections in rapid succession. Whether this can be empirically proven or not, I don't know. But I do have greater confidence in our network configuration, and continue to be very pleased with our DSL connection to the world.

Disclaimer: neither guinix nor the author Wayne Marshall nor any of their relatives or associates are in any way connected with, compensated by, or have any financial interest whatsoever in Qwest, OneWest.net, Actiontec, Cisco or other entity mentioned in this article. The comments here reflect only our own experiences and opinions; others may certainly be different.