WAN T1 Overview
For a network engineer or systems administrator confronting the WAN links in their network, three significant differences between WANs and LANs are immediately apparent.
WAN troubleshooting is a team effort
First, and in some ways the most important difference, the WAN is not under your sole control. Cooperation with others outside your company is a necessity.
You may lease a circuit connecting two sites within your own company, or you may lease a WAN connection to a service provider such as an ISP. Even if both ends of the circuit are “yours,” the common carrier (typically a telephone company) owns, operates, and maintains the intervening lines, with all their attendant switches, repeaters, conditioners, and so on.
Maintaining the performance of your WAN connection is a team effort. Those things that are within your own control, you will of course learn to manage. But it is equally important to be able to provide the right information to the common carrier and to the “other end” of your connection when a problem requires action by others.
Analog makes a difference
Second, the analog nature of the WAN connection is important. While it is beyond the scope of this manual, a careful look at the analog aspects of WAN connectivity is always in order when troubleshooting. Long runs from the Telco demarc (an extended Demarc), line build out (voltage adjustments to optimize signal clarity) and a number of other factors have a significant impact on your WAN connection.
Another reminder of the analog nature of WANs is the way in which they can slowly degrade over time. Keeping track of throughput, error rates and performance over extended periods of time is a good practice. The records you keep may help you not only to identify problems, but also to make your case with the Telco engineers, and to help them diagnose-and fix-the problem.
WAN protocols share a different legacy
Finally, WAN protocols are the predecessors of Ethernet and IP, and they evolved in a very different environment with very different ideas of perfection. Two ideas (deprecated or totally rejected by the designers of Ethernet and IP) have shaped WAN protocols: very high reliability (founded on extremely reliable hardware) and a topology of switched point-to-point connections.
The emphasis on reliability is understandable. A failure rate of 1 in 100,000 seems trivial with only 25 nodes to maintain. With tens and hundreds of millions of nodes, it becomes a nightmare. The corollary of this real need for near-perfection was a glacial pace of change. Exhaustive testing, a consensus approach to innovation, and a reluctance to discard anything that actually works, have all characterized traditional WAN developments.
The topology of the switched network that creates and tears down a unique end-to-end path between any two communicating nodes is more than just a legacy of the traditional telephone system. It has also been a principle part of the business strategy of the carriers.
The various physical specifications of T1 and E1 lines and the link layer WAN protocols that use them are all influenced by these factors.
Figure H.1 OSI 7-layer model, showing WAN and IEEE protocols.
Physical aspects of T1/E1
The voice communications requirements of the telephone network helped define the standards of the telecommunications industry. The physics of electrical signalling to create a full-duplex voice connection led to the world-wide adoption of 64 kbps as the standard for a single telephone “line,” now called a DS0 (Digital Signal zero).
Individual lines were bundled together and later multiplexed together to form larger units and higher bandwidth. For example, 24 DS0 lines are bundled together to form a T1, two T1’s to form a T1c, two of those to form a T2 (= 4 x T1), and seven T2’s to form a T3 (with 672 DS0s).
Although the 64Kbps bandwidth of a DS0 is a worldwide standard, the exact sizes of the bundles differ slightly among the three main “spheres of influence” carved out by telephone monopolies in North America (T), Europe (E), and Japan (J). For example, E1 = 30 x DS0, E2 = 4 x E1, (but 3 x E1 is approximately = T2), E3 = 4 x E2, and E4 = 4 x E3 (or 3 x T3). The J1, J1c and J2 are identical to the T carriers of the same number, but J3 = 5 x J2, and J4 = 3 x J3. The hierarchy as a whole is known as the “Plesiochronous Digital Hierarchy” (PDH).
Carrier |
data rate |
DS0s |
|---|---|---|
| DS0 | 0.064Mbps | 1 |
| DS1 | 1.544Mbps | 24 |
| DS2 | 6.176Mbps | 96 |
| DS3 | 44.736Mbps | 672 |
| DS4 | 274.176Mbps | 4032 |
T1 and E1 are roughly the same class of connection in terms of available bandwidth. The T1 line rate is 1.544 Mbps, which consists of 24 DS0 (1.536 Mbps) plus signalling. The line rate for E1 is 2.048 Mbps, with no room left over for signalling. Instead, the first channel (timeslot 0, in E1) is used for framing information. In some E1 framing schemes, channel 16 is also used for signalling. On an E1 line using PCM-30 framing, for example, channels 1-15 and 17-31 are available for user data, and these 30 DS0 provide 1.920 Mbps for data.
T1 signalling and framing
T1/E1 lines, whether whole or fractional, use time division multiplexing (TDM) to send multiple channels over a single pair of wires (one pair in each direction). Each DS0 has its own time slot.
T1 is a synchronous communications medium and accurate timing is absolutely crucial to performance. Devices take their timing information from the network.
At the most rudimentary level, each time slot in a T1 line allows the transmission of 192 bits, plus a framing bit at the beginning of each time slot or “frame.” The word “frame” is in quotes here because this “slug of data filling a time slot” bears little or no resemblance to anything that would be recognized as a packet or frame on a LAN. It has one bit and a very precise synchronization between the sender and receiver to distinguish it from the rest of the voltage pulses on the line. When a connection is first made, it can take a moment to establish the framing!
More sophisticated framing schemes introduced in the 1970s and ’80s allowed the condition of the link to be monitored. The first such scheme was the Superframe (SF) which groups 12 DS0 channels together. A further improvement was made with Extended Superframe (ESF) which groups all 24 channels of the T1 together.
As mentioned in the previous section, framing and multi framing on E1 lines is somewhat different. Framing information is carried in the first channel (channel 0). Depending on the framing in use, additional signalling information may be carried in channel 16, and/or additional error checking may be included.
Link Layer of T1
In chronological order of development, the three most important link layer protocols on the T1 WAN are High-level Data Link Control protocol (HDLC), Frame Relay, and Point to Point Protocol (PPP). PPP is sometimes considered an extension of HDLC, and in fact, HDLC, in one form or another, is a part of nearly every WAN protocol.
The ISDN link layer protocol is specified in Q.921, with aspects of the network layer specified in Q.930. Call set-up signalling on the D Channel is covered in Q.931. Essentially, ISDN uses some number of DS0s as B channels (bearer channels), to carry the user data, and a single D channel (data channel) to carry call set-up, control, and related signalling.
X.25 was an early effort to create a public data network using the existing telephone network. It emphasizes reliable delivery at the expense of data throughput. Having the Link Layer handle acknowledgements and the retransmission of any packets that remain unack’ed is a drag on throughput in X.25. The X.25 modulo 128 variant permits more data to be in flight unack’ed, and so improves throughput somewhat.
All of these link layer protocols make use of HDLC, or contain implementations of HDLC. The next section looks at a basic HDLC frame, and at Cisco’s widely used implementation of HDLC.
HDLC and Cisco HDLC
The HDLC frame begins and ends with identical flags (0x7E). To prevent confusion between a flag and other data within the frame, any data containing more than five consecutive one bits has a zero bit inserted after the fifth one bit. Any zero appearing in the frame after five one bits is stripped out at the receiving end.
The Address field in most implementations contains nothing more than a statement of which end of a point-to-point link sent the frame. Some implementations may use this field for other things (addressing to a particular station in a multi-drop environment, for example), and the Address field can be extended to two bytes. When the address field is actually used to distinguish among possible recipients, stations ignore frames which do not contain their address.
Figure H.2 HDLC frame structure, in Asynchronous Balanced Mode (ABM)
The Control field is normally one byte, and describes the type of frame: Information (I), Supervisory (S), or Unnumbered (U). Information frames carry higher protocols such as TCP/IP. Supervisory frames handle flow control. Unnumbered frames are used to set up and tear down links and for miscellaneous functions.
The 3-bit N(R) element in the Control field for I-Frames and S-Frames is the sequence number of the received frame. The N(S) element in the Control field of I-Frames is the sequence number of the sent frame. When the Control field is extended to two bytes, the additional space is either used to extend the possible length of the sequence numbers, or it is reserved. These elements support reliable data transmissions in HDLC under ABM (Asynchronous Balanced Mode). ABM is “asynchronous” because nodes do not have to wait until a scheduled moment in order to communicate, and “balanced” because either end may initiate a conversation.
The P/F bit in the Control field is the Poll/Final bit. This is a legacy of the mainframe computing environment in which HDLC’s predecessors evolved. Primary stations use this bit to force a response from secondary stations, and secondary stations use it to indicate that they are finished transmitting to the primary station.
The Code elements are used to send messages about the state of the transmission. Examples include: Receive Ready (RR), Receive Not Ready (RNR), Reject (REJ) and Selective Reject (SREJ). In combination with an included sequence number, these commands allow a modest degree of flow control.
A two-byte FCS-16 frame check sequence value appears immediately before the final flag byte.
HDLC formed the basis of many subsequent protocols, and is implemented in a variety of forms in many other protocols, including PPP, X.25, and Frame Relay. It heavily influenced the Link Access Procedures such as LAPB (…Balanced), LAPD (…on the D Channel, for ISDN), LAPF (…Frame), and LAPM (… Modem). HDLC also provided a model for the IEEE 802.2 LLC (Link Layer Control) protocol.
Cisco HDLC
The original HDLC offered no clue as to the higher layer protocol it might be carrying. The address feature was meaningless on a link with only two ends, and the control features were redundant with those of higher layer protocols such as TCP. Cisco’s implementation of HDLC addresses all of these issues and greatly simplifies HDLC.
In Cisco HDLC (cHDLC), the Address field is always one byte and takes only one of two values: 0x0F (unicast), or 0x8F (broadcast). This refers only to the encapsulated protocol data, as cHDLC does not support multi-drop. The Control field is one byte, set to zero. The Cisco implementation does not support HDLC windowing. A new two-byte Type Code field is added after the Control field to support the Ethertype code, naming the encapsulated protocol. The rest of the frame-flags and FCS-remains the same.
Figure H.3 Cisco HDLC frame
Troubleshooting
The following are some examples of how to interpret the LEDs on the T1/E1 Pod to troubleshoot potential problems.
LEDs on T1/E1 Pod
- The Signal light on the front panel is not illuminated. This could have several causes:
- If no lights are illuminated, check that the power cord is plugged into the electrical outlet.
- Check that the correct cables are used and that they are properly connected.
- Check that the router (with the internal CSU/DSU) is properly configured to the correct channels.
- The Signal light on the front panel is illuminated, but the Framing light is not (when using framed T1/E1 data links).
- Check that the router is in agreement with the carrier service agreement (line provisioning) and that the propechannels are configured.
- The Alarm light is illuminated (RED):
- If this is the Alarm light on the CPE side, check the timing configuration of the router.If this is the Alarm light on the Networkside, check with the service provider for this network–something may be wrong with the T1/E line.
Tables
Supported protocols and physical interfaces
T1/E1 Pod RJ-48 pin connections
RJ 48 Connector |
CPE jack |
Network jack |
|---|---|---|
| 1 | Tx | Rx |
| 2 | Tx | Rx |
| 3 | Rx | Tx |
| 4 | Rx | Tx |
Glossary of terms
WAN Addresses and Names
WANPeek NX recognizes three types of addresses: physical addresses, logical addresses, and symbolic names assigned to either of these.
Physical addresses
The concept of an “address” on a Wide Area Networks (WAN) is quite different from that on a LAN or the Internet. Designed to work within the circuit-switched, point-to-point model of the telephone network, many WAN protocols (such as HDLC and PPP) have an extremely limited address function. Depending on the implementation, this may amount to little more than distinguishing between two ends of a circuit. Even when a larger address space is implemented, it is more in support of a multi-drop (primary to multiple secondaries) local environment, and has little to do with uniquely identifying a particular node in the vastness of global telecommunications.
The first major effort to create a uniform standard for packet switching (as opposed to circuit switching) over the existing telephone network was X.25. Even X.25 essentially sets up a telephone call to create a virtual circuit to its destination. ISDN also sets up its own calls, and offers some point-to-multi-point capabilities.
Frame Relay comes closer to the familiar model of the Internet, in that frames are routed through the switch fabric over a variety of possible paths to their destination. Frame Relay itself does not specify that destination as a fixed address, however. Higher level protocols must do that.
Frame Relay frames do contain a value called the Data Link Connection Identifier (DLCI). This identifies a connection to a neighboring device. In most Frame Relay networks, a DLCI value has only local significance and may be reused at other places in the network. DLCIs may be of different lengths, depending on the particular implementation of Frame Relay in use on a particular network. In the most common implementations, there are about 1,000 available DLCI values (apart from those reserved for signalling, for future use, and so forth). A single customer may be assigned multiple DLCIs, allowing them to use a single line to establish multiple simultaneous connections to the Frame Relay switch fabric or “frame cloud.”
WANPeek NX treats DLCI values as physical addresses, but any analogy with an Ethernet MAC address is bound to be at least slightly misleading. DLCI identifies a connection, and only one DLCI value appears in a Frame Relay frame. Every frame on that particular connection is identified by that DLCI. By knowing which end of the conversation was assigned the DLCI and adding direction information, it is possible to see that a particular frame is being sent either from that DLCI or to that DLCI, but there is no need for either end of the conversation to mention any other DLCI value.
The Packets view of the Capture window in Figure I.1 shows DLCI values in the Source and Destination address columns.
Figure I.1 DLCI values displayed in a Capture window
Logical addresses
A logical address is a network-layer address that is interpreted by a protocol handler. Logical addresses are used by networking software to allow packets to be independent of the physical connection of the network, that is, to work with different network topologies and types of media. Each type of protocol has a different kind of logical address, for example:
- an IP address (IPv4) consists of four decimal numbers separated by period (.) characters, for example:
130.57.64.11 - an AppleTalk address consists of two decimal numbers separated by a period (.), for example:
2010.42 68.12
Depending on the type of protocol in a packet (such as IP or AppleTalk), a packet may also specify source and destination logical address information.
For example, in sending a packet to a different network, the higher-level, logical destination address might be for the computer on that network to which you are sending the packet, while the lower-level, physical address might be the physical address of an inter-network device, like a router, that connects the two networks and is responsible for forwarding the packet to the ultimate destination.
The following figure shows captured packets identified by logical addresses under two protocols: AppleTalk (two decimal numbers, separated by a period) and IP (four decimal numbers from 0 to 255 separated by a period). It also shows symbolic names substituted for IP addresses (www0.wildpackets.com and ftp4.wildpackets.com) and for an AppleTalk address (Caxton).
Figure I.2 Logical AppleTalk and IP addresses and symbolic names
Symbolic names
The strings of numbers typically used to designate physical and logical addresses are perfect for machines, but awkward for human beings to remember and use. Symbolic names stand in for either physical or logical addresses. The domain names of the Internet are an example of symbolic names. The relationship between the symbolic names and the logical addresses to which they refer is handled by DNS (Domain Name Services) in IP (Internet Protocol). WANPeek NX takes advantage of these services to allow you to resolve IP names and addresses either passively in the background or actively for any highlighted packets.
In addition, WANPeek NX allows you to identify devices by symbolic names of your own by creating a Name Table that associates the names you wish to use with their corresponding addresses.
To use symbolic names that are unique to your site, you must first create Name Table entries in WANPeek NX and then instruct WANPeek NX to use names instead of addresses when names are available.
Other classes of addresses
When one says “address,” one typically thinks of a particular workstation or device on the network, but there are other types of addresses equally important in networking. To send information to everyone, you need a broadcast address. To send it to some but not all, a multicast address is useful. If machines are to converse with more than one partner at a time, the protocol needs to define some way of distinguishing among services or among specific conversations. Ports and Sockets are used for these functions. Each of these is discussed in more detail below.
Broadcast and multicast addresses
It is often useful to send the same information to more than one device, or even to all devices on a network or group of networks. To facilitate this, the hardware and the protocol stacks designed to run on the IEEE 802 family of networks can tell devices to listen, not only for packets addressed to that particular device, but also for packets whose destination is a reserved broadcast or multicast address.
Broadcast packets are processed by every device on the originating network segment and on any other network segment to which the packet can be forwarded. Because broadcast packets work in this way, most routers are set up to refuse to forward broadcast packets. Without that provision, networks could easily be flooded by careless broadcasting.
An alternative to broadcasting is multicasting. Each protocol or network standard reserves certain addresses as multicast addresses. Devices may then choose to listen in for traffic addressed to one or more of these multicast addresses. They capture and process only the packets addressed to the particular multicast address(es) for which they are listening. This permits the creation of elective groups of devices, even across network boundaries, without adding anything to the packet processing load of machines not interested in the multicasts. Internet routers, for example, use multicast addresses to exchange routing information.
Figure I.3 Broadcast packets are processed by all nodes on the network
Some protocol types have logical Broadcast addresses. When an address space is subnetted, the last (highest number) address is typically reserved for broadcasts. For example:
IP Broadcast Addresses typically uses 255 as the host portion of the address; for example: 130.57.255.255
AppleTalk Broadcast Addresses use 255 as the node portion of the address: 200.255
While conceptually very powerful, broadcast packets can be very expensive in terms of network resources. Every single node on the network must spend the time and memory to receive and process a broadcast packet, even if the packet has no meaning or value for that node.
Figure I.4 AppleTalk broadcast and multicast packets
Multicast Address. In IPv4, all of the Class D addresses have been reserved for multicasting purposes. That is, all the addresses between 224.0.0.0 and 239.255.255.255 are associated with some form of multicasting. Multicasting under AppleTalk is handled by an AppleTalk router which associates hardware multicast addresses with addresses in an AppleTalk Zone.
Ports and sockets
Network servers, and even workstations, need to be able to provide a variety of services to clients and peers on the network. To help manage these various functions, protocol designers created the idea of logical ports to which requests for particular services could be addressed.
Ports and sockets have slightly different meanings in some protocols. What is called a port in TCP/UDP is essentially the same as what is called a socket in IPX, for example. WANPeek NX treats the two as equivalent. ProtoSpecs uses port assignments and socket information to deduce the type of traffic contained in packets.