TCP:
The Transmission Control Protocol (TCP) is a virtual circuit protocol that is one of the core protocols of the Internet protocol suite, often simply referred to as TCP/IP. Using TCP, applications on networked hosts can create connections to one another, over which they can exchange streams of data. The protocol guarantees reliable and in-order delivery of data from sender to receiver. TCP also distinguishes data for multiple connections by concurrent applications (e.g. Web server and e-mail server) running on the same host.
TCP supports many of the Internet's most popular application protocols and resulting applications, including the World Wide Web, e-mail and Secure Shell.
In the Internet protocol suite, TCP is the intermediate layer between the Internet Protocol (IP) below it, and an application above it. Applications often need reliable pipe-like connections to each other, whereas the Internet Protocol does not provide such streams, but rather only best effort delivery (i.e. unreliable packets). TCP does the task of the transport layer in the simplified OSI model of computer networks. The other main transport-level Internet protocol is UDP.
Applications send streams of octets (8-bit bytes) to TCP for delivery through the network, and TCP divides the byte stream into appropriately sized segments (usually delineated by the maximum transmission unit (MTU) size of the data link layer of the network the computer is attached to). TCP then passes the resulting packets to the Internet Protocol, for delivery through a network to the TCP module of the entity at the other end. TCP checks to make sure that no packets are lost by giving each packet a sequence number, which is also used to make sure that the data are delivered to the entity at the other end in the correct order. The TCP module at the far end sends back an acknowledgement for packets which have been successfully received; a timer at the sending TCP will cause a timeout if an acknowledgement is not received within a reasonable round-trip time (or RTT), and the (presumably lost) data will then be re-transmitted. The TCP checks that no bytes are damaged by using a checksum; one is computed at the sender for each block of data before it is sent, and checked at the receiver.
Contents
1 Protocol operation
1.1 Connection establishment
1.2 Data transfer
1.3 Congestion avoidance
1.4 TCP window size
1.5 Window scaling
1.6 Connection termination
2 TCP ports
3 Development of TCP
4 TCP Over Wireless
5 Hardware TCP Implementations
5.1 TCP Offload Engines
6 Debugging TCP
7 Alternatives to TCP
8 Packet structure
8.1 Header
8.2 Data
9 See also
10 External links
[edit] Protocol operation
An abridged version of the TCP state diagram.Unlike TCP's traditional counterpart — User Datagram Protocol — that can immediately start sending packets, TCP provides connections that need to be established before sending data. TCP connections have three phases:
connection establishment
data transfer
connection termination
Before describing these three phases, a note about the various states of a connection end-point or Internet socket:
LISTEN
SYN-SENT
SYN-RECEIVED
ESTABLISHED
FIN-WAIT-1
FIN-WAIT-2
CLOSE-WAIT
CLOSING
LAST-ACK
TIME-WAIT
CLOSED
LISTEN
represents waiting for a connection request from any remote TCP and port. (usually set by TCP servers)
SYN-SENT
represents waiting for the remote TCP to send back a TCP packet with the SYN and ACK flags set. (usually set by TCP clients)
SYN-RECEIVED
represents waiting for the remote TCP to send back an acknowledgment after have sent back a connection acknowledgment to the remote TCP. (usually set by TCP servers)
ESTABLISHED
represents that the port is ready to receive/send data from/to the remote TCP. (set by TCP clients and servers)
TIME-WAIT
represents waiting for enough time to pass to be sure the remote TCP received the acknowledgment of its connection termination request. According to RFC 793 a connection can stay in TIME-WAIT for a maximum of four minutes.
[edit] Connection establishment
To establish a connection, TCP uses a 3-way handshake. Before a client attempts to connect with a server, the server must first bind to a port to open it up for connections: this is called a passive open. Once the passive open is established, a client may initiate an active open. To establish a connection, the three-way (or 3-step) handshake occurs:
The active open is performed by sending a SYN to the server.
In response, the server replies with a SYN-ACK.
Finally the client sends an ACK back to the server.
At this point, both the client and server have received an acknowledgement of the connection.
Example:
The initiating host (client) sends a synchronization (SYN flag set) packet to initiate a connection. Any SYN packet holds a Sequence Number. The Sequence Number is a 32-bit field in TCP segment header. For example let the Sequence Number value for this session be x.
The other host receives the packet, records the Sequence Number of x from the client, and replies with an acknowledgment and synchronization (SYN-ACK). The Acknowledgment Number is a 32-bit field in TCP segment header. It contains the next sequence number that this host is expecting to receive (x + 1). The host also initiates a return session. This includes a TCP segment with its own initial Sequence Number value of y.
The initiating host responds with a next Sequence Number (x+1) and a simple Acknowledgment Number value of y + 1, which is the Sequence Number value of the other host + 1.
[edit] Data transfer
There are a few key features that set TCP apart from User Datagram Protocol:
Error-free data transfer
Ordered-data transfer
Retransmission of lost packets
Discarding duplicate packets
Congestion throttling
In the first two steps of the 3-way handshaking, both computers exchange an initial sequence number (ISN). This number can be arbitrary. This sequence number identifies the order of the bytes sent from each computer so that the data transferred is in order regardless of any fragmentation or disordering that occurs during transmission. For every byte transmitted the sequence number must be incremented.
Conceptually, each byte sent is assigned a sequence number and the receiver then sends an acknowledgement back to the sender that effectively states that they received it. What is done in practice is only the first data byte is assigned a sequence number which is inserted in the sequence number field and the receiver sends an acknowledgement value of the next byte they expect to receive.
For example, if computer A sends 4 bytes with a sequence number of 100 (conceptually, the four bytes would have a sequence number of 100, 101, 102, & 103 assigned) then the receiver would send back an acknowledgement of 104 since that is the next byte it expects to receive in the next packet. By sending an acknowledgement of 104, the receiver is signaling that it received bytes 100, 101, 102, & 103 correctly. If, by some chance, the last two bytes were corrupted then an acknowledgement value of 102 would be sent since 100 & 101 were received successfully.
This would not happen for a packet of 4 bytes but it can happen if, for example, 10,000 bytes are sent in 10 different TCP packets and a packet is lost during transmission. If the first packet is lost then the sender would have to resend all 10,000 bytes since the acknowledgement cannot say that it received bytes 1,000 to 10,000 but only that it expects byte 0 because 0 through 999 were lost. (This issue is addressed in SCTP by adding a selective acknowledgement.)
Sequence numbers and acknowledgments cover discarding duplicate packets, retransmission of lost packets, and ordered-data transfer. To assure correctness a checksum field is included (see #Packet structure for details on checksumming).
The TCP checksum is a quite weak check by modern standards. Data Link Layers with a high probability of bit error rates may require additional link error correction/detection capabilities. If TCP were to be redesigned today, it would most probably have a 32-bit cyclic redundancy check specified as an error check instead of the current checksum. The weak checksum is partially compensated for by the common use of a CRC or better integrity check at layer 2, below both TCP and IP, such as is used in PPP or the Ethernet frame. However, this does not mean that the 16-bit TCP checksum is redundant: remarkably, surveys of Internet traffic have shown that software and hardware errors that introduce errors in packets between CRC-protected hops are common, and that the end-to-end 16-bit TCP checksum catches most of these simple errors. This is the end-to-end principle at work.
[edit] Congestion avoidance
The final part to TCP is congestion throttling. Acknowledgements for data sent, or lack of acknowledgements, are used by senders to implicitly interpret network conditions between the TCP sender and receiver. Coupled with timers, TCP senders and receivers can alter the behavior of the flow of data. This is more generally referred to as flow control, congestion control and/or network congestion avoidance. TCP uses a number of mechanisms to achieve high performance and avoid congesting the network (i.e., send data faster than either the network, or the host on the other end, can utilize it). These mechanisms include the use of a sliding window, the slow-start algorithm, the congestion avoidance algorithm, the fast retransmit and fast recovery algorithms, and more.
Enhancing TCP to reliably handle loss, minimize errors, manage congestion and go fast in very high-speed environments are ongoing areas of research and standards development.
[edit] TCP window size
TCP sequence numbers and windows behave very much like a clock. The window, whose width (in bytes) is defined by the receiving host, shifts each time it receives and acks a segment of data. Once it runs out of sequence numbers, it loops back to 0.The TCP receive window size is the amount of received data (in bytes) that can be buffered during a connection. The sending host can send only that amount of data before it must wait for an acknowledgment and window update from the receiving host.
[edit] Window scaling
For more efficient use of high bandwidth networks, a larger TCP window size may be used. The TCP window size field controls the flow of data and is limited to 2 bytes, or a window size of 65,535 bytes.
Since the size field cannot be expanded, a scaling factor is used. The TCP window scale option, as defined in RFC 1323, is an option used to increase the maximum window size from 65,535 bytes to 1 Gigabyte. Scaling up to larger window sizes is a part of what is necessary for TCP Tuning.
The window scale option is used only during the TCP 3-way handshake. The window scale value represents the number of bits to left-shift the 16-bit window size field. The window scale value can be set from 0 (no shift) to 14.
[edit] Connection termination
The connection termination phase uses, at most, a four-way handshake, with each side of the connection terminating independently. When an endpoint wishes to stop its half of the connection, it transmits a FIN packet, which the other end acknowledges with an ACK. Therefore, a typical teardown requires a pair of FIN and ACK segments from each TCP endpoint.
A connection can be "half-open", in which case one side has terminated its end, but the other has not. The side that has terminated can no longer send any data into the connection, but the other side can.
It is also possible for a 3-way handshake when host A sends a FIN and host B replies with a FIN & ACK (merely combines 2 steps into one) and host A replies with an ACK. This is perhaps the most common method.
Finally, it is possible for both hosts to send FINs simultaneously then both just have to ACK. This could possibly be considered a 2-way handshake since the FIN/ACK sequence is done in parallel for both directions.
[edit] TCP ports
TCP uses the notion of port numbers to identify sending and receiving application end-points on a host, or Internet sockets. Each side of a TCP connection has an associated 16-bit unsigned port number (1-65535) reserved by the sending or receiving application. Arriving TCP data packets are identified as belonging to a specific TCP connection by its sockets, that is, the combination of source host address, source port, destination host address, and destination port. This means that a server computer can provide several clients with several services simultaneously, as long as a client takes care of initiating any simultaneous connections to one destination port from different source ports.
Port numbers are categorized into three basic categories: well-known, registered, and dynamic/private. The well-known ports are assigned by the Internet Assigned Numbers Authority (IANA) and are typically used by system-level or root processes. Well-known applications running as servers and passively listening for connections typically use these ports. Some examples include: FTP (21), TELNET (23), SMTP (25) and HTTP (80). Registered ports are typically used by end user applications as ephemeral source ports when contacting servers, but they can also identify named services that have been registered by a third party. Dynamic/private ports can also be used by end user applications, but are less commonly so. Dynamic/private ports do not contain any meaning outside of any particular TCP connection.
[edit] Development of TCP
TCP is both a complex and evolving protocol. However, while significant enhancements have been made and proposed over the years, its most basic operation has not changed significantly since RFC 793, published in 1981. RFC 1122, Host Requirements for Internet Hosts, clarified a number of TCP protocol implementation requirements. RFC 2581, TCP Congestion Control, one of the most important TCP related RFCs in recent years, describes updated algorithms to be used in order to avoid undue congestion. In 2001, RFC 3168 was written to describe explicit congestion notification (ECN), a congestion avoidance signalling mechanism. In the early 21st century, TCP is typically used in approximately 95% of all Internet packets [citation needed]. Common applications that use TCP include HTTP (World Wide Web), SMTP (e-mail) and FTP (file transfer).
The original TCP congestion control was called TCP Reno, but recently, several alternative congestion control algorithms have been proposed:
High Speed TCP proposed by Sally Floyd in RFC 3649
TCP Vegas by Brakmo and Peterson at University of Arizona
TCP Westwood by UCLA
BIC TCP by Injong Rhee at North Carolina State University
H-TCP by Hamilton Institute
Fast TCP (Fast Active queue management Scalable Transmission Control Protocol) by Caltech.
TCP Hybla by University of Bologna
An extension mechanism TCP Interactive (iTCP) allows applications to subscribe to TCP events and respond accordingly enabling various functional extensions to TCP including application assisted congestion control.
[edit] TCP Over Wireless
TCP has been optimized for wired networks. Any packet loss is considered to be the result of congestion and the window size is reduced dramatically as a precaution. However, wireless links are known to experience sporadic and usually temporary losses due to fading, shadowing, handoff etc. that cannot be considered congestion. Erroneous back-off of the window size due to wireless packet loss is followed by a congestion avoidance phase with a conservative decrease in window size which causes the radio link to be underutilized. Extensive research has been done on this subject on how to combat these harmful effects. Suggested solutions can be categorized as end-to-end solutions (which require modifications at the client and/or server), link layer solutions (such as RLP in CDMA2000), or proxy based solutions (which require some changes in the network without modifying end nodes).
[edit] Hardware TCP Implementations
[edit] TCP Offload Engines
One way to overcome the processing power requirements of TCP is building hardware implementations of it, widely known as TCP Offload Engines (TOE). The main problem of TOEs is that they are hard to integrate into computing systems, requiring extensive changes in the operating system of the computer or device. The first company to develop such a device was Alacritech.
[edit] Debugging TCP
A packet sniffer, which intercepts TCP traffic on a network link, can be useful in debugging networks, network stacks and applications which use TCP by showing the user what packets are passing through a link.
[edit] Alternatives to TCP
For many applications TCP is not appropriate. One big problem (at least with normal implementations) is that the application cannot get at the packets coming after a lost packet until the retransmitted copy of the lost packet is received. This causes problems for real-time applications such as streaming multimedia (such as Internet radio), real-time multiplayer games and voice over IP (VoIP) where it is sometimes more useful to get most of the data in a timely fashion than it is to get all of the data in order.
Also for embedded systems, network booting and servers that serve simple requests from huge numbers of clients (e.g. DNS servers) the complexity of TCP can be a problem. Finally some tricks such as transmitting data between two hosts that are both behind NAT (using STUN or similar systems) are far simpler without a relatively complex protocol like TCP in the way.
Generally where TCP is unsuitable the User Datagram Protocol (UDP) is used. This provides the application multiplexing and checksums that TCP does, but does not handle building streams or retransmission giving the application developer the ability to code those in a way suitable for the situation and/or to replace them with other methods like forward error correction or interpolation.
SCTP is another IP protocol that provides reliable stream oriented services not so dissimilar from TCP. It is newer and considerably more complex than TCP so has not yet seen widespread deployment, however it is especially designed to be used in situations where reliability and near-real-time considerations are important.
TCP also has some issues in high bandwidth utilization environments. The TCP congestion avoidance algorithm works very well for ad-hoc environments where it is not known who will be sending data, but if the environment is predictable a timing based protocol such as ATM can avoid the overhead of retransmits that TCP needs.
[edit] Packet structure
A TCP packet consists of two sections:
header
data
The header consists of 11 fields, of which only 10 are required. The eleventh field is optional (pink background in table) and aptly named: options.
[edit] Header
+ Bits 0–3 4–9 10–15 16–31
0 Source Port Destination Port
32 Sequence Number
64 Acknowledgment Number
96 Data Offset Reserved Flags Window
128 Checksum Urgent Pointer
160 Options (optional)
160/192+
Data
Source port
This field identifies the sending port.
Destination port
This field identifies the receiving port.
Sequence number
The sequence number has a dual role. If the SYN flag is present then this is the initial sequence number and the first data byte is the sequence number plus 1. Otherwise if the SYN flag is not present then the first data byte is the sequence number.
Acknowledgement number
If the ACK flag is set then the value of this field is the sequence number the sender expects next.
Data offset
This 4-bit field specifies the size of the TCP header in 32-bit words. The minimum size header is 5 words and the maximum is 15 words thus giving the minimum size of 20 bytes and maximum of 60 bytes. This field gets its name from the fact that it is also the offset from the start of the TCP packet to the data.
Reserved
6-bit reserved field for future use and should be set to zero.
Flags (aka Control bits)
This field contains 6 bit flags:
URG
Urgent pointer field is significant
ACK
Acknowledgement field is significant
PSH
Push function
RST
Reset the connection
SYN
Synchronize sequence numbers
FIN
No more data from sender
Window
The number of bytes the sender is willing to receive starting from the acknowledgement field value
Checksum
The 16-bit checksum field is used for error-checking of the header and data.
With IPv4
When TCP runs over IPv4, the method used to compute the checksum is defined in RFC 793:
The checksum field is the 16 bit one's complement of the one's complement sum of all 16-bit words in the header and text. If a segment contains an odd number of header and text octets to be checksummed, the last octet is padded on the right with zeros to form a 16-bit word for checksum purposes. The pad is not transmitted as part of the segment. While computing the checksum, the checksum field itself is replaced with zeros.
In other words, all 16-bit words are summed together using one's complement (with the checksum field set to zero). The sum is then one's complemented. This final value is then inserted as the checksum field. Algorithmically speaking, this is the same as for IPv4.
The difference is in the data used to make the checksum. Included is a pseudo-header that mimics the IPv4 header:
+ Bits 0–3 4–7 8–9 10–15 16–31
0 Source address
32 Destination address
64 Zeros Protocol TCP length
96 Source Port Destination Port
128 Sequence Number
160 Acknowledgement Number
192 Data Offset Reserved Flags Window
224 Checksum Urgent Pointer
256 Options (optional)
256/288+
Data
The source and destination addresses are those in the IPv4 header. The protocol is that for TCP (see List of IPv4 protocol numbers): 6. The TCP length field is the length of the TCP header and data.
With IPv6
When TCP runs over IPv6, the method used to compute the checksum is changed, as per RFC 2460:
Any transport or other upper-layer protocol that includes the addresses from the IP header in its checksum computation must be modified for use over IPv6, to include the 128-bit IPv6 addresses instead of 32-bit IPv4 addresses.
When computing the checksum, a pseudo-header that mimics the IPv6 header is included:
+ Bits 0 - 7 8–15 16–23 24–31
0 Source address
32
64
96
128 Destination address
160
192
256
288 TCP length
320 Zeros Next Header
352 Source Port Destination Port
384 Sequence Number
416 Acknowledgement Number
448 Data Offset Reserved Flags Window
480 Checksum Urgent Pointer
512 Options (optional)
512/544+
Data
The source address is the one in the IPv6 header. The destination address is the final destination; if the IPv6 packet doesn't contain a Routing header, that will be the destination address in the IPv6 header, otherwise, at the originating node, it will be the address in the last element of the Routing header, and, at the receiving node, it will be the destination address in the IPv6 header. The Next Header value is the protocol value for TCP: 6. The TCP length field is the length of the TCP header and data.
Urgent pointer
If the URG flag is set, then this 16-bit field is an offset from the sequence number indicating the last urgent data byte.
Options
Additional header fields (called options) may follow the urgent pointer. If any options are present then the total length of the option field must be a multiple of a 32-bit word and the data offset field adjusted appropriately.
[edit] Data
The last field is not a part of the header. The contents of this field are whatever the upper layer protocol wants but this protocol is not set in the header and is presumed based on the port selection.
[edit] See also
TCP congestion avoidance algorithms for more on TCP Reno, TCP Vegas, TCP Westwood, BIC TCP and Hybla
TCP and UDP port
TCP and UDP port numbers for a complete (growing) list of ports/services
Connection-oriented protocol
TCP Tuning for high performance networks
T/TCP variant of TCP
Path MTU discovery
TCP Sequence Prediction Attack
SYN flood
UDP:
The Internet makes two transport protocols available to its applications, UDP and TCP. In this section we take a close look at UDP: how it works and what it does. The reader is encouraged to refer back to material in Section 2.1, which includes an overview of the UDP service model, and to the material in Section 2.7, which discusses socket programming over UDP.
To motivate our discussion about UDP, suppose you were interested in designing a no-frills, bare-bones transport protocol. How might you go about doing this? You might first consider using a vacuous transport protocol. In particular, on the sending side, you might consider taking the messages from the application process and passing them directly to the network layer; and on the receiving side, you might consider taking the messages arriving from the network layer and passing them directly to the application process. But as we learned in the previous section, we have to do a little more than nothing. At the very least, the transport layer must provide a multiplexing/demultiplexing service in order to pass data between the network layer and the correct application.
UDP, defined in [RFC 768], does just about as little as a transport protocol can. Aside from the multiplexing/demultiplexing function and some light error checking, it adds nothing to IP. In fact, if the application developer chooses UDP instead of TCP, then the application is talking almost directly with IP. UDP takes messages from application process, attaches source and destination port number fields for the multiplexing/demultiplexing service, adds two other fields of minor importance, and passes the resulting "segment" to the network layer. The network layer encapsulates the segment into an IP datagram and then makes a best-effort attempt to deliver the segment to the receiving host. If the segment arrives at the receiving host, UDP uses the port numbers and the IP source and destination addresses to deliver the data in the segment to the correct application process. Note that with UDP there is no handshaking between sending and receiving transport-layer entities before sending a segment. For this reason, UDP is said to be connectionless.
DNS is an example of an application-layer protocol that uses UDP. When the DNS application (see section 2.5) in a host wants to make a query, it constructs a DNS query message and passes the message to a UDP socket (see Section 2.7). Without performing any handshaking, UDP adds a header fields to the message and passes the resulting segment to the network layer. The network layer encapsulates the UDP segment into a datagram and sends the datagram to a name server. The DNS application at the querying host then waits for a reply to its query. If it doesn't receive a reply (possibly because UDP lost the query or the reply), it either tries sending the query to another nameserver, or it informs the invoking application that it can't get a reply. We mention that the DNS specification permits DNS to run over TCP instead of UDP; in practice, however, DNS almost always runs over UDP.
Now you might be wondering why an application developer would ever choose to build an application over UDP rather than over TCP. Isn't TCP always preferable to UDP since TCP provides a reliable data transfer service and UDP does not? The answer is no, as many applications are better suited for UDP for the following reasons:
No connection establishment. As we shall discuss in Section 3.5, TCP uses a three-way handshake before it starts to transfer data. UDP just blasts away without any formal preliminaries. Thus UDP does not introduce any delay to establish a connection. This is probably the principle reason why DNS runs over UDP rather than TCP -- DNS would be much slower if it ran over TCP. HTTP uses TCP rather than UDP, since reliability is critical for Web pages with text. But, as we briefly discussed in Section 2.2, the TCP connection establishment delay in HTTP is an important contributor to the "world wide wait".
No connection state. TCP maintains connection state in the end systems. This connection state includes receive and send buffers, congestion control parameters, and sequence and acknowledgment number parameters. We will see in Section 3.5 that this state information is needed to implement TCP's reliable data transfer service and to provide congestion control. UDP, on the other hand, does not maintain connection state and does not track any of these parameters. For this reason, a server devoted to a particular application can typically support many more active clients when the application runs over UDP rather than TCP.
Small segment header overhead. The TCP segment has 20 bytes of header overhead in every segment, whereas UDP only has 8 bytes of overhead.
Unregulated send rate. TCP has a congestion control mechanism that throttles the sender when one or more links between sender and receiver becomes excessively congested. This throttling can have a severe impact on real-time applications, which can tolerate some packet loss but require a minimum send rate. On the other hand, the speed at which UDP sends data is only constrained by the rate at which the application generates data, the capabilities of the source (CPU, clock rate, etc.) and the access bandwidth to the Internet. We should keep in mind, however, that the receiving host does not necessarily receive all the data - when the network is congested, a significant fraction of the UDP-transmitted data could be lost due to router buffer overflow. Thus, the receive rate is limited by network congestion even if the sending rate is not constrained.
Table 3.1-1 lists popular Internet applications and the transport protocols that they use. As we expect, e-mail, remote terminal access, the Web and file transfer run over TCP -- these applications need the reliable data transfer service of TCP. Nevertheless, many important applications run over UDP rather TCP. UDP is used for RIP routing table updates (see Chapter 4 on the network layer), because the updates are sent periodically, so that lost updates are replaced by more up-to-date updates. UDP is used to carry network management (SNMP - see Chapter 8) data. UDP is preferred to TCP in this case, since network management must often run when the network is in a stressed state - precisely when reliable, congestion-controlled data transfer is difficult to achieve. Also, as we mentioned earlier, DNS runs over UDP, thereby avoiding TCP's connection establishment delays.
Application Application-layer protocol Underlying Transport Protocol
electronic mail SMTP TCP
remote terminal access Telnet TCP
Web HTTP TCP
file transfer FTP TCP
remote file server NFS typically UDP
streaming multimedia proprietary typically UDP
Internet telephony proprietary typically UDP
Network Management SNMP typically UDP
Routing Protocol RIP typically UDP
Name Translation DNS typically UDP
Figure 3.1-1: Popular Internet applications and their underlying transport protocols.
As shown in Figure 3.1-1, UDP is also commonly used today with multimedia applications, such as Internet phone, real-time video conferencing, and streaming of stored audio and video. We shall take a close look at these applications in Chapter 6. We just mention now that all of these applications can tolerate a small fraction of packet loss, so that reliable data transfer is not absolutely critical for the success of the application. Furthermore, interactive real-time applications, such as Internet phone and video conferencing, react very poorly to TCP's congestion control. For these reasons, developers of multimedia applications often choose to run the applications over UDP instead of TCP. Finally, because TCP cannot be employed with multicast, multicast applications run over UDP.
Although commonly done today, running multimedia applications over UDP is controversial to say the least. As we mentioned above, UDP lacks any form of congestion control. But congestion control is needed to prevent the network from entering a congested state in which very little useful work is done. If everyone were to start streaming high bit-rate video without using any congestion control, there would be so much packet overflow at routers that no one would see anything. Thus, the lack of congestion control in UDP is a potentially serious problem. Many researchers have proposed new mechanisms to force all sources, including UDP sources, to perform adaptive congestion control [Mahdavi].
Before discussing the UDP segment structure, we mention that it is possible for an application to have reliable data transfer when using UDP. This can be done if reliability is built into the application itself (e.g., by adding acknowledgement and retransmission mechanisms, such as those we shall study in the next section). But this a non-trivial task that would keep an application developer busy debugging for a long time. Nevertheless, building reliability directly into the application allows the application to "have its cake and eat it too" -- that is, application processes can communicate reliably without being constrained by the transmission rate constraints imposed by TCP's congestion control mechanism. Application-level reliability also allows an application to tailor its own application-specific form of error control. An interactive real-time may occasionally choose to retransmit a lost message, provided that round trip network delays are small enough to avoid adding significant playout delays [Papadopoulos 1996].
Many of today's proprietary streaming applications do just this -- they run over UDP, but they have built acknowledgements and retransmissions into the application in order reduce packet loss.
UDP Segment Structure
The UDP segment structure, shown in Figure 3.3-2, is defined in [RFC 768].
Figure 3.3-2: UDP segment structure
The application data occupies the data field of the UDP datagram. For example, for DNS, the data field contains either a query message or a response message. For a streaming audio application, audio samples fill the data field. The UDP header has only four fields, each consisting of four bytes. As discussed in the previous section, the port numbers allow the destination host to pass the application data to the correct process running on that host (i.e., perform the demultiplexing function). The checksum is used by the receiving host to check if errors have been introduced into the segment during the course of its transmission from source to destination. (Basic principles of error detection are described in Section 5.2.).
UDP Checksum
The UDP checksum provides for error detection. UDP at the sender side performs the one's complement of the sum of all the 16-bit words in the segment. This result is put in the checksum field of the UDP segment. (In truth, the checksum is also calculated over a few of the fields in the IP header in addition to the UDP segment. But we ignore this detail in order to see the forest through the trees.) When the segment arrives (if it arrives!) at the receiving host, all 16-bit words are added together, including the checksum. If this sum equals 1111111111111111, then the segment has no detected errors. If one of the bits is a zero, then we know that errors have been introduced into the segment.
Here we give a simple example of the checksum calculation. You can find details about efficient implementation of the calculation in the [RFC 1071]. As an example, suppose that we have the following three 16-bit words:
0110011001100110
0101010101010101
0000111100001111
The sum of first of these 16-bit words is:
0110011001100110
0101010101010101
---------------------
1011101110111011
Adding the third word to the above sum gives
1011101110111011
0000111100001111
---------------------
1100101011001010
The 1's complement is obtained by converting all the 0s to 1s and converting all the 1s to 0s. Thus the 1's complement of the sum 1100101011001010 is 0011010100110101, which becomes the checksum. At the receiver, all four 16-bit words are added, including the checksum. If no errors are introduced into the segment, then clearly the sum at the receiver will be 1111111111111111. If one of the bits is a zero, then we know that errors have been introduced into the segment. In section 5.1, we'll see that the Internet checksum is not foolproof -- even if the sum equals 111111111111111, it is still possible that there are undetected errors in the segment. For this reason, a number of protocols use more sophisticated error detection techniques than simple checksumming.
You may wonder why UDP provides a checksum in the first place, as many link-layer protocols (including the popular Ethernet protocol) also provide error checking? The reason is that there is no guarantee that all the links between source and destination provide error checking -- one of the links may use a protocol that does not provide error checking. Because IP is supposed to run over just about any layer-2 protocol, it is useful for the transport layer to provide error checking as a safety measure. Although UDP provides error checking, it does not do anything to recover from an error. Some implementations of UDP simply discard the damaged segment; others pass the damaged segment to the application with a warning.
That wraps up our discussion of UDP. We will soon see that TCP offers reliable data transfer to its applications as well as other services that UDP doesn't offer. Naturally, TCP is also more complex than UDP. Before discussing TCP, however, it will be useful to step back and first discuss the underlying principles of reliable data transfer, which we do in the subsequent section. We will then explore TCP in Section 3.5, where we will see that TCP has it foundations in these underlying principles.
References
[Papadopoulos 1996] C. Papadopoulos and G. Parulkar, "Retransmission-Based Error Control for Continuous Media Applications," Proceedings of the 6th International Workshop on Network and Operating System Support for Digital Audio and Video (NOSSDAV), April 1996.
[Mahdavi] J. Mahdavi and S. Floyd, "The TCP-Friendly Website," http://www.psc.edu/networking/tcp_friendly.html
[RFC 768] J.Postel, "User Datagram Protocol," RFC 768, August 1980.
[RFC 1071] R. Braden, D. Borman, C. Partridge, "Computing The Internet Checksum," RFC 1071, September 1988.
Search RFCs and Internet Drafts
--------------------------------------------------------------------------------
If you are interested in an RFC relating to a certain subject or protocol, enter the keyword(s) here:
--------------------------------------------------------------------------------
If you are interested in an Internet Draft relating to a certain subject or protocol enter the keyword(s) here.
Query:
Press button to submit your query or reset the form:
Query Options:
Case insensitive
Maximum number of hits: 10 25 50 75 100 500 1000
____________
Port:
1) On computer and telecommunication devices, a port (noun) is generally a specific place for being physically connected to some other device, usually with a socket and plug of some kind. Typically, a personal computer is provided with one or more serial ports and usually one parallel port. The serial port supports sequential, one bit-at-a-time transmission to peripheral devices such as scanners and the parallel port supports multiple-bit-at-a-time transmission to devices such as printers.
2) In programming, a port (noun) is a "logical connection place" and specifically, using the Internet's protocol, TCP/IP, the way a client program specifies a particular server program on a computer in a network. Higher-level applications that use TCP/IP such as the Web protocol, Hypertext Transfer Protocol, have ports with preassigned numbers. These are known as "well-known ports" that have been assigned by the Internet Assigned Numbers Authority (IANA). Other application processes are given port numbers dynamically for each connection. When a service (server program) initially is started, it is said to bind to its designated port number. As any client program wants to use that server, it also must request to bind to the designated port number.
Port numbers are from 0 to 65536. Ports 0 to 1024 are reserved for use by certain privileged services. For the HTTP service, port 80 is defined as a default and it does not have to be specified in the Uniform Resource Locator (URL).
3) In programming, to port (verb) is to move an application program from an operating system environment in which it was developed to another operating system environment so it can be run there. Porting implies some work, but not nearly as much as redeveloping the program in the new environment. open standard programming interface (such as those specified in X/Open's 1170 C language specification and Sun Microsystem's Java programming language) minimize or eliminate the work required to port a program. Also see portability
serial
- Serial means one event at a time. It is usually contrasted with parallel, meaning more than one event happening at a time. In data transmission, the techniques of time division and space division are used, where time separates the transmission of individual bits of information sent serially and space (on multiple lines or paths) can be used to have multiple bits sent in parallel.
In the context of computer hardware and data transmission: serial connection, operation, and media usually indicate a simpler, slower operation and parallel indicates a faster operation. This indication doesn't always hold since a serial medium (for example, fiber optic cable) can be much faster than a slower medium that carries multiple signals in parallel.
On your PC, the printer is usually attached through a parallel interface and cable so that it will print faster. Your keyboard and mouse are one-way devices that only require a serial interface and line. Inside your computer, much of its circuitry supports bits being moved around in parallel.
Your computer modem uses one of your PC's serial connections or COM ports. Serial communication between your PC and the modem and other serial devices adheres to the RS-232C standard.
Conventional computers and their programs operate in a serial manner, with the computer reading a program and performing its instructions one after the other. However, some of today's computers have multiple processors and can perform instructions in parallel.
Parallel
- In the context of the Internet and computing, parallel means more than one event happening at a time. It is usually contrasted with serial, meaning only one event happening at a time. In data transmission, the techniques of time division and space division are used, where time separates the transmission of individual bits of information sent serially and space (in multiple lines or paths) can be used to have multiple bits sent in parallel.
In the context of computer hardware and data transmission, serial connection, operation, and media usually indicate a simpler, slower operation (think of your serial mouse attachment). Parallel connection and operation (think of multiple characters being sent to your printer) indicates faster operation. This indication doesn't always hold since a serial medium (for example, fiber optic cable) can be much faster than a slower medium that carries multiple signals in parallel.
A conventional phone connection is generally thought of as a serial line since its usual transmission protocol is serial.
Conventional computers and their programs operate in a serial manner, with the computer reading a program and performing its instructions one after the other. However, some of today's computers have multiple processors that divide up the instructions and perform them in parallel.
__________
Proxy:A proxy server is a computer that offers a computer network service to allow clients to make indirect network connections to other network services. A client connects to the proxy server, then requests a connection, file, or other resource available on a different server. The proxy provides the resource either by connecting to the specified server or by serving it from a cache. In some cases, the proxy may alter the client's request or the server's response for various purposes.
Types
Web
A common proxy application is a caching web proxy. This provides a nearby cache of web pages and files available on remote web servers, allowing local network clients to access documents from the cache, according to their age, size, and access history. Two simple cache algorithms are Least Recently Used (LRU) and Least Frequently Used (LFU). LRU removes the documents that have been left the longest, while LFU removes the least popular documents. The algorithms can also be combined.
Some censorware applications — which attempt to block offensive web content — are implemented as web proxies. Other web proxies reformat web pages for a specific purpose or audience; for example, Skweezer reformats web pages for cell phones and PDAs. Network operators can also deploy proxies to intercept computer viruses and other hostile content served from remote web pages.
A special case of web proxies are "CGI proxies." These are web sites which allow a user to access a site through them. They generally use PHP or CGI to implement the proxying functionality. CGI proxies are frequently used to gain access to web sites blocked by corporate or school proxies. Since they also hide the user's own IP address from the web sites they access through the proxy, they are sometimes also used to gain a degree of anonymity, called "Proxy Avoidance."
SSL
A growing subset of Web traffic uses HTTPS to create an encrypted tunnel. Regulations often require all Intranet and ASP applications handling sensitive data to use encryption. Traditional (HTTP) proxies can not cache encrypted traffic, hence provide no acceleration, control or bandwidth savings for SSL applications.
Some HTTPS Proxies are specifically designed to operate in the middle of a client-server communication to decrypt, apply policy, cache and re-encrypt SSL traffic. As a result, these HTTPS Proxies can monitor, control and accelerate SSL traffic. These HTTPS Proxies can also apply malware scanning and content filtering to inhibit phishing, spyware and computer viruses hiding inside encrypted tunnels.
There are privacy concerns with SSL proxies. In essence, the IT department is conducting a Man-in-the-middle attack, potentially exposing sensitive corporate information, personal online banking information, etc. Countries such as Sweden have very strong privacy laws, which may require the proxy to handle corporate information with different policy than personal traffic.In Sweden, privacy laws apply to non-SSL proxies too. Personuppgiftslagen (PUL, a Swedish privacy law) doesn't differ between encrypted and unencrypted data. But as long as no details about the identity of the user is stored or collected, PUL doesn't apply, and unrestricted scanning of SSL information is allowed.
Content filtering can differentiate this traffic. Additional compromise policy options include caching GIF and JPEG objects (which typically are user interface elements and lack confidential data), but exclude from caching HTML and TXT traffic.
Sometimes the term "SSL proxy" also refers to CGI web proxies that are accessible via encrypted SSL connections. In this case, SSL adds an extra layer of security on top of the CGI proxy system, lessening the chance of data interception.
Intercepting
Many organizations — including corporations, schools, and families — use a proxy server to enforce acceptable network use policies (see censorware) or to provide security, anti-malware and/or caching services. A traditional web proxy is not transparent to the client application, which must be configured to use the proxy (manually or with a configuration script). In some cases, where alternative means of connection to the Internet are available (e.g. a SOCKS server or NAT connection), the user may be able to avoid policy control by simply resetting the client configuration and bypassing the proxy. Furthermore administration of browser configuration can be a burden for network administrators.
An intercepting proxy, often incorrectly called transparent proxy (also known as a forced proxy) combines a proxy server with NAT. Connections made by client browsers through the NAT are intercepted and redirected to the proxy without client-side configuration (or often knowledge).
Intercepting proxies are commonly used in businesses to prevent avoidance of acceptable use policy, and to ease administrative burden, since no client browser configuration is required.
Intercepting proxies are also commonly used by Internet Service Providers in many countries in order to reduce upstream link bandwidth requirements by providing a shared cache to their customers.
It is often possible to detect the use of an intercepting proxy server by comparing the external IP address to the address seen by an external web server, or by examining the HTTP headers on the server side.
Some poorly implemented intercepting proxies have historically had certain downsides, e.g. an inability to use user authentication if the proxy does not recognise that the browser was not intending to talk to a proxy. Some problems are described in RFC 3143. A well-implemented proxy however should not inhibit browser authentication at all.
The term transparent proxy, which is often incorrectly used instead of intercepting proxy to describe the same behavior, is defined in RFC 2616 (which defines the HTTP/1.1 protocol) this way:
A "transparent proxy" is a proxy that does not modify the request or response beyond what is required for proxy authentication and identification.
and when talking about HTTP proxy, terminology defined there should be used.
Open
An open proxy is a proxy server which will accept client connections from any IP address and make connections to any Internet resource. Abuse of open proxies is currently implicated in a significant portion of e-mail spam delivery. Spammers frequently install open proxies on unwitting end users' operating systems by means of computer viruses designed for this purpose. Internet Relay Chat (IRC) abusers also frequently use open proxies to cloak their identities.
Because proxies might be used for abuse, system administrators have developed a number of ways to refuse service to open proxies. IRC networks such as the Blitzed network automatically test client systems for known types of open proxy [1]. Likewise, an email server may be configured to automatically test e-mail senders for open proxies, using software such as Michael Tokarev's proxycheck [2].
Groups of IRC and electronic mail operators run DNSBLs publishing lists of the IP addresses of known open proxies, such as AHBL, CBL, NJABL, and SORBS.
The ethics of automatically testing clients for open proxies are controversial. Some experts, such as Vernon Schryver, consider such testing to be equivalent to an attacker portscanning the client host. [3] Others consider the client to have solicited the scan by connecting to a server whose terms of service include testing.
Reverse
A reverse proxy is a proxy server that is installed in the neighborhood of one or more web servers. All traffic coming from the Internet and with a destination of one of the web servers is going through the proxy server. There are several reasons for installing reverse proxy servers:
Security: the proxy server is an additional layer of defense and therefore protects the web servers further up the chain
Encryption / SSL acceleration: when secure websites are created, the SSL encryption is often not done by the web server itself, but by a reverse proxy that is equipped with SSL acceleration hardware. See Secure Sockets Layer.
Load balancing: the reverse proxy can distribute the load to several web servers, each web server serving its own application area. In such a case, the reverse proxy may need to rewrite the URLs in each web page (translation from externally known URLs to the internal locations)
Serve/cache static content: A reverse proxy can offload the web servers by caching static content like pictures and other static graphical content
Compression: the proxy server can optimize and compress the content to speed up the load time.
Spoon feeding: if a program is producing the web page on the web servers, the web servers can produce it, serve it to the reverse-proxy, which can spoon-feed it however slowly the clients need and then close the program rather than having to keep it open while the clients insist on being spoon fed.
Split
A split proxy is essentially a pair of proxies installed across two computers. Since they are effectively two parts of the same program, they can communicate with each other in a more efficient way than they can communicate with a more standard resource or tool such as a website or browser. This is ideal for compressing data over a slow link, such as a wireless or mobile data service and also for reducing the issues regarding high latency links (such as satellite internet) where establishing a TCP connection is time consuming. Taking the example of web browsing, the user's browser is pointed to a local proxy which then communicates with its other half at some remote location. This remote server fetches the requisite data, repackages it and sends it back to the user's local proxy, which unpacks the data and presents it to the browser in the standard fashion .
See Google's Web Accelerator
Circumventor
A circumventor is a web-based page that takes a site that is blocked and "circumvents" it through to an unblocked website, allowing the user to view blocked pages. A famous example is 'elgoog', which allowed users in China to use Google after it had been blocked there. Elgoog differs from most circumventors in that it circumvents only one block.
The most common use is in schools where many blocking programs block by site rather than by code; students are able to access blocked sites (games, chatrooms, messenger, pornography, weapons, racism, forbidden knowledge, etc.) through a circumventor. As fast as the filtering software blocks circumventors, others spring up. It should be noted, however, that in some cases the filter may still intercept traffic to the circumventor, thus the person who manages the filter can still see the sites that are being visited.
Circumventors are also used by people who have been blocked from a website.
Another use of a circumventor is to allow access to country-specific services, so that Internet users from other countries may also make use of them. An example is country-restricted reproduction of media and webcasting.
The use of circumventors is usually safe with the exception that circumventor sites run by an untrusted third party can be run with hidden intentions, such as collecting personal information, and as a result users are typically advised against running personal data such as credit card numbers or passwords through a circumventor.
Anonymous
In using a proxy server (for example, anonymizing HTTP proxy), all data sent to the service being used (for example, HTTP server in a website) must pass through the proxy server before being sent to the service, mostly in unencrypted form. It is therefore possible, and has been demonstrated (see, for example, Sugarcane) for a malicious proxy server to record everything sent to the proxy: including unencrypted logins and passwords.
By chaining proxies which do not reveal data about the original requester, it is possible to obfuscate activities from the eyes of the user's destination. However, more traces will be left on the intermediate hops, which could be used or offered up to trace the user's activities. If the policies and administrators of these other proxies are unknown, the user may fall victim to a false sense of security just because those details are out of sight and mind.
The bottom line of this is to be wary when using proxy servers, and only use proxy servers of known integrity (e.g., the owner is known and trusted, has a clear privacy policy, etc.), and never use proxy servers of unknown integrity. If there is no choice but to use unknown proxy servers, do not pass any private information (unless it is properly encrypted) through the proxy.
In what is more of an inconvenience than a risk, proxy users may find themselves being blocked from certain Web sites, as numerous forums and Web sites block IP addresses from proxies known to have spammed or trolled the site.
Popular proxy software
The Squid cache is a popular HTTP proxy server in the UNIX/Linux world.*
The HTTP-Tunnel is a popular SOCKS proxy server and Client for Windows.*
The Apache HTTP Server can be configured to act as a proxy server.
Blue Coat's (formerly Cacheflow's) purpose-built SGOS proxies 15 protocols including HTTPS/SSL, has an extensive policy engine and runs on a range of appliances from branch-office to enterprise.
WinGate is a multi-protocol proxy server and NAT solution that can be used to redirect any kind of traffic on a Microsoft Windows host. It also provides firewall, VPN and mail server functionality. Its WWW proxy supports integrated windows authentication, intercepting proxy, and multi-host reverse-proxying.
Privoxy is a free, open source web proxy with privacy features
Microsoft Internet Security and Acceleration Server is a product that runs on Windows 2000/2003 servers and combines the functions of both a proxy server and a firewall.
Tor - A proxy-based anonymizing Internet communication system
Proxomitron - User-configurable web proxy used to re-write web pages on the fly. Most noted for blocking ads, but has many other useful features.
PHProxy is a Web HTTP proxy programmed in PHP to bypass firewalls and other proxy restrictions through a Web interface very similar to the popular CGIProxy.
SJSWebProxy (SunMicrosystems) is a proxy server for HTTP and HTTPS (CONNECT) requests. It can also serve as a gateway for Ftp and Gopher traffic. It is also free for download.
Nginx Web and Reverse proxy server, that can act as POP3 proxy server.
Ssh Secure Shell can be configured to proxify a connection, by setting up a SOCKS proxy on the client, and tunneling the traffic through the SSH connection.
-------->Local host:
The IP address 127.0.0.1 is a reserved IP address specified in the Internet Protocol used to represent the host computer, whichever computer you happen to be sitting in front of with TCP/IP capabilities. This is often necessary as a background way for TCP/IP to route packets from the host to itself. But it is often used as a simple troubleshooting procedure. You can send data to and from yourself to test the installation of TCP/IP. Type "ping 127.0.0.1" from a command prompt. If you get no response, or "destination host unreachable", it's time to reinstall TCP/IP. If you get responses, they should show extremely low ping times, like 0.1ms or less. (TCP/IP may still need to be reinstalled, if it's not possible to ping elsewhere.)
--------->ip address:
An identifier for a computer or device on a TCP/IP network. Networks using the TCP/IP protocol route messages based on the IP address of the destination. The format of an IP address is a 32-bit numeric address written as four numbers separated by periods. Each number can be zero to 255. For example, 1.160.10.240 could be an IP address.
Within an isolated network, you can assign IP addresses at random as long as each one is unique. However, connecting a private network to the Internet requires using registered IP addresses (called Internet addresses) to avoid duplicates.
The four numbers in an IP address are used in different ways to identify a particular network and a host on that network. Four regional Internet registries -- ARIN, RIPE NCC, LACNIC and APNIC -- assign Internet addresses from the following three classes.
Class A - supports 16 million hosts on each of 126 networks
Class B - supports 65,000 hosts on each of 16,000 networks
Class C - supports 254 hosts on each of 2 million networks
The number of unassigned Internet addresses is running out, so a new classless scheme called CIDR is gradually replacing the system based on classes A, B, and C and is tied to adoption of IPv6