... {toc:type=flat}
h1. Measurements
bq. _Let me get this straight: you want me to troubleshoot a problem that happened yesterday you think may be related to performance and your production system is not collecting historical performance data?_
Automatic System Activity Reporting (SAR) via cron should be in place on all production, testing, development and staging systems. There is simply no way to examine comprehensive historical performance data if SAR data is not being collected at regular intervals, unless some other mechanism is in place. The impact on the operating system in terms of performance is minimal, barely measurable, and the disk space consumed by SAR is quite small when compared to the value of having historical performance data. Unfortunately, SAR does not collect network statistics. netstat is used for this purpose.
A properly configured staging or test environment is of great help in the accurate determination of TCP/IP parameters on Solaris. Using software load test tools can shorten the time it takes to determine appropriate values for operating system settings in general and TCP/IP settings in particular.
Sniffers should be in place on all production networks when problems are suspected. The Solaris utility snoop is useful in some situations, but sniffers are generally the best way to examine network activity at the packet level. Use netstat -s to examine the counters kept by the kernel. netstat is invasive to the extent that each invocation results in locking the IP perimeter resulting in processes waiting to retrieve or transmit data momentarily. There have been situations on extremely busy servers where a group of administrators ran so many netstats and pings that system performance was affected by the measurement process (There was once a production E10K that was running so many netstats, tops, and pings it was literally choked to death. Once administrators were convinced to stop running so many netstats,tops and pings the system became responsive again, and the SME was able to make an accurate performance diagnosis.)
h1. TCP/IP Parameters in Solaris
h2. Overview
TCP/IP parameters in Solaris are set in one of two ways: the /usr/sbin/ndd command, or by changing values in the /etc/system file. If the /etc/system file is changed, the system must be rebooted before the changes will take effect.
To set parameters using /usr/sbin/ndd, use the following syntax:
{noformat}
# /usr/sbin/ndd set driver parameter value
{noformat}
for example,
{noformat}
# /usr/sbin/ndd set /dev/ip ip_forwarding 0
{noformat}
h2. nddconfig
TCP/IP settings on Solaris that are set by ndd are best managed with a shell script called nddconfig which is available from Sun Blueprints online. Do not enter TCP/IP settings in /etc/rc2.d/S69inetinit because patches may overwrite this file causing changes to be lost. Install nddconfig in /etc/init.d, change it to mode 0744, and install a symbolic link to it called /etc/rc2.d/S71nddconfig.
h2. Parameters
h3. ip_ignore_redirect
The Internet Control Message Protocol (ICMP) contains control messages and error messages for IP. One of the possible types of ICMP messages is a redirect. ICMP redirects cause a host to re-route packets and alter the host routing table. This mechanism may be part of a sophisticated DoS attack.
h4. General Guidelines for ip_ignore_redirect
If the administrator suspects a redirect DoS attack, set ip_ignore_redirect to 1 . The default value of ip_ignore_redirect is 0 .11 IP redirects should not be necessary in a well-designed, well-maintained network.
h3. tcp_slow_start_initial
{quote}
_The TCP specification requires an algorithm known as "slow start". The algorithm applies to the sender side and is described in RFC2001. The intent of the slow start algorithm is to avoid a "congestion collapse" in a network by ensuring that each TCP sender doesn't overwhelm the network. The algorithm mandates that the first transmission be a single packet. If the recipient acknowledges the first packet successfully (i.e. the communication doesn't time out and the recipient believes that the packet has arrived without error), the sender sends two more packets. Successful transmission results in the sender sending yet more packets in parallel, until the capability of the underlying network is reached and one or more packets are not acknowledged successfully. Essentially the sender uses ACKs as a "clock" to regulate and gradually increase the rate packets are injected into the network until it reaches an equilibrium._
_The TCP specification describes another technique known as "delayed ACK", which concerns the receive side. The technique calls for an acknowledgement of a data packet to be delayed for a short period of time - the delayed-ACK interval. Different TCP implementations use different delay intervals. The TCP specification (RFC1122) mandates that the delayed-ACK interval must be less than 0.5 second. Delayed ACK serves to give the application an opportunity to send an immediate response, in which case the ACK can be piggyback'ed with the packet carrying the response. This technique is very useful, both in saving the network bandwidth and in reducing the protocol processing overhead, and is widely adopted by TCP implementations. The TCP standard also recommend that an ACK not to be delayed for more than two data packets. This is to keep the slow start algorithm on the sender side flowing, which counts on the ACK packets coming back from the receive side in order to strobe more data packets into the network._
{quote}
Windows clients at the time of this writing have a simple-minded delayed-ACK mechanism which causes hesitation upon creation of a new TCP.
h3. ip_forward_src_routed
IP provides a way to route a packet via information provided in the IP header as opposed to routing tables contained the host or router. Typically, routers and routes define the path that packets follow to reach their destination. There are two types of source routing available, strict and loose. Using strict source routing the sender defines every intermediate hop between source and destination, with loose source routing the sender only defines a few intermediate hops.
bq. _The loose source and record route (LSRR) option provides a means for the source of an internet datagram to supply routing information to be used by the gateways in forwarding the datagram to the destination, and to record the route information._
bq. _The strict source and record route (SSRR) option provides a means for the source of an internet datagram to supply routing information to be used by the gateways in forwarding the datagram to the destination, and to record the route information._
h4. General Guidelines for ip_forward_src_routed
Source routing circumvents security measures in a network and should be disabled on Solaris hosts that are being used as routers (see ip_forwarding). To disable the forwarding of source-routed packets, set ip_forward_src_routed to 0.
h3. ip_forward_directed_broadcasts
If this parameter is set to 1, Solaris forwards broadcast packets for a specific net or subnet if that net or subnet is directly connected to the machine. If Solaris is configured as a router (see ip_forwarding) broadcast attacks5 can generate excessive broadcast network traffic. The default value of ip_forward_directed_broadcasts is 1.
h4. General Guidelines for ip_forward_directed_broadcasts
An attacker forges ICMP packets by setting the host address in the source address field and sends these forged packets to the broadcast address of a remote network. All stations on the remote network respond to the forged source address, resulting in a broadcast flood . The default value of 1 (enabled) results in Solaris forwarding these packets. Setting ip_forward_directed_broadcasts to 0 prevents Solaris from forwarding directed broadcast packets and can alleviate the effects of broadcast attacks.6 Disabling ip_forward_directed_broadcasts disables broadcast pings.
h3. arp_cleanup_interval
The Address Resolution Protocol enables a host to determine the Ethernet address of a station given the station s IP address. If the host does not know the station s Ethernet address in other words, the address is not in the ARP cache the host sends an ARP message to all stations requesting the Ethernet address of the station with a given IP address. The returned entry is stored in the ARP cache. There are two types of ARP cache entries: solicited and unsolicited. Solicited entries are entered into the ARP cache because of requests by the host. Unsolicited entries come from other hosts that have issued ARP requests. arp_ cleanup_interval is the length of time, in milliseconds, that an unsolicited ARP entry remains in the ARP cache. Unsolicited ARP cache entries are discarded when IP does not request the entry before arp_cleanup_interval expires. The default entry in Solaris 8/9 is 300000 ms (5 minutes).
h4. General Guidelines for arp_cleanup_interval
ARP attacks may be effective if arp_cleanup_interval is set to a value allowing unsolicited entries to remain in the ARP cache for long periods3. An attacker could arrange to fill the ARP cache with bogus entries. On Solaris, the time-tolive of unsolicited entries is arp_cleanup_interval. Shorten the interval to reduce effectiveness of ARP attacks. Set arp_cleanup_interval to 60000 ms (1 minute). Static ARP table entries entered with /usr/sbin/arp's do not expire.
h3. tcp_time_wait_interval
This [article|tcp_time_wait_interval] will explain the parameter *tcp_time_wait_interval* and address a methodology to determine the proper way to determine values the parameter.
h1. External Links
|