TCP parameter tuning to avoid socket exception
Known issue description
The Mailbox Spider does not reuse sockets, which can lead to an error message like the following when running out of sockets:
message: Microsoft.Exchange.WebServices.Data.ServiceRequestException: The request failed.
Unable to connect to the remote server ---> System.Net.WebException:
Unable to connect to the remote server ---> System.Net.Sockets.SocketException:
Only one usage of each socket address (protocol/network address/port) is normally permitted
This can cause undesired delays with sockets being blocked for at least 4 minutes with follow-up errors like the following for the Batch Archiver:
The requested mail '...' from store '...' could not be found/opened. --->
System.Runtime.InteropServices.COMException: Error in IMsgStore.OpenEntry: MAPI_E_NOT_FOUND
The issue is related to the way Windows handles TCP connections. Since Windows Server 2008, by default there is a dynamic ports range 49152-65535 available for using. When a connection is closed, the port will be set in TIME_WAIT state for 240 seconds by default, therefore the port will be unavailable for new connections during that time.
Under excessive network load, a system may run out of sockets available. The following tips help to avoid the issue.
Increase number of available dynamic ports
Run the following commands as administrator:
This command will show the current range of dynamic ports (start port and number of ports):
Figure 379: netsh – show current dynamic port range
Change the port range with the netsh command like this:
The illustrated command changes the port range to 48000-65535 and thus adds 1000 more dynamic ports:
Note:
-
The lowest starting port that you can set is 1025.
-
The highest end port (based on the range you set) cannot exceed 65535.
Reduce TIME_WAIT state time for closed connections
By default the TIME_WAIT state time is 240 seconds. To change this, create the registry value:
Services\Tcpip\Parameters\TcpTimedWaitDelay (REG_DWORD)
Set a lower value in decimal or in HEX. (In general, any value in the range 30-300 seconds is supported.)