I have spent the past few days working on migrating a bunch of classic ASP web applications from a Windows 2000 server to a Windows 2003 server. A long overdue task, but one who’s time has finally come. The hardware under the Windows 2000 box has been acting up a bit lately and I’m rapidly losing confidence in it. Not to mention that Windows 2000 is getting closer to the end of support.
The good news is that most of the migrated sites worked right off the bat. The key word being “most.” In one of the apps we have some code that’s a bit out of the norm – deeply recursive with lots of SQL Server database IO. Every time we’d hit that code we would get an error after over a minute. And every error was on a different line of code:
Microsoft OLE DB Provider for ODBC Drivers error '80004005'
[Microsoft][ODBC SQL Server Driver][DBNETLIB]SQL Server does not exist or access denied.
Now, we know the database server didn’t suddenly go away because it had to exist to even get to this point.
After confirming that this stuff still works on the old Win2K server (against the same database server) we started poking around the code. No recent changes there and after way too many hours we took a break last night to mull it over some more.
This morning I decided that it might be interesting to install the SQL Server native drivers instead of using the ones that are bundled with Win2K3 (remember ODBC?). I installed the drivers, rebuilt the ODBC DSNs and tried again.
Still get an error, but now it is a little different:
[Microsoft][SQL Server Native Client 10.0]Named Pipes Provider: Could not open a connection to SQL Server [1326].
Funny, but that error was caused by the new drivers. I had to change the connection string to force TCP/IP instead of named pipes (the DB server is on a different subnet behind a firewall).
OK, back on TCP/IP connections we get yet another new error:
[Microsoft][SQL Server Native Client 10.0]TCP Provider: Only one usage of each socket address (protocol/network address/port) is normally permitted.
Oh, but wait – we have a new little twist. On a whim, my testing partner hit refresh on that error page and it reloaded and completed without error.
Clearly we have an environmental issue, not a coding issue (we can debate if deeply recursive ASP is an issue another day!)
After a bit of searching I turned up an interesting blog post titled, “Only one usage of each socket address (protocol/network address/port) is normally permitted.” It starts with a nice summary of why I’m getting these errors:
Here is the scoop
1. When you make authenticated calls, the client is closing connections. And when you are making authenticated calls repeatedly to the same server, you are making and closing connections repeatedly
2. The same might happen when you are making regular http [un authenticated] calls but setting keep-alive = false.When a connection is closed, on the side that is closing the connection the 5 tuple { Protocol, Local IP, Local Port, Remote IP, Remote Port} goes into a TIME_WAIT state for 240 seconds by default.
In this case, the protocol is fixed – TCP. [T]he local IP, remote IP and remote PORT are also typically fixed. So the variable is the local port. What happens is that when you don’t bind a port in the range 1024-5000 is used. So roughly you have 4000 ports. If you use all of them in 4 minutes – meaning roughly you make 16 web service calls per second for 4 minutes you will exhaust all the ports. That is the cause of this exception.
The article continues with a solution based on a couple registry modifications:
1. One of the ways is to increase the dynamic port range. The max by default is 5000. You can set this up to 65534.
HKLM\System\CurrentControlSet\Services\Tcpip\Parameters\MaxUserPortis the key to use.2. The second thing you can do is once the connection does get into an TIME_WAIT state you can reduce the time it is in that state, Default is 4 monutes, but you can set this to 30 seconds
HKLM\System\CurrentControlSet\Services\Tcpip\Parameters\TCPTimedWaitDelayis the key to use.
Set this to 30 seconds
Neither of those keys existed for me so I created both (as DWORD type). I used 32768 for the first one and 30 for the second. Rebooted the box and stuff started working!
As we add load to this machine I’ll likely have to revisit both parameters – and probably the code itself to try and streamline things a bit. I’m pretty sure I’m going to knock that 30 seconds parm down to closer to 10. But for now I’m happy. I have a server ready to take on the load and everything works.
Possibly Related posts:
November 8th, 2009 
Tags: 
