Unable to connect to the remote server

27 Jun 2009 12:49 AM

cj2

I have a TCP/IP server that accepts incoming TCP/IP traffic creating a
new thread to answer each request. The thread then takes the info and
calls a web service. The results of the web service call are then then
returned to the TCP/IP requestor and the thread ends. This process
usually takes less than a second. Each thread increments a counter when
starting and decrements it when finishing so the counter can be read to
find out how many threads or concurrent conversations are taking place
at any given time.

The problem seem to start as the count starts climbing into the teens.
The TCP/IP requests are being answered but then the threads last for
even a couple minutes before ending reporting they were "Unable to
connect to the remote server". The web service records all requests it
gets and it acts like it never saw any of those requests.

So for some reason after we get a certain number of concurrent
conversations going on the calls to the web service are not able to
connect. Why? Any ideas?

27 Jun 2009 1:13 AM

cj2

Second question

to kinda prevent the log jam of concurrent connections that happens when
requests take too long then time out, I've been asked to make sure no
thread lives more than 20 seconds. How's the best way to accomplish
this keeping in mind the thread will have to end in a controlled manner
by decrementing the counter?

cj2 wrote:
Show quoteHide quote

> I have a TCP/IP server that accepts incoming TCP/IP traffic creating a
> new thread to answer each request. The thread then takes the info and
> calls a web service. The results of the web service call are then then
> returned to the TCP/IP requestor and the thread ends. This process
> usually takes less than a second. Each thread increments a counter when
> starting and decrements it when finishing so the counter can be read to
> find out how many threads or concurrent conversations are taking place
> at any given time.
>
> The problem seem to start as the count starts climbing into the teens.
> The TCP/IP requests are being answered but then the threads last for
> even a couple minutes before ending reporting they were "Unable to
> connect to the remote server". The web service records all requests it
> gets and it acts like it never saw any of those requests.
>
> So for some reason after we get a certain number of concurrent
> conversations going on the calls to the web service are not able to
> connect. Why? Any ideas?

27 Jun 2009 4:42 AM

Mike

I provided an answer in the post when you first ask about this timing
requirement. Reread that post. Its about using the select() method
for the socket class.

--

cj2 wrote:

Show quoteHide quote

> Second question
>
> to kinda prevent the log jam of concurrent connections that happens when
> requests take too long then time out, I've been asked to make sure no
> thread lives more than 20 seconds. How's the best way to accomplish
> this keeping in mind the thread will have to end in a controlled manner
> by decrementing the counter?
>
> cj2 wrote:
>> I have a TCP/IP server that accepts incoming TCP/IP traffic creating a
>> new thread to answer each request. The thread then takes the info and
>> calls a web service. The results of the web service call are then then
>> returned to the TCP/IP requestor and the thread ends. This process
>> usually takes less than a second. Each thread increments a counter
>> when starting and decrements it when finishing so the counter can be
>> read to find out how many threads or concurrent conversations are
>> taking place at any given time.
>>
>> The problem seem to start as the count starts climbing into the teens.
>> The TCP/IP requests are being answered but then the threads last for
>> even a couple minutes before ending reporting they were "Unable to
>> connect to the remote server". The web service records all requests
>> it gets and it acts like it never saw any of those requests.
>>
>> So for some reason after we get a certain number of concurrent
>> conversations going on the calls to the web service are not able to
>> connect. Why? Any ideas?

27 Jun 2009 4:38 AM

Mike

Without showing some code, its only swags, but it could be that:

    1) You are not releasing sockets,
    2) You have some contention issue, which can be related
    3) to some synchronous calls, blocks or bottlenecks,
    4) You have a context switching performance issue,

or all the above.

What I do is go with vanilla programming here. Make sure the basic
client/server thread framework is working first by removing its guts
(the backend web server call). In other word, the thread object
(function) is empty or it add a Sleep(RandomNumber(300,1000)) to
simulate a random 300 to 1000 ms action. The goal is to get this part
working flawlessly, your counters are synchronized, all handles and
memory is released, etc, including loading it high with many client
request to resolve any high loading issues and you can also get a
optimal "baseline" performance measurement. Do that before you add
back any of the real work that could thread contentions and
performance issues.

--

cj2 wrote:

Show quoteHide quote

> I have a TCP/IP server that accepts incoming TCP/IP traffic creating a
> new thread to answer each request. The thread then takes the info and
> calls a web service. The results of the web service call are then then
> returned to the TCP/IP requestor and the thread ends. This process
> usually takes less than a second. Each thread increments a counter when
> starting and decrements it when finishing so the counter can be read to
> find out how many threads or concurrent conversations are taking place
> at any given time.
>
> The problem seem to start as the count starts climbing into the teens.
> The TCP/IP requests are being answered but then the threads last for
> even a couple minutes before ending reporting they were "Unable to
> connect to the remote server". The web service records all requests it
> gets and it acts like it never saw any of those requests.
>
> So for some reason after we get a certain number of concurrent
> conversations going on the calls to the web service are not able to
> connect. Why? Any ideas?

29 Jun 2009 2:03 PM

Jie Wang [MSFT]

Hi cj2,

Just like Mike said, without any code it is hard for us to figure out what
is actually going wrong.

And it seems to me like there is a design problem in your program.

Usually you don't just create a new worker thread when a request comes in -
there need to be an upper limit number of worker threads, not "the more the
better".

So in this case you may need to make use the ThreadPool class. Let this
class take care of the worker thread managing job, it is usually better
than do it yourself.

You may want to read this article about "How to use a Thread Pool":
http://msdn.microsoft.com/en-us/library/3dasc8as(VS.80).aspx

It is in C#, but the sample code is not using much C# specific language
elements. Or you can search the web for VB.NET ThreadPool samples, should
be a lot out there.

Hope this helps. And if you could share some code with us, could be
possible to find out more.

Regards,

Jie Wang

Microsoft Online Community Support

Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
msd***@microsoft.com.

==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/en-us/subscriptions/aa948868.aspx#notifications.

Note: MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 2 business days is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions. Issues of this
nature are best handled working with a dedicated Microsoft Support Engineer
by contacting Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/en-us/subscriptions/aa948874.aspx
==================================================
This posting is provided "AS IS" with no warranties, and confers no rights.

29 Jun 2009 8:15 PM

cj2

Thanks, I think the webservice timeout has fixed our problem so I will
not be needing further timing of the threads.

Jie Wang [MSFT] wrote:
Show quoteHide quote

> Hi cj2,
>
> Just like Mike said, without any code it is hard for us to figure out what
> is actually going wrong.
>
> And it seems to me like there is a design problem in your program.
>
> Usually you don't just create a new worker thread when a request comes in -
> there need to be an upper limit number of worker threads, not "the more the
> better".
>
> So in this case you may need to make use the ThreadPool class. Let this
> class take care of the worker thread managing job, it is usually better
> than do it yourself.
>
> You may want to read this article about "How to use a Thread Pool":
> http://msdn.microsoft.com/en-us/library/3dasc8as(VS.80).aspx
>
> It is in C#, but the sample code is not using much C# specific language
> elements. Or you can search the web for VB.NET ThreadPool samples, should
> be a lot out there.
>
> Hope this helps. And if you could share some code with us, could be
> possible to find out more.
>
> Regards,
>
> Jie Wang
>
> Microsoft Online Community Support
>
> Delighting our customers is our #1 priority. We welcome your comments and
> suggestions about how we can improve the support we provide to you. Please
> feel free to let my manager know what you think of the level of service
> provided. You can send feedback directly to my manager at:
> msd***@microsoft.com.
>
> ==================================================
> Get notification to my posts through email? Please refer to
> http://msdn.microsoft.com/en-us/subscriptions/aa948868.aspx#notifications.
>
> Note: MSDN Managed Newsgroup support offering is for non-urgent issues
> where an initial response from the community or a Microsoft Support
> Engineer within 2 business days is acceptable. Please note that each follow
> up response may take approximately 2 business days as the support
> professional working with you may need further investigation to reach the
> most efficient resolution. The offering is not appropriate for situations
> that require urgent, real-time or phone-based interactions. Issues of this
> nature are best handled working with a dedicated Microsoft Support Engineer
> by contacting Microsoft Customer Support Services (CSS) at
> http://msdn.microsoft.com/en-us/subscriptions/aa948874.aspx
> ==================================================
> This posting is provided "AS IS" with no warranties, and confers no rights.
>

30 Jun 2009 7:27 AM

Jie Wang [MSFT]

Good to know that.

If any further questions in the future, feel free to post here.

Have a nice day!

Jie Wang

Microsoft Online Community Support

Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
msd***@microsoft.com.

==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/en-us/subscriptions/aa948868.aspx#notifications.

Note: MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 2 business days is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions. Issues of this
nature are best handled working with a dedicated Microsoft Support Engineer
by contacting Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/en-us/subscriptions/aa948874.aspx
==================================================
This posting is provided "AS IS" with no warranties, and confers no rights.