How do I get COMPLETE response from HttpWebResponse using StreamReader???

19 Jun 2006 7:11 AM

Hexman

Code below ----

I've asked a similar question on this forum earlier. This is a slightly different situation.

Previous Question ----

I'm trying to save some specific web pages to disk as text files. I searched the Internet and found a basic example which I changed to fit my needs.
I tested it out first on a single URL and it worked fine. Now when I incorporate an array of URL's, it fails to work. The first "responseFromServer"
properly retrieves and displays "http://finance.yahoo.com" (or any other valid URL for the first webrequest.create), but when I use the urls from the
array in the for/next structure, I never get a responseFromServer. My guess is that there must be some kind of initialization to the WebRequest,
Stream, or WebResponse.

Current Question ----

After changing some code, I now get responses returned to me. BUT, the pages I am going after have JavaScript in them to build the retrieve data,
tables, layout, etc. What I get back is the JavaScript code but none of the data I'm looking for. I thought the problem was an initialization of
some object, but now I believe that the actual page is not being built by the time my program is ready for the streamReader. I may be all wrong on
that assumption, but I need someone to steer me in the right direction.

Even checking EndOfStream doesn't wait for the page to be built.

Thanks,

Hexman

P.S. I can run the program repeatedly and on some runs, SOME of the pages are comeplete! So to me that points me to the building of the requested
pages.

Newer Code ----
--------------------------------------------------------------------------------------------
This code replaces the processing of the Old Code.   Changes: Try-Catch, ReadLine
rather than ReadToEnd, URL from a datatable. None of these changes should affect the
problem area of the program.

                Try
                    Dim request As HttpWebRequest = WebRequest.Create(URL)
                    Dim response As HttpWebResponse = request.GetResponse()
                    Dim reader As StreamReader = New StreamReader(response.GetResponseStream())
                    strToWrite = ""
                    Do While reader.EndOfStream = False
                        str = reader.ReadLine()
                        strToWrite = strToWrite & str
                    Loop

                    strOutFile = OutFN(Idx)
                    File.WriteAllText(strOutFile, strToWrite)
                    reader.Close()
                    response.Close()

                Catch ex As WebException
                    Console.WriteLine(ex.Message)
                    If ex.Status = WebExceptionStatus.ProtocolError Then
                        Console.WriteLine("Status Code : {0}", CType(ex.Response, HttpWebResponse).StatusCode)
                        Console.WriteLine("Status Description : {0}", CType(ex.Response, HttpWebResponse).StatusDescription)
                    End If
                Catch ex As Exception
                    Console.WriteLine(ex.Message)
                End Try
            End If

Old Code ----
--------------------------------------------------------------------------------------------
Imports System
Imports System.IO
Imports System.Net
Imports System.Net.WebClient
Imports System.Windows.Forms
Public Class Form1

    Private Sub btnProcess_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnProcess.Click
        Dim x As Integer
        Dim aryUrls(15) As String
        aryUrls(0) = "http://finance.yahoo.com/q/bc?s=DOW&t=1d"
        aryUrls(1) = "http://finance.yahoo.com/q/bc?s=IBM&t=6m"
        aryUrls(2) = "http://finance.yahoo.com/q/bc?s=MSFT&t=6m"
        aryUrls(3) = "http://biz.yahoo.com/top.html"

        Dim request As WebRequest = WebRequest.Create("http://finance.yahoo.com/")
        Dim response As WebResponse = request.GetResponse()
        Dim dataStream As Stream = response.GetResponseStream()
        Dim reader As New StreamReader(dataStream)
        Dim responseFromServer As String = reader.ReadToEnd()

        MsgBox(responseFromServer) <=== Shows finance.yahoo.com

        For x = 0 To 3
            request = WebRequest.Create(aryUrls(x))
            dataStream = response.GetResponseStream()
            responseFromServer = reader.ReadToEnd()
            MsgBox(responseFromServer) <=== Shows nothing
        Next

        reader.Close()
        response.Close()
    End Sub

19 Jun 2006 9:56 AM

Cor Ligthert [MVP]

Hexman,

You be aware that many webpages don't exist from one document but from more
documents containing frames (with there own urls)

If you really need a complete page in HTML than in my idea fits the
webbrowser for that the best.

I hope this helps,

Cor.

Show quoteHide quote

"Hexman" <Hex***@binary.com> schreef in bericht
news:28jc929lm07jiqjumfcikhnbnimrvs8u89@4ax.com...
> Code below ----
>
> I've asked a similar question on this forum earlier. This is a slightly
> different situation.
>
> Previous Question ----
>
> I'm trying to save some specific web pages to disk as text files. I
> searched the Internet and found a basic example which I changed to fit my
> needs.
> I tested it out first on a single URL and it worked fine. Now when I
> incorporate an array of URL's, it fails to work. The first
> "responseFromServer"
> properly retrieves and displays "http://finance.yahoo.com" (or any other
> valid URL for the first webrequest.create), but when I use the urls from
> the
> array in the for/next structure, I never get a responseFromServer. My
> guess is that there must be some kind of initialization to the WebRequest,
> Stream, or WebResponse.
>
> Current Question ----
>
> After changing some code, I now get responses returned to me. BUT, the
> pages I am going after have JavaScript in them to build the retrieve data,
> tables, layout, etc. What I get back is the JavaScript code but none of
> the data I'm looking for. I thought the problem was an initialization of
> some object, but now I believe that the actual page is not being built by
> the time my program is ready for the streamReader. I may be all wrong on
> that assumption, but I need someone to steer me in the right direction.
>
> Even checking EndOfStream doesn't wait for the page to be built.
>
> Thanks,
>
> Hexman
>
> P.S. I can run the program repeatedly and on some runs, SOME of the pages
> are comeplete! So to me that points me to the building of the requested
> pages.
>
>
>
> Newer Code ----
> --------------------------------------------------------------------------------------------
> This code replaces the processing of the Old Code.   Changes: Try-Catch,
> ReadLine
> rather than ReadToEnd, URL from a datatable. None of these changes should
> affect the
> problem area of the program.
>
>                Try
>                    Dim request As HttpWebRequest = WebRequest.Create(URL)
>                    Dim response As HttpWebResponse = request.GetResponse()
>                    Dim reader As StreamReader = New
> StreamReader(response.GetResponseStream())
>                    strToWrite = ""
>                    Do While reader.EndOfStream = False
>                        str = reader.ReadLine()
>                        strToWrite = strToWrite & str
>                    Loop
>
>                    strOutFile = OutFN(Idx)
>                    File.WriteAllText(strOutFile, strToWrite)
>                    reader.Close()
>                    response.Close()
>
>                Catch ex As WebException
>                    Console.WriteLine(ex.Message)
>                    If ex.Status = WebExceptionStatus.ProtocolError Then
>                        Console.WriteLine("Status Code : {0}",
> CType(ex.Response, HttpWebResponse).StatusCode)
>                        Console.WriteLine("Status Description : {0}",
> CType(ex.Response, HttpWebResponse).StatusDescription)
>                    End If
>                Catch ex As Exception
>                    Console.WriteLine(ex.Message)
>                End Try
>            End If
>
> Old Code ----
> --------------------------------------------------------------------------------------------
> Imports System
> Imports System.IO
> Imports System.Net
> Imports System.Net.WebClient
> Imports System.Windows.Forms
> Public Class Form1
>
>    Private Sub btnProcess_Click(ByVal sender As System.Object, ByVal e As
> System.EventArgs) Handles btnProcess.Click
>        Dim x As Integer
>        Dim aryUrls(15) As String
>        aryUrls(0) = "http://finance.yahoo.com/q/bc?s=DOW&t=1d"
>        aryUrls(1) = "http://finance.yahoo.com/q/bc?s=IBM&t=6m"
>        aryUrls(2) = "http://finance.yahoo.com/q/bc?s=MSFT&t=6m"
>        aryUrls(3) = "http://biz.yahoo.com/top.html"
>
>        Dim request As WebRequest =
> WebRequest.Create("http://finance.yahoo.com/")
>        Dim response As WebResponse = request.GetResponse()
>        Dim dataStream As Stream = response.GetResponseStream()
>        Dim reader As New StreamReader(dataStream)
>        Dim responseFromServer As String = reader.ReadToEnd()
>
>        MsgBox(responseFromServer) <=== Shows finance.yahoo.com
>
>        For x = 0 To 3
>            request = WebRequest.Create(aryUrls(x))
>            dataStream = response.GetResponseStream()
>            responseFromServer = reader.ReadToEnd()
>            MsgBox(responseFromServer) <=== Shows nothing
>        Next
>
>        reader.Close()
>        response.Close()
>    End Sub

19 Jun 2006 7:56 PM

Hexman

Thanks, I'll read up on the webbrowser.

Hexman

Show quoteHide quote

On Mon, 19 Jun 2006 11:56:00 +0200, "Cor Ligthert [MVP]" <notmyfirstn***@planet.nl> wrote:

>Hexman,
>
>You be aware that many webpages don't exist from one document but from more
>documents containing frames (with there own urls)
>
>If you really need a complete page in HTML than in my idea fits the
>webbrowser for that the best.
>
>I hope this helps,
>
>Cor.
>
>
>"Hexman" <Hex***@binary.com> schreef in bericht
>news:28jc929lm07jiqjumfcikhnbnimrvs8u89@4ax.com...
>> Code below ----
>>
>> I've asked a similar question on this forum earlier. This is a slightly
>> different situation.
>>
>> Previous Question ----
>>
>> I'm trying to save some specific web pages to disk as text files. I
>> searched the Internet and found a basic example which I changed to fit my
>> needs.
>> I tested it out first on a single URL and it worked fine. Now when I
>> incorporate an array of URL's, it fails to work. The first
>> "responseFromServer"
>> properly retrieves and displays "http://finance.yahoo.com" (or any other
>> valid URL for the first webrequest.create), but when I use the urls from
>> the
>> array in the for/next structure, I never get a responseFromServer. My
>> guess is that there must be some kind of initialization to the WebRequest,
>> Stream, or WebResponse.
>>
>> Current Question ----
>>
>> After changing some code, I now get responses returned to me. BUT, the
>> pages I am going after have JavaScript in them to build the retrieve data,
>> tables, layout, etc. What I get back is the JavaScript code but none of
>> the data I'm looking for. I thought the problem was an initialization of
>> some object, but now I believe that the actual page is not being built by
>> the time my program is ready for the streamReader. I may be all wrong on
>> that assumption, but I need someone to steer me in the right direction.
>>
>> Even checking EndOfStream doesn't wait for the page to be built.
>>
>> Thanks,
>>
>> Hexman
>>
>> P.S. I can run the program repeatedly and on some runs, SOME of the pages
>> are comeplete! So to me that points me to the building of the requested
>> pages.
>>
>>
>>
>> Newer Code ----
>> --------------------------------------------------------------------------------------------
>> This code replaces the processing of the Old Code.   Changes: Try-Catch,
>> ReadLine
>> rather than ReadToEnd, URL from a datatable. None of these changes should
>> affect the
>> problem area of the program.
>>
>>                Try
>>                    Dim request As HttpWebRequest = WebRequest.Create(URL)
>>                    Dim response As HttpWebResponse = request.GetResponse()
>>                    Dim reader As StreamReader = New
>> StreamReader(response.GetResponseStream())
>>                    strToWrite = ""
>>                    Do While reader.EndOfStream = False
>>                        str = reader.ReadLine()
>>                        strToWrite = strToWrite & str
>>                    Loop
>>
>>                    strOutFile = OutFN(Idx)
>>                    File.WriteAllText(strOutFile, strToWrite)
>>                    reader.Close()
>>                    response.Close()
>>
>>                Catch ex As WebException
>>                    Console.WriteLine(ex.Message)
>>                    If ex.Status = WebExceptionStatus.ProtocolError Then
>>                        Console.WriteLine("Status Code : {0}",
>> CType(ex.Response, HttpWebResponse).StatusCode)
>>                        Console.WriteLine("Status Description : {0}",
>> CType(ex.Response, HttpWebResponse).StatusDescription)
>>                    End If
>>                Catch ex As Exception
>>                    Console.WriteLine(ex.Message)
>>                End Try
>>            End If
>>
>> Old Code ----
>> --------------------------------------------------------------------------------------------
>> Imports System
>> Imports System.IO
>> Imports System.Net
>> Imports System.Net.WebClient
>> Imports System.Windows.Forms
>> Public Class Form1
>>
>>    Private Sub btnProcess_Click(ByVal sender As System.Object, ByVal e As
>> System.EventArgs) Handles btnProcess.Click
>>        Dim x As Integer
>>        Dim aryUrls(15) As String
>>        aryUrls(0) = "http://finance.yahoo.com/q/bc?s=DOW&t=1d"
>>        aryUrls(1) = "http://finance.yahoo.com/q/bc?s=IBM&t=6m"
>>        aryUrls(2) = "http://finance.yahoo.com/q/bc?s=MSFT&t=6m"
>>        aryUrls(3) = "http://biz.yahoo.com/top.html"
>>
>>        Dim request As WebRequest =
>> WebRequest.Create("http://finance.yahoo.com/")
>>        Dim response As WebResponse = request.GetResponse()
>>        Dim dataStream As Stream = response.GetResponseStream()
>>        Dim reader As New StreamReader(dataStream)
>>        Dim responseFromServer As String = reader.ReadToEnd()
>>
>>        MsgBox(responseFromServer) <=== Shows finance.yahoo.com
>>
>>        For x = 0 To 3
>>            request = WebRequest.Create(aryUrls(x))
>>            dataStream = response.GetResponseStream()
>>            responseFromServer = reader.ReadToEnd()
>>            MsgBox(responseFromServer) <=== Shows nothing
>>        Next
>>
>>        reader.Close()
>>        response.Close()
>>    End Sub
>

20 Jun 2006 4:42 AM

Hexman

Cor,

I've looked into WebBrowser and that appears to be able to give me what I need. I tried it in code using textbox for displaying the html source and
webbrowser for the html using ONE URL. I've written the text to a file. All is well......EXCEPT.....when I use an array (actually a datatable)
containing the URLs I want to retrieve, they're processed so fast that the pages are not completely loaded before the next one starts to be navigated
to.

I've added a "WebBrowserDocumentCompletedEventHandler" and it does fire but not when I'm thinking it would.

I need some kind of delay mechinism until each page is completely loaded.

For this app I need to run the webbrowser procedurally (modal) so it doesn't come back to the next instruction until it has completed loading the
page. (it seems that when webbrowser navigate command is given, a new (independent thread) is started.

Is there a "Wait for Event" (WaitForEvent(Webbrowser.DocumentCompleted) or something like that?)

I'm almost over the edge,

Hexman

Show quoteHide quote

On Mon, 19 Jun 2006 11:56:00 +0200, "Cor Ligthert [MVP]" <notmyfirstn***@planet.nl> wrote:

>Hexman,
>
>You be aware that many webpages don't exist from one document but from more
>documents containing frames (with there own urls)
>
>If you really need a complete page in HTML than in my idea fits the
>webbrowser for that the best.
>
>I hope this helps,
>
>Cor.
>
>
>"Hexman" <Hex***@binary.com> schreef in bericht
>news:28jc929lm07jiqjumfcikhnbnimrvs8u89@4ax.com...
>> Code below ----
>>
>> I've asked a similar question on this forum earlier. This is a slightly
>> different situation.
>>
>> Previous Question ----
>>
>> I'm trying to save some specific web pages to disk as text files. I
>> searched the Internet and found a basic example which I changed to fit my
>> needs.
>> I tested it out first on a single URL and it worked fine. Now when I
>> incorporate an array of URL's, it fails to work. The first
>> "responseFromServer"
>> properly retrieves and displays "http://finance.yahoo.com" (or any other
>> valid URL for the first webrequest.create), but when I use the urls from
>> the
>> array in the for/next structure, I never get a responseFromServer. My
>> guess is that there must be some kind of initialization to the WebRequest,
>> Stream, or WebResponse.
>>
>> Current Question ----
>>
>> After changing some code, I now get responses returned to me. BUT, the
>> pages I am going after have JavaScript in them to build the retrieve data,
>> tables, layout, etc. What I get back is the JavaScript code but none of
>> the data I'm looking for. I thought the problem was an initialization of
>> some object, but now I believe that the actual page is not being built by
>> the time my program is ready for the streamReader. I may be all wrong on
>> that assumption, but I need someone to steer me in the right direction.
>>
>> Even checking EndOfStream doesn't wait for the page to be built.
>>
>> Thanks,
>>
>> Hexman
>>
>> P.S. I can run the program repeatedly and on some runs, SOME of the pages
>> are comeplete! So to me that points me to the building of the requested
>> pages.
>>
>>
>>
>> Newer Code ----
>> --------------------------------------------------------------------------------------------
>> This code replaces the processing of the Old Code.   Changes: Try-Catch,
>> ReadLine
>> rather than ReadToEnd, URL from a datatable. None of these changes should
>> affect the
>> problem area of the program.
>>
>>                Try
>>                    Dim request As HttpWebRequest = WebRequest.Create(URL)
>>                    Dim response As HttpWebResponse = request.GetResponse()
>>                    Dim reader As StreamReader = New
>> StreamReader(response.GetResponseStream())
>>                    strToWrite = ""
>>                    Do While reader.EndOfStream = False
>>                        str = reader.ReadLine()
>>                        strToWrite = strToWrite & str
>>                    Loop
>>
>>                    strOutFile = OutFN(Idx)
>>                    File.WriteAllText(strOutFile, strToWrite)
>>                    reader.Close()
>>                    response.Close()
>>
>>                Catch ex As WebException
>>                    Console.WriteLine(ex.Message)
>>                    If ex.Status = WebExceptionStatus.ProtocolError Then
>>                        Console.WriteLine("Status Code : {0}",
>> CType(ex.Response, HttpWebResponse).StatusCode)
>>                        Console.WriteLine("Status Description : {0}",
>> CType(ex.Response, HttpWebResponse).StatusDescription)
>>                    End If
>>                Catch ex As Exception
>>                    Console.WriteLine(ex.Message)
>>                End Try
>>            End If
>>
>> Old Code ----
>> --------------------------------------------------------------------------------------------
>> Imports System
>> Imports System.IO
>> Imports System.Net
>> Imports System.Net.WebClient
>> Imports System.Windows.Forms
>> Public Class Form1
>>
>>    Private Sub btnProcess_Click(ByVal sender As System.Object, ByVal e As
>> System.EventArgs) Handles btnProcess.Click
>>        Dim x As Integer
>>        Dim aryUrls(15) As String
>>        aryUrls(0) = "http://finance.yahoo.com/q/bc?s=DOW&t=1d"
>>        aryUrls(1) = "http://finance.yahoo.com/q/bc?s=IBM&t=6m"
>>        aryUrls(2) = "http://finance.yahoo.com/q/bc?s=MSFT&t=6m"
>>        aryUrls(3) = "http://biz.yahoo.com/top.html"
>>
>>        Dim request As WebRequest =
>> WebRequest.Create("http://finance.yahoo.com/")
>>        Dim response As WebResponse = request.GetResponse()
>>        Dim dataStream As Stream = response.GetResponseStream()
>>        Dim reader As New StreamReader(dataStream)
>>        Dim responseFromServer As String = reader.ReadToEnd()
>>
>>        MsgBox(responseFromServer) <=== Shows finance.yahoo.com
>>
>>        For x = 0 To 3
>>            request = WebRequest.Create(aryUrls(x))
>>            dataStream = response.GetResponseStream()
>>            responseFromServer = reader.ReadToEnd()
>>            MsgBox(responseFromServer) <=== Shows nothing
>>        Next
>>
>>        reader.Close()
>>        response.Close()
>>    End Sub
>

20 Jun 2006 5:42 AM

Cor Ligthert [MVP]

Hexman,

I have the same problem. In the AxWebbrowser, that is from the previous
version, there was a "download complete" event. I am missing that now. I
think that I would in your situation try if the AxWebbrowser would help me.

Axwebbrowser download complete
http://msdn.microsoft.com/library/default.asp?url=/workshop/browser/webbrowser/reference/events/downloadcomplete.asp

Webbrowser
http://msdn2.microsoft.com/en-us/library/system.windows.forms.webbrowser_members.aspx

I will ask this on another place to by the way.

Cor

Show quoteHide quote

"Hexman" <Hex***@binary.com> schreef in bericht
news:vste929ujjehi6oo38svq0o3646bhff7fh@4ax.com...
> Cor,
>
> I've looked into WebBrowser and that appears to be able to give me what I
> need. I tried it in code using textbox for displaying the html source and
> webbrowser for the html using ONE URL. I've written the text to a file.
> All is well......EXCEPT.....when I use an array (actually a datatable)
> containing the URLs I want to retrieve, they're processed so fast that the
> pages are not completely loaded before the next one starts to be navigated
> to.
>
> I've added a "WebBrowserDocumentCompletedEventHandler" and it does fire
> but not when I'm thinking it would.
>
> I need some kind of delay mechinism until each page is completely loaded.
>
> For this app I need to run the webbrowser procedurally (modal) so it
> doesn't come back to the next instruction until it has completed loading
> the
> page. (it seems that when webbrowser navigate command is given, a new
> (independent thread) is started.
>
> Is there a "Wait for Event" (WaitForEvent(Webbrowser.DocumentCompleted) or
> something like that?)
>
> I'm almost over the edge,
>
> Hexman
>
> On Mon, 19 Jun 2006 11:56:00 +0200, "Cor Ligthert [MVP]"
> <notmyfirstn***@planet.nl> wrote:
>
>>Hexman,
>>
>>You be aware that many webpages don't exist from one document but from
>>more
>>documents containing frames (with there own urls)
>>
>>If you really need a complete page in HTML than in my idea fits the
>>webbrowser for that the best.
>>
>>I hope this helps,
>>
>>Cor.
>>
>>
>>"Hexman" <Hex***@binary.com> schreef in bericht
>>news:28jc929lm07jiqjumfcikhnbnimrvs8u89@4ax.com...
>>> Code below ----
>>>
>>> I've asked a similar question on this forum earlier. This is a slightly
>>> different situation.
>>>
>>> Previous Question ----
>>>
>>> I'm trying to save some specific web pages to disk as text files. I
>>> searched the Internet and found a basic example which I changed to fit
>>> my
>>> needs.
>>> I tested it out first on a single URL and it worked fine. Now when I
>>> incorporate an array of URL's, it fails to work. The first
>>> "responseFromServer"
>>> properly retrieves and displays "http://finance.yahoo.com" (or any other
>>> valid URL for the first webrequest.create), but when I use the urls from
>>> the
>>> array in the for/next structure, I never get a responseFromServer. My
>>> guess is that there must be some kind of initialization to the
>>> WebRequest,
>>> Stream, or WebResponse.
>>>
>>> Current Question ----
>>>
>>> After changing some code, I now get responses returned to me. BUT, the
>>> pages I am going after have JavaScript in them to build the retrieve
>>> data,
>>> tables, layout, etc. What I get back is the JavaScript code but none of
>>> the data I'm looking for. I thought the problem was an initialization
>>> of
>>> some object, but now I believe that the actual page is not being built
>>> by
>>> the time my program is ready for the streamReader. I may be all wrong
>>> on
>>> that assumption, but I need someone to steer me in the right direction.
>>>
>>> Even checking EndOfStream doesn't wait for the page to be built.
>>>
>>> Thanks,
>>>
>>> Hexman
>>>
>>> P.S. I can run the program repeatedly and on some runs, SOME of the
>>> pages
>>> are comeplete! So to me that points me to the building of the requested
>>> pages.
>>>
>>>
>>>
>>> Newer Code ----
>>> --------------------------------------------------------------------------------------------
>>> This code replaces the processing of the Old Code.   Changes: Try-Catch,
>>> ReadLine
>>> rather than ReadToEnd, URL from a datatable. None of these changes
>>> should
>>> affect the
>>> problem area of the program.
>>>
>>>                Try
>>>                    Dim request As HttpWebRequest =
>>> WebRequest.Create(URL)
>>>                    Dim response As HttpWebResponse =
>>> request.GetResponse()
>>>                    Dim reader As StreamReader = New
>>> StreamReader(response.GetResponseStream())
>>>                    strToWrite = ""
>>>                    Do While reader.EndOfStream = False
>>>                        str = reader.ReadLine()
>>>                        strToWrite = strToWrite & str
>>>                    Loop
>>>
>>>                    strOutFile = OutFN(Idx)
>>>                    File.WriteAllText(strOutFile, strToWrite)
>>>                    reader.Close()
>>>                    response.Close()
>>>
>>>                Catch ex As WebException
>>>                    Console.WriteLine(ex.Message)
>>>                    If ex.Status = WebExceptionStatus.ProtocolError Then
>>>                        Console.WriteLine("Status Code : {0}",
>>> CType(ex.Response, HttpWebResponse).StatusCode)
>>>                        Console.WriteLine("Status Description : {0}",
>>> CType(ex.Response, HttpWebResponse).StatusDescription)
>>>                    End If
>>>                Catch ex As Exception
>>>                    Console.WriteLine(ex.Message)
>>>                End Try
>>>            End If
>>>
>>> Old Code ----
>>> --------------------------------------------------------------------------------------------
>>> Imports System
>>> Imports System.IO
>>> Imports System.Net
>>> Imports System.Net.WebClient
>>> Imports System.Windows.Forms
>>> Public Class Form1
>>>
>>>    Private Sub btnProcess_Click(ByVal sender As System.Object, ByVal e
>>> As
>>> System.EventArgs) Handles btnProcess.Click
>>>        Dim x As Integer
>>>        Dim aryUrls(15) As String
>>>        aryUrls(0) = "http://finance.yahoo.com/q/bc?s=DOW&t=1d"
>>>        aryUrls(1) = "http://finance.yahoo.com/q/bc?s=IBM&t=6m"
>>>        aryUrls(2) = "http://finance.yahoo.com/q/bc?s=MSFT&t=6m"
>>>        aryUrls(3) = "http://biz.yahoo.com/top.html"
>>>
>>>        Dim request As WebRequest =
>>> WebRequest.Create("http://finance.yahoo.com/")
>>>        Dim response As WebResponse = request.GetResponse()
>>>        Dim dataStream As Stream = response.GetResponseStream()
>>>        Dim reader As New StreamReader(dataStream)
>>>        Dim responseFromServer As String = reader.ReadToEnd()
>>>
>>>        MsgBox(responseFromServer) <=== Shows finance.yahoo.com
>>>
>>>        For x = 0 To 3
>>>            request = WebRequest.Create(aryUrls(x))
>>>            dataStream = response.GetResponseStream()
>>>            responseFromServer = reader.ReadToEnd()
>>>            MsgBox(responseFromServer) <=== Shows nothing
>>>        Next
>>>
>>>        reader.Close()
>>>        response.Close()
>>>    End Sub
>>