Simplest way to save a CSV file from the internet to my computer?

microsoft.public.dotnet.languages.vb

19 Mar 2006 9:19 PM

Scott

Say there is a file on a website as a CSV. Not protected, just a data dump.
I'd like to collect this data to my computer to be analyzed.

Right now I am using the the MyWebClient feature. Code as follows:

Dim MyWebClient As New System.Net.WebClient

MyWebClient.DownloadFile(StringOfCSVURL, "c:\testdl.csv")

Then I take this information from the now-local CSV file and have it loaded
into a datatable to be edited and then finally update my local database with
the entire dataset.

Is there an easier way to do this? When this system is setup completely, it
will do this download and update about 100 times per day. Right now it's
taking about 14.5 seconds just to get the MyWebClient to open one CSV file
and save it locally. I had this initially in a PHP system that was much
quicker on this portion of the program.

Any help is appreciated.

20 Mar 2006 8:11 AM

Cor Ligthert [MVP]

Scott,

Not that I know, in the fetch process.

However to save it, you did not give us any information. Do you want to save
the CSV as a kind of blob or row by row.

In that case you have to see if there are duplicates and how you will handle
those. What is something we cannot help you with. (Override, update, or
maybe they even dont exist).

My expirience (and from others) is that the webclient downloadfile is
extremely fast by the way.

I hope this helps,

Cor