I am looking for suggestions on the most efficient way to import 7
different fixed width files into a DataSet.  Not all at the same time.
One file at a time, but the format could change from file to file.

I have 7 (and more coming) different fixed width files that my clients
provide to me, within each one the field sizes and ordering is slightly
different...maddening I know.

Each file could possibly be as large as 150 MG with around 70,000
records in them.

I need to be able to import these files into a DataSet so I can do
further requirements/validation checking on certain fields.

I've looked at FileHelpers (http://filehelpers.sf.net) and
TextFileParser and neither one will do what I need to do.

Any suggestions?

I would use the Microsoft OLE Text Driver.

Do you have MS Access? That would be my first goto.

BostonNode,

As it is done from the first time the Mid method exist.

Create your dataset and read the files where you using Mid put the right
parts in the right parts from the datasets.

This is all information I can give based on your description from your
problem.

Cor

the answer depends on the situation    ( that you not fully describe )

1.  if this is a one time operation  i would use Access  for this   , as it
has all sorts of nice wizzards to import data from different sources ,
especially text   with even the ability to save the import specifications
and automate the hole process from VBA

2. if you want the fastest way , and this is occuring frequently ,,,, i
would write my own data import routine    i have once written a Mysql dump
import class in VB.Net with filestreams and a stringbuilder that was capable
of importing and converting a flat file  Mysql dump  of 7,5 GB   ( giga
bytes !!!  ) to MS SQL  in only + 20 minutes

Ofcourse honestly needs to say that this was a client server proggy design
where the server was a Poweredge   4600  ;-)  the client was a AMD XP 2400+
( standard pc with 512 MB ) wich was actually performing the conversion to
the SQL server with batches of 1000 records in one time

So lightening speed  , but also a bit work to write   go for your own import
routine


regards

Michel Posseth [MCP]




Show quoteHide quote

Starting a Windows app from a windows service
need ideas on multi threaded db update
Extending the TreeNode - - -> HELP!
get a listing of assemblies from GAC ?
between two times
"Specified cast is not valid" converting from object A to object B
Copying a project...
Image Analysis, OCR
Guess I have to give up trying to clone a menu!
Need a Dummy's guide to printing 3 listboxes