Program flow with multi-threading app

Program flow with multi-threading app - corrupted datatables

microsoft.public.dotnet.languages.vb

21 Jul 2006 12:56 PM

Mike

Hi,

I am building a windows app. I have two threads that continually run
in the background pulling information from a website. The data I
extract from the two separate threads needs to be put into various
columns of the same datatable (which is defined in a dataset on my form
in design time). The threads take different amounts of time to cycle
so the updates to the datatable frequently overlap.

I have run into various issues in the that the datatable frequently
becomes corrupted by doing this (datatable internal index is
corrupted:13 message). I am not sure if this is the result of
simultaneous updates / lockings or trying to access the datatable from
the worker threads.

I would be grateful if you could let me know the correct way of
updating this datatable. Should I be passing the returned data back to
the thread that the datatable is on before trying to update it? If so
how would I do this? Also, are there any other considerations I am
missing in terms of the program design?

Thank you very much,

Mike

21 Jul 2006 1:54 PM

Cor Ligthert [MVP]

Mike,

As long as you do not show the datatable in a UI and do synclock the adding
of the newrows there should not be any problem.

The only thing can be that it takes a little bit long, so I would put a
Queue between it.

Cor

Show quoteHide quote

"Mike" <my***@pearcey2001.freeserve.co.uk> schreef in bericht
news:1153486602.157945.322210@i3g2000cwc.googlegroups.com...
> Hi,
>
> I am building a windows app. I have two threads that continually run
> in the background pulling information from a website. The data I
> extract from the two separate threads needs to be put into various
> columns of the same datatable (which is defined in a dataset on my form
> in design time). The threads take different amounts of time to cycle
> so the updates to the datatable frequently overlap.
>
> I have run into various issues in the that the datatable frequently
> becomes corrupted by doing this (datatable internal index is
> corrupted:13 message). I am not sure if this is the result of
> simultaneous updates / lockings or trying to access the datatable from
> the worker threads.
>
> I would be grateful if you could let me know the correct way of
> updating this datatable. Should I be passing the returned data back to
> the thread that the datatable is on before trying to update it? If so
> how would I do this? Also, are there any other considerations I am
> missing in terms of the program design?
>
> Thank you very much,
>
> Mike
>

21 Jul 2006 5:16 PM

Mike

Thanks for replying Cor. Apologies but I forgot to mention that I have
this datatable bound to a datagridview on the form. Assuming this is
the problem, would the synclock solution still work? I thought that
these kinds of updates were not thread safe and this is probably why it
is crashing.

Thanks again for any help,

Mike

Cor Ligthert [MVP] wrote:
Show quoteHide quote

> Mike,
>
> As long as you do not show the datatable in a UI and do synclock the adding
> of the newrows there should not be any problem.
>
> The only thing can be that it takes a little bit long, so I would put a
> Queue between it.
>
> Cor
>
>
> "Mike" <my***@pearcey2001.freeserve.co.uk> schreef in bericht
> news:1153486602.157945.322210@i3g2000cwc.googlegroups.com...
> > Hi,
> >
> > I am building a windows app. I have two threads that continually run
> > in the background pulling information from a website. The data I
> > extract from the two separate threads needs to be put into various
> > columns of the same datatable (which is defined in a dataset on my form
> > in design time). The threads take different amounts of time to cycle
> > so the updates to the datatable frequently overlap.
> >
> > I have run into various issues in the that the datatable frequently
> > becomes corrupted by doing this (datatable internal index is
> > corrupted:13 message). I am not sure if this is the result of
> > simultaneous updates / lockings or trying to access the datatable from
> > the worker threads.
> >
> > I would be grateful if you could let me know the correct way of
> > updating this datatable. Should I be passing the returned data back to
> > the thread that the datatable is on before trying to update it? If so
> > how would I do this? Also, are there any other considerations I am
> > missing in terms of the program design?
> >
> > Thank you very much,
> >
> > Mike
> >

21 Jul 2006 6:11 PM

Cor Ligthert [MVP]

Mike,

I have no time now anymore to make a sample (and try it), but when I would
make your solution than.

I would use the queue class (you know what that is in Britain, I have
refreshed my knowledge of that in a Holiday there last week).

http://msdn2.microsoft.com/en-us/library/system.collections.queue.aspx

Let the threads fill (enqueue) the queue with datarows while synclocking it.

Than I would use a forms timer and set that to about 5 or 10 seconds
whatever you like.

Than I would add those datarows using that queue class (dequeue) into the
datatable in the timer event, what has as well has to be done synclocked.

At the end of my timer event I would refresh my datagridview.

I think that this does not even need much changes in your program.

I hope this helps,

Cor

Show quoteHide quote

"Mike" <my***@pearcey2001.freeserve.co.uk> schreef in bericht
news:1153502211.877109.78820@i3g2000cwc.googlegroups.com...
> Thanks for replying Cor. Apologies but I forgot to mention that I have
> this datatable bound to a datagridview on the form. Assuming this is
> the problem, would the synclock solution still work? I thought that
> these kinds of updates were not thread safe and this is probably why it
> is crashing.
>
> Thanks again for any help,
>
> Mike
>
> Cor Ligthert [MVP] wrote:
>> Mike,
>>
>> As long as you do not show the datatable in a UI and do synclock the
>> adding
>> of the newrows there should not be any problem.
>>
>> The only thing can be that it takes a little bit long, so I would put a
>> Queue between it.
>>
>> Cor
>>
>>
>> "Mike" <my***@pearcey2001.freeserve.co.uk> schreef in bericht
>> news:1153486602.157945.322210@i3g2000cwc.googlegroups.com...
>> > Hi,
>> >
>> > I am building a windows app. I have two threads that continually run
>> > in the background pulling information from a website. The data I
>> > extract from the two separate threads needs to be put into various
>> > columns of the same datatable (which is defined in a dataset on my form
>> > in design time). The threads take different amounts of time to cycle
>> > so the updates to the datatable frequently overlap.
>> >
>> > I have run into various issues in the that the datatable frequently
>> > becomes corrupted by doing this (datatable internal index is
>> > corrupted:13 message). I am not sure if this is the result of
>> > simultaneous updates / lockings or trying to access the datatable from
>> > the worker threads.
>> >
>> > I would be grateful if you could let me know the correct way of
>> > updating this datatable. Should I be passing the returned data back to
>> > the thread that the datatable is on before trying to update it? If so
>> > how would I do this? Also, are there any other considerations I am
>> > missing in terms of the program design?
>> >
>> > Thank you very much,
>> >
>> > Mike
>> >
>

21 Jul 2006 7:34 PM

Mike

Yes we love our queues :-) I hope you enjoyed your stay.

The synclock on its own was enough to fix this problem - the majority
of the updates to the datatable were value updates rather than row
additions.
Using synclock slowed down the app quite a bit (I have to lock the
table 2-3 times per datarefresh, datarefresh is 2 times per second).
I don't yet understand all the details as I'm still quite an amateur
but using a delegate from the worker thread to make all the changes to
the datatable (including the use of the synclock) works excellently.
The time difference was roughly 50ms vs 500ms without delegate. I
guess there is a very good reason for this.

Incredibly happy to have solved the problem and really appreciate the
help!

Thank you,

Mike

Cor Ligthert [MVP] wrote:
Show quoteHide quote

> Mike,
>
> I have no time now anymore to make a sample (and try it), but when I would
> make your solution than.
>
> I would use the queue class (you know what that is in Britain, I have
> refreshed my knowledge of that in a Holiday there last week).
>
> http://msdn2.microsoft.com/en-us/library/system.collections.queue.aspx
>
> Let the threads fill (enqueue) the queue with datarows while synclocking it.
>
> Than I would use a forms timer and set that to about 5 or 10 seconds
> whatever you like.
>
> Than I would add those datarows using that queue class (dequeue) into the
> datatable in the timer event, what has as well has to be done synclocked.
>
> At the end of my timer event I would refresh my datagridview.
>
> I think that this does not even need much changes in your program.
>
> I hope this helps,
>
> Cor
>
>
>
> "Mike" <my***@pearcey2001.freeserve.co.uk> schreef in bericht
> news:1153502211.877109.78820@i3g2000cwc.googlegroups.com...
> > Thanks for replying Cor. Apologies but I forgot to mention that I have
> > this datatable bound to a datagridview on the form. Assuming this is
> > the problem, would the synclock solution still work? I thought that
> > these kinds of updates were not thread safe and this is probably why it
> > is crashing.
> >
> > Thanks again for any help,
> >
> > Mike
> >
> > Cor Ligthert [MVP] wrote:
> >> Mike,
> >>
> >> As long as you do not show the datatable in a UI and do synclock the
> >> adding
> >> of the newrows there should not be any problem.
> >>
> >> The only thing can be that it takes a little bit long, so I would put a
> >> Queue between it.
> >>
> >> Cor
> >>
> >>
> >> "Mike" <my***@pearcey2001.freeserve.co.uk> schreef in bericht
> >> news:1153486602.157945.322210@i3g2000cwc.googlegroups.com...
> >> > Hi,
> >> >
> >> > I am building a windows app. I have two threads that continually run
> >> > in the background pulling information from a website. The data I
> >> > extract from the two separate threads needs to be put into various
> >> > columns of the same datatable (which is defined in a dataset on my form
> >> > in design time). The threads take different amounts of time to cycle
> >> > so the updates to the datatable frequently overlap.
> >> >
> >> > I have run into various issues in the that the datatable frequently
> >> > becomes corrupted by doing this (datatable internal index is
> >> > corrupted:13 message). I am not sure if this is the result of
> >> > simultaneous updates / lockings or trying to access the datatable from
> >> > the worker threads.
> >> >
> >> > I would be grateful if you could let me know the correct way of
> >> > updating this datatable. Should I be passing the returned data back to
> >> > the thread that the datatable is on before trying to update it? If so
> >> > how would I do this? Also, are there any other considerations I am
> >> > missing in terms of the program design?
> >> >
> >> > Thank you very much,
> >> >
> >> > Mike
> >> >
> >