large arrays and system.outofmemoryexception

29 Mar 2006 1:05 AM

chad

assuming I have 2GB memory. And 1.5GB available.

dim c1(,) as byte
redim c1(1000, 1024*1024)

throws system.outofmemoryexception. I'm like... huh? Initially I thought a
lot of space is taken up by GC. So, I up my memory to 4GB. Same thing....

So, I read microsoft documents on this type of exception. Doesn't help much.
They were saying to try and keep arrays to 64KB. But, that's not much to work
with.

I suspected that it's gotta do with 'available contiguous memory' and not
just the total available memory. For example,

redim c1(800, 1024*1024)
redim c2(200, 1024*1024)
may work but not...
redim c1(500, 1024*1024)
redim c2(500, 1024*1024)

can someone shed some light on this? Sound like a .net memory oddity. ASP
..net seems to have similar problem (not with arrays though)

29 Mar 2006 5:32 AM

Cor Ligthert [MVP]

Chad,

If I have this kind of problems, than I start to investigate in what kind of
a table will work and in what format not.

However, probably is this what you are looking for.

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/4gt_ram_tuning.asp

I hope this helps,

Cor

Show quoteHide quote

"chad" <c***@discussions.microsoft.com> schreef in bericht
news:3C1F26D9-3237-462E-B6F5-F865D6A0F01A@microsoft.com...
> assuming I have 2GB memory. And 1.5GB available.
>
> dim c1(,) as byte
> redim c1(1000, 1024*1024)
>
> throws system.outofmemoryexception. I'm like... huh? Initially I thought a
> lot of space is taken up by GC. So, I up my memory to 4GB. Same thing....
>
> So, I read microsoft documents on this type of exception. Doesn't help
> much.
> They were saying to try and keep arrays to 64KB. But, that's not much to
> work
> with.
>
> I suspected that it's gotta do with 'available contiguous memory' and not
> just the total available memory. For example,
>
> redim c1(800, 1024*1024)
> redim c2(200, 1024*1024)
> may work but not...
> redim c1(500, 1024*1024)
> redim c2(500, 1024*1024)
>
> can someone shed some light on this? Sound like a .net memory oddity. ASP
> .net seems to have similar problem (not with arrays though)

29 Mar 2006 7:07 AM

chad

hey Cor, thanks for the reply.

> http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/4gt_ram_tuning.asp

Ah, the /3GB thing. Yes, that was the first thing I tried after upping the
memory from 2 to 4GB. Didn't work though. But, I seriously doubt that it's
got anything to do with total memory.

I think that .net just can't handle arrays that size :(

Before I upped the memory, outofmemoryexception crashed out at 800MB of
array size. After upping to 4GB, it's still 800MB. So, I don't think it's the
total physical memory. I investigated page file size too, as well as stopping
unnecessary processes. Same thing. Oh well, sounds like recoding is the way
to go. I'm just curious why it's like this.

Show quoteHide quote

"Cor Ligthert [MVP]" wrote:

> Chad,
>
> If I have this kind of problems, than I start to investigate in what kind of
> a table will work and in what format not.
>
> However, probably is this what you are looking for.
>
> http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/4gt_ram_tuning.asp
>
> I hope this helps,
>
> Cor
>
> "chad" <c***@discussions.microsoft.com> schreef in bericht
> news:3C1F26D9-3237-462E-B6F5-F865D6A0F01A@microsoft.com...
> > assuming I have 2GB memory. And 1.5GB available.
> >
> > dim c1(,) as byte
> > redim c1(1000, 1024*1024)
> >
> > throws system.outofmemoryexception. I'm like... huh? Initially I thought a
> > lot of space is taken up by GC. So, I up my memory to 4GB. Same thing....
> >
> > So, I read microsoft documents on this type of exception. Doesn't help
> > much.
> > They were saying to try and keep arrays to 64KB. But, that's not much to
> > work
> > with.
> >
> > I suspected that it's gotta do with 'available contiguous memory' and not
> > just the total available memory. For example,
> >
> > redim c1(800, 1024*1024)
> > redim c2(200, 1024*1024)
> > may work but not...
> > redim c1(500, 1024*1024)
> > redim c2(500, 1024*1024)
> >
> > can someone shed some light on this? Sound like a .net memory oddity. ASP
> > .net seems to have similar problem (not with arrays though)
>
>
>

29 Mar 2006 10:22 AM

Herfried K. Wagner [MVP]

"chad" <c***@discussions.microsoft.com> schrieb:
> Before I upped the memory, outofmemoryexception crashed out at 800MB of
> array size. After upping to 4GB, it's still 800MB. So, I don't think it's
> the
> total physical memory. I investigated page file size too, as well as
> stopping
> unnecessary processes. Same thing. Oh well, sounds like recoding is the
> way
> to go. I'm just curious why it's like this.

IIRC ~ 800 MB was the per-process memory limit for .NET 1.*, and maybe still
is for .NET 2.0.

--
M S Herfried K. Wagner
M V P <URL:http://dotnet.mvps.org/>
V B <URL:http://classicvb.org/petition/>

29 Mar 2006 1:41 PM

chad

> IIRC ~ 800 MB was the per-process memory limit for .NET 1.*, and maybe still
> is for .NET 2.0.

oh I see. that explains it then :-)
thank you very much.

Show quoteHide quote

"Herfried K. Wagner [MVP]" wrote:

> "chad" <c***@discussions.microsoft.com> schrieb:
> > Before I upped the memory, outofmemoryexception crashed out at 800MB of
> > array size. After upping to 4GB, it's still 800MB. So, I don't think it's
> > the
> > total physical memory. I investigated page file size too, as well as
> > stopping
> > unnecessary processes. Same thing. Oh well, sounds like recoding is the
> > way
> > to go. I'm just curious why it's like this.
>
> IIRC ~ 800 MB was the per-process memory limit for .NET 1.*, and maybe still
> is for .NET 2.0.
>
> --
> M S Herfried K. Wagner
> M V P <URL:http://dotnet.mvps.org/>
> V B <URL:http://classicvb.org/petition/>
>
>

29 Mar 2006 1:54 PM

Herfried K. Wagner [MVP]

"chad" <c***@discussions.microsoft.com> schrieb:
>> IIRC ~ 800 MB was the per-process memory limit for .NET 1.*, and maybe
>> still
>> is for .NET 2.0.
>
> oh I see. that explains it then :-)

Additional information on this topic can be found here (text is in German,
but links reference articles written in English):

<URL:http://groups.google.de/group/microsoft.public.de.german.entwickler.dotnet.csharp/msg/1e14bd8b0886e36c>

--
M S Herfried K. Wagner
M V P <URL:http://dotnet.mvps.org/>
V B <URL:http://classicvb.org/petition/>

30 Mar 2006 2:59 AM

chad

<URL:http://groups.google.de/group/microsoft.public.de.german.entwickler.dotnet.csharp/msg/1e14bd8b0886e36c>

Awesome! Especially the 2nd article. It explains a lot about memory issues.

thanks again. You're the best :-)

Show quoteHide quote

"Herfried K. Wagner [MVP]" wrote:

> "chad" <c***@discussions.microsoft.com> schrieb:
> >> IIRC ~ 800 MB was the per-process memory limit for .NET 1.*, and maybe
> >> still
> >> is for .NET 2.0.
> >
> > oh I see. that explains it then :-)
>
> Additional information on this topic can be found here (text is in German,
> but links reference articles written in English):
>
> <URL:http://groups.google.de/group/microsoft.public.de.german.entwickler.dotnet.csharp/msg/1e14bd8b0886e36c>
>
> --
> M S Herfried K. Wagner
> M V P <URL:http://dotnet.mvps.org/>
> V B <URL:http://classicvb.org/petition/>
>
>