|
web
newsgroups
|
|||||||||||||||||||||||
|
|||||||||||||||||||||||
i can't roundI can't round my numbers. i have a single, and i'm trying to round it so that
there's only one decimal place. here's what i've tried: ..Val = CStr(Format(sngTimeToTarget, "#0.0")) and i also tried ..Val = CStr(round(CDbl(sngTimeToTarget), 1)) neither of which do anything. i've even tried multiplying by 10, coercing to an integer, coercing back to a single, and then dividing by 10 - this will cause an overflow arithmatic error. what am i doing wrong?
Show quote
Hide quote
"Paul" <P***@discussions.microsoft.com> wrote in message fwiw, that looks an awful lot like VB6 code (if it is, this is the wrong news:0D620DBD-1F99-40D3-A42A-2631BC6D01EF@microsoft.com... >I can't round my numbers. i have a single, and i'm trying to round it so >that > there's only one decimal place. here's what i've tried: > > .Val = CStr(Format(sngTimeToTarget, "#0.0")) > and i also tried > .Val = CStr(round(CDbl(sngTimeToTarget), 1)) > > neither of which do anything. i've even tried multiplying by 10, coercing > to > an integer, coercing back to a single, and then dividing by 10 - this will > cause an overflow arithmatic error. > > what am i doing wrong? group).... the code below will show a random number between 0 and 100 with 2 decimal places. Note that Round uses bankers rounding (which explains why my checks bounce <g>) Format uses the rounding I learned in school... that is, any decimal greater than 5 is rounded up, otherwise, rounded down. '=== Private Sub Command1_Click() Debug.Print Format$(Rnd * 100, "0.00") End Sub '=== Also note that, in VB, or any other language, you should avoid using Keywords for property names. Val is definitely a Keyword. -- Ken Halter - MS-MVP-VB (visiting from VB6 world) - http://www.vbsight.com Please keep all discussions in the groups.. "Paul" <P***@discussions.microsoft.com> wrote in message How about:news:0D620DBD-1F99-40D3-A42A-2631BC6D01EF@microsoft.com... >I can't round my numbers. i have a single, and i'm trying to round it so >that > there's only one decimal place. here's what i've tried: > > .Val = CStr(Format(sngTimeToTarget, "#0.0")) > and i also tried > .Val = CStr(round(CDbl(sngTimeToTarget), 1)) ..Val=Math.Round(CDbl(sngTimeToTarget),1).ToString("#0.0") Watch out though, the Math.Round by default assumes you want accounting rounding which is different from the rounding we learned in grade school. In accounting rounding, 1.5 is rounded to 2 whereas .5 is rounded to 0. The behavior of this method follows IEEE Standard 754, section 4. This kind of rounding is sometimes called rounding to nearest, or banker's rounding. If you want the traditional rounding, you must add an additional parameter to the Round method as follows: ..Val = Math.Round(CDbl(sngTimeToTarget),1,MidpointRounding.AwayFromZero).ToString("#0.0") Jim Wooley Paul wrote:
> I can't round my numbers. i have a single, and i'm trying to round it \\\> so that there's only one decimal place. .Val = Math.Round(sngTimeToTarget, 1) /// Math.Round returns a Double, so if your .Val property is expecting a Single you'll need to convert it: \\\ .Val = CSng(Math.Round(sngTimeToTarget, 1)) /// HTH, -- (O)enone using round and format in either of those ways doesn't work. i'm sure i've
got my conversions right. the weird thing is that they do work in two other instances of this custom control, but in the third they don't. the program runs in vb.net (2003), but it was converted from vb6 by someone other then myself. i'm learning both in order to take his place. Paul wrote:
> using round and format in either of those ways doesn't work. i'm sure I can only think to say that Round and Format are working (they definitely > i've got my conversions right. the weird thing is that they do work > in two other instances of this custom control, but in the third they > don't. do work), it has to be something else in your code that's not doing what you want. If you try displaying the results in a MessageBox, does it display the right value there? Have you tried stepping into the .Val property assignment to see what's happening internally? -- (O)enone
Web activation
Using Classes as List Items date/time fields SystemIndexOutOfRangeException error CheckForIllegalCrossThreadCalls ? Conditional statements File upload to website and rezise at the same time using Generics dynamically (?) ADO.NET 2.0 TableAdapter Configuration Wizard FileSystemWatcher UNC Path Invalid |
|||||||||||||||||||||||