Strange rounding issues for small custom units

Strange rounding issues for small custom units - strange behavoir when implementing milli darcies - 消息

#1 已发布: 2017/10/23 23:10:15
Sadneophyte

Sadneophyte

0个赞,共1个帖子。

组: User

I implemented several units to deal with field unit conversions for oil field calculations.

In the oil field we use strange units:

millidarcies (mD),
centipoise (cP), and
barrels/day,

the other units have largely already been implemented by smath: wheee!.

mD is an absurdly small unit which I defined as:















where 295 is

which I hope is alright.

if I make a statement, like


k:=20'mD

"and then query k,

I get results that seem to round to 1 decimal place, ie

2*10^-14 m^2

ohh my goodness that is ~1.5%, which is poor performance compared to all the other awesome features!!

it seems to do this in decimal mode, and not in fraction mode, which sort of makes sense, that the fractions would not be rounded the same way.

Seems like a wierd floating point error.

also the behavior isn't always reproducible. very strange.

thanks for the awesome program!

#2 已发布: 2017/10/24 10:59:27
Jean Giraud

Jean Giraud

983个赞,共6,866个帖子。

组: User

Wrote

... many more Engineering don't come as easily as 2*3=6 m²
You don't have to freak Smath unit system, just like this:



... rather like this

Darcy.PNG
  • 有新帖
  • 无新帖