Strange rounding issues for small custom units

Strange rounding issues for small custom units - strange behavoir when implementing milli darcies - Сообщения

#1 Опубликовано: 23.10.2017 23:10:15
Sadneophyte

Sadneophyte

0 сообщений из 1 понравились пользователям.

Группа: User

I implemented several units to deal with field unit conversions for oil field calculations.

In the oil field we use strange units:

millidarcies (mD),
centipoise (cP), and
barrels/day,

the other units have largely already been implemented by smath: wheee!.

mD is an absurdly small unit which I defined as:















where 295 is

which I hope is alright.

if I make a statement, like


k:=20'mD

"and then query k,

I get results that seem to round to 1 decimal place, ie

2*10^-14 m^2

ohh my goodness that is ~1.5%, which is poor performance compared to all the other awesome features!!

it seems to do this in decimal mode, and not in fraction mode, which sort of makes sense, that the fractions would not be rounded the same way.

Seems like a wierd floating point error.

also the behavior isn't always reproducible. very strange.

thanks for the awesome program!

#2 Опубликовано: 24.10.2017 10:59:27
Jean Giraud

Jean Giraud

983 сообщений из 6866 понравились пользователям.

Группа: User

Wrote

... many more Engineering don't come as easily as 2*3=6 m²
You don't have to freak Smath unit system, just like this:



... rather like this

Darcy.PNG
  • Новые сообщения Новые сообщения
  • Нет новых сообщений Нет новых сообщений