Is that done in some environments? I think Maxima gives answers as fractions by default, but didn't know how that worked. And my Casio calculator can stick to fractions and surds up to a point, but I think it's just using float under the surface and converting back ok until too much rounding error creeps in...
think Maxima gives answers as fractions by default, but didn’t know how that worked.
It's likely based on the theory of continued fractions and the calculator has a limit to the maximum denominator allowed to return an approximation of the fraction.
The coolest part is you can do log math since you have the mantissa and exponent already for extremely fast multiplication and division (+ or - log values)
6 Comments
Semi_Hemi_Demigod@lemmy.world · 8 pts · 32d
Just store the fraction as a varchar like “11/17”, duh
Sadbutdru@sopuli.xyz · 3 pts · 32d
Is that done in some environments? I think Maxima gives answers as fractions by default, but didn't know how that worked. And my Casio calculator can stick to fractions and surds up to a point, but I think it's just using float under the surface and converting back ok until too much rounding error creeps in...
bamboo@lemmy.blahaj.zone · 3 pts · 32d
It's likely based on the theory of continued fractions and the calculator has a limit to the maximum denominator allowed to return an approximation of the fraction.
Semi_Hemi_Demigod@lemmy.world · 2 pts · 32d
I’ve written econmerce systems and to avoid floating point errors I’d just use ints and divide by 100 on the front end.
iocase@lemmy.zip · 4 pts · 32d
The coolest part is you can do log math since you have the mantissa and exponent already for extremely fast multiplication and division (+ or - log values)
far_university1990@reddthat.com · 2 pts · 32d
Fast inverse square root go brrrr.