no. IEEE splits the power of two exponent from the base 2 mantissa. Yes, the exponents are added during a multiply, but the mantissas do an ordinary multiply.
The idea is rather than storing a number x as (exponent, mantissa), just store (log x) as a fixed precision number. Multiplying two such numbers is just addition, dividing is just subtraction. TBH I didn't read the article, but my reaction is that yes, that works, but one must sum all those products, and now summing becomes an expensive operation. Maybe the total cost saves area and power, but it beggars belief that it is 10x more efficient. They must be doing PR math: our low precision log scheme is 10x more efficient than a higher precision traditional approach.
Another thing to keep in mind is a lot of inference is done using very low precision math and so the cost of doing multiplies isn't that bad. Yes, it is still (n bits) squared, but as n gets small, n^2 still isn't too bad.
This is essentially the way many 8 bit games did 3d rendering ( for example the world famous Elite )... you just need two tables, one linear2log and another log2linear, with careful measurement of the ranges and number of elements needed in you application ( which is easy for inference ).
That’s not standard practice for USA tech companies.
A WARN notice includes text like “such-and-such org headcount to be reduced by 36 by Sep 11,” and doesn’t include specific names.
Google bricks your MacBook and revokes your corporate account access the moment they make the decision. You may be notified via personal email if you properly set that up in Workday, otherwise you will have to ask your manager via off-corp channels what happened.
normalize the two numbers A and B to have the same exponent, add the mantissa, then convert back to IEEE 754?
reply