Menu

#2 Spurious roots apparently due to use of int types

open
nobody
None
5
2004-10-18
2004-10-18
No

When I draw a function graph of y = x^3 with bounds XMin = -15,
XMax = 15, YMin = -10, YMax = 10, I see the function I expect to
see in the middle, and then six near-vertical lines at the edges.
Turning on the tracing and investigating these lines I find that y is
positive on both sides, and that they occur at the points where y =
1024 N (for integral N). It appears that the function in evaluated in
a floating point type and then coerced into an integral type for
display, but that that coercion is lacking some overflow checks.

Discussion


Log in to post a comment.

MongoDB Logo MongoDB