R's isoreg function generating more knots than unique fitted values
R's isoreg function generating more knots than unique fitted values
I have been working with R's isoreg function and have experienced a problem: the function is generating more knots than unique fitted values.
From the R help,
iKnots [is an] integer vector giving indices where the fitted curve jumps, i.e., where the convex minorant has kinks
I believe I have an idea about the cause of the problem, and I have a reproducible example:
# Demonstrating the problem
set.seed(100)
x<-runif(88000,0,1)
x<-x[order(x)]
y<- c(rep(c(0.1000001,0.1000000),11000),rep(c(0.1000002,0.1000003),11000),rep(c(0.2000002,0.2000003),11000),rep(1,22000))
plot(test<-isoreg(x,y))
length(unique(test$yf))
length(test$iKnots)
# Evidence of a floating point arithmetic problem
unique(test$yf)
print(c(unique(test$yf)[1],unique(test$yf)[2]),digits=18)
unique(test$yf)[1]==unique(test$yf)[2]
print(c(unique(test$yf)[4],unique(test$yf)[5]),digits=18)
unique(test$yf)[4]==unique(test$yf)[5]}
Here is the plot produced by this example:
You can see that R's isoreg function is identifying many more knots than it should (where there are a lot of red Xs in the plot). However at other places, it correctly uses only 2 knots (the black lines).
It is clear that the problem is connected to R's floating point arithmetic. I also note that isoreg uses .Call
to call a C routine to actually do the isotonic regression, so perhaps the problem lies with differences between the C and R languages.
.Call
I am using isoreg to calibrate model probabilities, and I would like to be as precise as possible. Therefore, I have 2 questions:
1) Is there some way I could alter the x and y variables used in the isoreg function to avoid this problem while maintaining as high precision as possible?
2) I can manually find the unique fitted values and the respective knots. However, is this ok? Can I assume that the algorithm found the best fit or could this problem invalidate that assumption?
By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.