You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
imshow in 2.1: color resolution depends on outliers? #10567
Comment in that new code:
```
# This may introduce numeric instabilities in very broadly
# scaled data
```
Looks like we might have found that situation. The data is getting scaled
for the purposes of the AGG renderer (somewhat independent of the norm that
was specified). Unfortunately, due to some fun effects with floating point
numbers, that normalization gets very digitized when the range is very
large. I would say that this is unintentional, but I am not sure how we can
fix it, though.
Image interpolation is the bug that just keeps giving!
The chain of bugs and fixes here goes:
while doing 2.0 we noticed that we were interpolating after doing norm + rgb mapping -> leads to colors not in the color map in the final image so we moved the interpolation to be done on the normed, but not color mapped data.
Is there some reason to not apply the norm (but not RGB mapping) first? If I do that, it seems to work for this case. It fails a bunch of the image tests meant to catch the other bugs, but I'm not clear why my version is worse than the test version...
Well, we could do something like pre-clip the data to some suitably large number around the vmin/vmax, but not 17 orders of magnitude. That should preserve both behaviours
Or folks could mask their invalid data instead of passing in huge (I assume) invalid values.
I would not assume that huge data points are automatically invalid. In our application we have values between -1 and 1 in most of the image, but it can include regions with values approaching Inf.
I would find it strange to mask some of the data before plotting them. I'm also not aware that you have to do something like this in Matlab or gnuplot.
Activity
dstansby commentedon Feb 22, 2018
Hmm, I'm not sure about whether this is intended or not, it bisects to 12c27f3 so maybe @tacaswell will know more?
WeatherGod commentedon Feb 22, 2018
jklymak commentedon Feb 22, 2018
We messed around w/ all of this for 2.1.2. #10133 W/o looking carefully, does that fix this?
jklymak commentedon Feb 22, 2018
Pretty sure it does - it was a round off error that caused this, but its now been fixed for large floats.
dstansby commentedon Feb 22, 2018
It didn't seem fixed when I ran the above test on the master branch.
jklymak commentedon Feb 22, 2018
agreed. The image is stored as float64. You lose resolution at 2^53=9e15. I’m not sure how this worked in 2.1, but that’s the problem here.
WeatherGod commentedon Feb 22, 2018
jklymak commentedon Feb 22, 2018
I tried to set the dtype to float 128 fro big numbers, but I was told there is no such thing...
tacaswell commentedon Feb 23, 2018
Image interpolation is the bug that just keeps giving!
The chain of bugs and fixes here goes:
jklymak commentedon Feb 23, 2018
It’s the same bug, it’s just that #10133 only fixed up to 10^16 or so.
jklymak commentedon Feb 23, 2018
Is there some reason to not apply the norm (but not RGB mapping) first? If I do that, it seems to work for this case. It fails a bunch of the image tests meant to catch the other bugs, but I'm not clear why my version is worse than the test version...
Test
Moving the norm step up (pre interp)
tacaswell commentedon Feb 23, 2018
The ringing / saturation in the lower left of the test is correct. If you normalize first you are effectively clipping the impact of the outliers.
jklymak commentedon Feb 23, 2018
Well, we could do something like pre-clip the data to some suitably large number around the vmin/vmax, but not 17 orders of magnitude. That should preserve both behaviours
Or folks could mask their invalid data instead of passing in huge (I assume) invalid values.
hagenw commentedon Feb 23, 2018
I would not assume that huge data points are automatically invalid. In our application we have values between -1 and 1 in most of the image, but it can include regions with values approaching Inf.
I would find it strange to mask some of the data before plotting them. I'm also not aware that you have to do something like this in Matlab or gnuplot.
jklymak commentedon Feb 26, 2018
See #10613 for a fix....
tacaswell commentedon Mar 3, 2018
closed by #10613