Modifying colormaps (as used in xarray)

@dcherian

xarray's plotting functionality sometimes needs to turn a colormap into a discrete colormap. I came across this in fixing a bug pydata/xarray#3601 where xarray was ignoring the effects of set_under(), etc., which had been called on the colormap. We've fixed that problem, but ended up with this https://github.com/pydata/xarray/blob/ae6609b025d62a0d863869cbd16db167a8855167/xarray/plot/utils.py#L94-L105

    n_colors = len(levels) + ext_n - 1
    pal = _color_palette(cmap, n_colors)

    new_cmap, cnorm = mpl.colors.from_levels_and_colors(levels, pal, extend=extend)
    # copy the old cmap name, for easier testing
    new_cmap.name = getattr(cmap, "name", cmap)

    # copy colors to use for bad, under, and over values in case they have been set to
    # non-default values
    new_cmap._rgba_bad = getattr(cmap, "_rgba_bad", new_cmap._rgba_bad)
    new_cmap._rgba_under = getattr(cmap, "_rgba_under", new_cmap._rgba_under)
    new_cmap._rgba_over = getattr(cmap, "_rgba_over", new_cmap._rgba_over)

This works, but is not particularly elegant, and if there are any other properties of the original cmap that we want to keep, we'd have to copy them over explicitly. @dcherian asked me to see if anyone on here could suggest a better way of modifying a cmap into a discrete cmap?

I guess its a fair question whether we should have ever provided from_levels_and_colors in the first place, since we don't use it internally. If we think we are keeping it, I'm sure a PR would be welcomed to extend the functionality to include over/under.

from_levels_and_colors takes in colors, not a colormap. Hence it has no use of over/under/bad. Of course it could take those in as arguments as well, but that does not help with the original problem here.

Potential actionable things here are:

Should over/under/bad exposed, e.g. via .get_{over/under/bad}?
Should there be a public version of Colormap.resample that returns a copy of the original colormap, just with a different number of colors?

Worth mentionning that matplotlib does not even take over/under/bad too seriously itself. E.g. neither the .reversed(), nor the ._resample() method take those into account, as seen from the below:

import numpy as np
import matplotlib.pyplot as plt

a = np.ma.array([[1, 2], [3, 1]], mask=[[False, False],[False, True]])

cmap = plt.get_cmap("cividis")
cmap.set_over("red")
cmap.set_bad("orange")

fig, (ax1, ax2) = plt.subplots(ncols=2, figsize=(8,4))

im1 = ax1.imshow(a, cmap=cmap, vmin=1, vmax=2)
fig.colorbar(im1, ax=ax1, extend="both")

im2 = ax2.imshow(a, cmap=cmap.reversed(), vmin=1, vmax=2)
fig.colorbar(im2, ax=ax2, extend="both")

ax1.set_title("cmap, over=red, bad=orange")
ax2.set_title("cmap.reversed()")
plt.show()

For the getters, see recent discussion in #16075.

Should there be a public version of Colormap.resample that returns a copy of the original colormap, just with a different number of colors?

Isn't that get_cmap('name', lut=count)?

No, because that gives a new colormap without the original over/under/bad settings.

cmap = plt.get_cmap("cividis")
cmap.set_over("red")
print(cmap(1.1))              #  prints (1.0, 0.0, 0.0, 1.0) which is "red"

cmap2 = plt.get_cmap("cividis")
print(cmap2(1.1))             #  prints (1.0, 0.0, 0.0, 1.0) which is "red"

cmap3 = plt.get_cmap("cividis", lut=5)
print(cmap3(1.1))             #  prints (0.995737, 0.909344, 0.217772, 1.0)
                              #  which is the last color of cividis

It's also not really desired, since there needs to be a way to get the inbuilt colormap back even after modifications.

Your second line looks like a bug to me. Why would setting over be sticky like that to the colormap name?

Builtin colormaps are created once and stored globally in cmap_d.

get_cmap just returns the element from the dictionary (probably assuming that it's immutable). But in fact, you get the builtin reference colormap object and can modify that using set_over().

Proposal for changes

IMHO this needs some cleanup, which will include breaking changes (but OTOH I expect that not many people would be affected by the breaking).

Should cmaps be immutable (including over/under/bad)? I tend to say yes to keep it simple.
get_cmap() should always return the builtin colormap. Whether that can be the same object everytime or a copy depends on 1.
over/under/bad should be treated consistently (propagated via reverse, resample).
add a public .resampled() method. Deprecate get_cmap(..., lut=...).

@timhoffm

I am 👍 on all of @timhoffm 's proposals. For 1) we should go with an API like named tuple of new_cmap = cmap.replace(over=, under=, bad=).

We will definitely need a transition period and a public way to set a replacement color map for a built in one. I am sure that we have significant use of

plt.get_cmap('viridis').set_under('k')
plt.imshow(data, cmap='viridis')

The replace() API is basically implemented (under a different name) at #14645 ;-)

I have definitely messed myself up before by accidentally setting an under/over value somewhere early-on in a Jupyter notebook and been very confused why the cmaps I was using later were looking different than the builtin only to realize I'd made this very mistake! I'd be all for returning a copy of the colormaps when calling get_cmap rather than a pointer to the object.

I see that I have now argued both sides of this (here being in favor of changing the behavior and in #16943 comments against changing the behavior).

This issue has been marked "inactive" because it has been 365 days since the last comment. If this issue is still present in recent Matplotlib releases, or the feature request is still wanted, please leave a comment and this label will be removed. If there are no updates in another 30 days, this issue will be automatically closed, but you are free to re-open or create a new issue if needed. We value issue reports, and this procedure is meant to help us resurface and prioritize issues that have not been addressed yet, not make them disappear. Thanks for your help!

Most of the above aspects are implemented by now:

Colormap.set_extremes() / Colormap.with_extremes()
Returning a copy from the colormap registry
Colormap.resampled(n)

The only open aspect is immutable colormaps. It would have been nice to have immutability designed in from the start. But that's not the world we're in. I'm inclined to say that enforcing immutability now would be too much of an API change compared to what we gain. The above changes already make it much more unlikely that changing an existing colormap will have unwanted side effects.

I therefore close this issue as done. If there was still need for a discussion on immutable colormaps, please open a new ticket for that.

Navigation Menu

Repository navigation

Modifying colormaps (as used in xarray) #16296

Proposal for changes

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Modifying colormaps (as used in xarray) #16296

Description

Activity

jklymak commented on Jan 22, 2020

ImportanceOfBeingErnest commented on Jan 22, 2020

anntzer commented on Jan 25, 2020

QuLogic commented on Mar 10, 2020

ImportanceOfBeingErnest commented on Mar 10, 2020

jklymak commented on Mar 10, 2020

timhoffm commented on Mar 10, 2020

Proposal for changes

tacaswell commented on Mar 10, 2020

anntzer commented on Mar 10, 2020

greglucas commented on Mar 27, 2020

tacaswell commented on Mar 28, 2020

github-actions commented on Jul 7, 2023

timhoffm commented on Jul 10, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions