Problem

One limiting factor of interactivity in Matplotlib is that we draw almost all visual elements via Artists onto the figure. This has poor performance for fast changing elements like crosshair cursors, because we always have to redraw the complete figure (or try to fiddle with blitting).

Proposed solution

The fundamental solution idea here is to pull such elements out of the actual figure and instead overlay them on the not-changing figure image.

This is currently in a very early idea stage.

What can be achieved with this
Interactivity with fast changing elments such as:

• Zoom box (already implemented as a special case)
• Crosshair cursors (see also PoC: GUI-native crosshair cursor #30516).
• Tooltips
• Selectors

What still won't be possible
Dynamically changing plot properties like greying out data depending on a selector like https://docs.bokeh.org/en/latest/docs/examples/interaction/linking/linked_brushing.html.

What would be needed

• A concept of overlays. This could be:
  a) GUI specific, e.g. we define a set of native primitives (lines, rectangles, maybe semantic ones like CrosshairCursor, similar to artists) that are drawn by the canvas on top of the actual figure image. This requires implementing of the primitives in all backends and will provide the fastest rendering.
  b) Create the concept of an "overlay figure" that uses our exising rendering mechanisms to create a (mostly transparent) full image that is drawn on top of the existing figure image.

• Some mapping and event logic, so e.g. that the overlay can draw a line across a complete Axes, or get tooltip information for the current coordinate.