Paintable User Interfaces

It seems like it should be possible to have some kind of GUI library in which user interface elements can be painted onto the screen frame-by-frame, instead of the common practice of assembling an object-oriented hierarchy of widgets.

I’m thinking of something like (rough ECMA-like pseudocode):

static var clicks = 0;

drawRectangle( 0, 0, 100, 100)
drawText( 0, 50, "This button has been clicked " + clicks + " times" )
drawClickListener( 0, 0, 100, 100, function() { clicks++ } )

Now we should be able to wrap this “button” inside affine transformations and clipping operations. We would call and draw the button during each frame of animation, and the button would cease to exist on the first frame in which it stopped being called.

I’m not looking for a revolutionary GUI paradigm; I just want the ability to do some specific, weird effects when the situation calls for it. For example, a car driving by with a clickable button on it’s side. If the button is behind a tree in the same scene, then it shouldn’t be click-able.

In fact, ideally, drawing user interface elements would use the same path as drawing visible shapes, but would pass in an event handler instead of a color or fill pattern. I could even imagine that there would be one or more channels of callback information alongside the RGB channels.

This is in the category of “things that I can’t possibly be the first person to think of but haven’t actually ever seen done.”