Link Search Menu Expand Document

How do you create an interaction semantics?

Programming semantics are well-studied. What about direct manipulation semantics? Can they be in bijection?

I'm especially interested in, say, reflecting mathematical semantics in interactive diagrams. With Penrose, we've been asking, eg, what dragging a point outside a circle in a venn diagram means mathematically - it's an "edit" in the mathematical construction.

This is true for other domains that have a formal representation underlying a highly interactive one.

(UI Continuations)

Expressive Query Construction through Direct Manipulation of Nested Relational Results

Karger's lab has a disproportionate number of these kinds of papers!


(Fructure. BTW I'm not just talking about program editors, they're just a convenient example to point to.)

Direct manipulation (mouse, not keyboard)

What even is a program? You can make great Smalltalk (System Archaeology Seminar) programs without writing much code at all. I'm not talking about no-code/future-code, but rather, writing a trace of behavior by live (not premeditated) interactions on objects.

  • Self Language (and other Morphic environments) turns green empty "morphs" into rich interactive objects just through right-clicking

Structure editing (keyboard, not mouse)

Programming by demonstration


  1. Algebraic Interfaces

Page last modified: 11/23/2021.