Linear Algebra as Spreadsheet

Thanks to Garth Goldwater for a lot of the ideas here

The metaphor isn't mine. But it's a good one!

Is there an intrinsic reason MATLAB-esque tasks can't be as WYSIWYG as a spreadsheet?

How do you represent matrix multiplication in a graphical environment? I want to drag and drop my eigendecomposition as tangibly as a sort operation.
— Max Krieger (@maxkriegers) April 5, 2020

Some tools on the way to supporting the magical, playful mode of waves:https://t.co/fa5Sunkwhh https://t.co/FQKTRWzyje
and the unimplemented direct manipulation APL-spreadsheet-incantation environment of my dreams https://t.co/TFxhVS1UIq
— Max Krieger (@maxkriegers) September 16, 2020

So I was wondering why nobody took it literally.

When you're manipulating matrices, you [often] lose track of, "what am I doing? what does this mean?" I've got this funny matrix, I multiply it by a vector, what was this supposed to do?

Keenan Crane, Discrete Exterior Calculus

I can't find any existing direct manipulation interfaces to perform linear algebra on raw data. Stuff like eigendecomposition or factorization. Or constructing proofs of SVD.

The key, I think, is to map the language of linear algebra to an interaction semantics for linear algebra (How do you create an interaction semantics?).

Here's what I have found:

Excel!?

Sure I guess?

Performing Operations with Matrices on Spreadsheets

APL Is Really Cool

Aesthetics

"Spectral decomposition? Oh you mean like when a ghost dies?"

other

What if there was a set of flashcards to learn all the APL symbols? Like Elixir Cards/PixelSpirit.

One subproject (Downscoping as a skill (subprojecting)) is not APL, but some CAS matrix toy - "Linearkit".

Nicky Case's transformation matrix toy

Direct manipulation refs

isolate a data column by dragging it out of the table. also got a little semantic text editing going pic.twitter.com/ZW6ukuk2UK
— Paul Shen (@_paulshen) March 2, 2021

WebGL State Diagram - the "cells" are pointers
- Maybe each matrix is a floating table and you can multiply by constructing a graph of edges?
- Graphical Linear Algebra
Graspable Math
Responsive Matrix Cells: A Focus+Context Approach for Multivariate Graphs
Loglo
Math Inspector
- Math inspector makes it easy for anyone to leverage the power of numpy, the most popular computational mathematics library in the world, without needing to know anything about programming or writing any code. “If watching math videos is like going to the movies instead of reading a book, then math inspector is like playing a video game instead of doing your homework.”
Interactivity of Visual Mathematical Representations: Factors Affecting Learning and Cognitive Processes (pdf)

Forms/3

Semantics refs

Every right click transformation (eg transpose, get eigens) is not in-place but is placed somewhere else in space, immutable copy, like Paul Shen's systems.

Applications

Learning, proving, playing
graphics pipelines (in 2d case of env layout)
statistics
DSP (vision - convolutions, audio)
ML

Why?

Not sure.
- Might be internalized trauma taking linear algebra without something that feels nice to use (Domain-Learning with Computational Microworlds).
- Practitioners make numpy mistakes a lot I bet
- How do you communicate intuition about powerful ideas in linalg? Are pictures the best we have? What about the algebra, and the data embedded in every matrix?
The semantics of Linear Algebra are pretty well-defined I think? That's why it's an algebra, right?
I want to do casual mathematics
Linear Algebra can model a lot of other domains of math. Not to mention it's pretty useful for physics. So maybe the interface paradigm can be juiced into domain-specific applications (On Juicing, Bottom Up Design).