Flo's Blog

Notes on Magic Ink Bret Victors Timeless Insights

One of the greatest user interface design minds in the world today - Alan Kay (About the author of magic ink)

I first encountered Bret Victor through the Future of Coding community and his influential talk "Inventing on Principle". His work challenges conventional thinking about software and user interfaces. Victor's work draws comparisons to that of Alan Kay, with Victor being referred to as Kay's protégé. In his career, he worked as part of the 'HID Proto' future-interface prototyping team at Apple, where he designed UI concepts for experimental hardware platforms.

Before his work at Apple, in 2006 he wrote a paper on a new approach to user interface design: "Magic Ink: Information Software and the Graphical Interface". This paper stemmed from his belief that "software has remained frustrating, and as the importance of software to society has grown, so too has the crisis. The crusade marches on, with believers rarely questioning the sacred premise—that software must be interactive in the first place. That software is meant to be 'used.'" It showed me a new perspective on UI I hadn't seen before. The article remains remarkably relevant, as it predicted recent trends - the idea of software anticipating user needs is now being realised through AI and machine learning.

'Magic Ink' in a Nutshell: Victor's Core Points

While Victor provides his own summary at the end of the article, this is my personal interpretation of his key points for learning purposes.

Victor begins by reframing software design as a combination of "drawing pictures" (graphic design) and creating "things to push" (industrial design). He then identifies three primary reasons people use software: to learn, create, and communicate. This leads to his classification of software:

That most software is a mix of information and manipulation software and also communication is clearly acknowledged: "Although manipulation is the focus, good manipulation software must provide superb visualization as well."^2.

Moreover, he argues that manipulation software is the hardest to design. Yet, most software is information software. Thus, most software is not a machine but a medium for visual communication and that interaction should be a last resort when designing software. He defines the design of information software as the design of “context-sensitive information graphics”. The introduction of context is another key idea of his, which is ubiquitous since mobile phones took over: "Print has one supreme flaw: ink is indelible. An ink-and-paper design is static—it must display all its data, all the time." He lays out how software back then (the article is from 2005) could be improved by utilizing context (environment, history, interaction). He was even suggesting using machine learning to infer historical context! Victor gives multiple examples in his text while being most into details with his widget BART, which is a train trip scheduler. For the detailed comparison, please refer to the original paper.

His ideas like context-sensitive information graphics have been realised to a great extent. For example, Google Maps and other map applications can accurately guess the location you will likely explore by using your device's GPS. The minimizing interaction aspect has become a core concern when designing software^1. On the other hand, there is room for improvement in many applications. For instance, his critique of email applications remains relevant even today:

Most current designs revolve around the manipulation of individual messages—reading them one-by-one, searching them, sorting them, filing them, deleting them. But the purpose of reading email has nothing to do with the messages themselves. I read email to keep a complex set of mental understandings up-to-date—the statuses of personal conversations, of projects at work, of invitations and appointments and business transactions and packages in the mail. That this information happens to be parceled out in timestamped chunks of text is an implementation detail of the communication process. It is not necessarily a good way to present the information to a learner.

The email apps I know that move slightly more in Victor's direction are Gmail, with its categorization of messages ("Primary," "Social," and "Promotions"), and Superhuman, which automatically triages emails.

Glossary

Quotes

Software and Sorcery

A computational process is indeed much like a sorcerer’s idea of a spirit. It cannot be seen or touched. It is not composed of matter at all. However, it is very real. It can perform intellectual work. It can answer questions. It can affect the world by disbursing money at a bank or by controlling a robot arm in a factory. The programs we use to conjure processes are like a sorcerer’s spells.

Abelson and Sussman, Structure and Interpretation of Computer Programs (1984)

The Ideal of a PC as seen by Licklider, Bush, Engelbart

For Licklider and other early visionaries such as Vanevar Bush and Doug Engelbart,** See Bush’s paper “As We May Think” (1945) and Engelbart’s paper “Augmenting Human Intellect” (1962). the ideal of the then-hypothetical personal computer was a brain supplement, enhancing human memory and amplifying human reasoning through data visualization and automated analysis. Their primary concern was how a machine could help a person find and understand relevant knowledge. Although they were generally discussing scientific and professional work, their prescience fully applies in the modern home.

Magic Ink

Liberating us from the permanence of publication is the undersung crux of the computer—the dynamic display screen. Its pixels are magic ink—capable of absorbing their context and reflecting a unique story for every reader. And the components surrounding the display—CPU, storage, network, input devices—are its peripherals for inferring context.

Design of Information Software as Graphic Design Project

I suggest that the design of information software should be approached initially and primarily as a graphic design project. […] The designer must start by considering what the software looks like, because the user is using it to learn, and she learns by looking at it.

Interaction harmful

Unless it is enjoyable or educational in and of itself, interaction is an essentially negative aspect of information software. […] Interaction should be used judiciously and sparingly, only when the environment and history provide insufficient context to construct an acceptable graphic.

Good Information Software

Good information software reflects how humans, not computers, deal with information.

Information Software as Art

The hero of tomorrow is not the next Steve Wozniak, but the next William Playfair. An artist who redefines how people learn. An artist who paints with magic ink.

Settling for a better standard

People constantly settle for ugly, clunky software, but demand informative, professionally-designed books, newspapers, magazines, and—ironically—brochures, ads, and manuals for that very software.

Random Thoughts & Ideas

References From the Paper