Brian Hempel

Postdoc
The Creativity Lab @ UCSD
bhempel@ucsd.edu
View CV

Can we make interoperability, notebook-like interactions, and end-user programming ubiquitous by lifting the responsibility for data interpretation and interaction to the operating system level?

I am a postdoc working on a GUI+code interface for plotting in Haijun Xia's Creativity Lab at UCSD! This is a stepping stone towards the long-term vision of elevating value-centric interactions up to the operating system level.

For summer/fall 2022, I was a postdoc at CIWRO and the Storm Prediction Center to improve and operationalize my Nadocast severe weather predictor.

In March 2022, I finished my Ph.D. in the PL group at UChicago, advised by Ravi Chugh.

Current long-term research question: Can we make interoperability, notebook-like interactions, and end-user programming ubiquitous by lifting the responsibility for data interpretation and interaction to the operating system level?

Ph.D. research question: Can we augment programming with direct manipulation interactions?

I enjoy discussing programming UIs, programming pain points, and the future of programming. If you have any questions or comments or just want to bounce your crazy ideas around, shoot me an email!

Goals

The main thing I learned from my Ph.D. is that languages aren't the problem, and programming environments aren't either: operating systems are the problem. Unix and friends are decent abstractions and resource managers over the hardware, but are inadequate for facilitating communication between programs and inadequate for user-defined composition of programs. Unix brainwashed us into thinking that a pile of bytes is the correct medium for IPC. Thus, I no longer believe IDE work is the most important frontier for improving the experience of programming—at least not until we get the OS-level issues worked out.

Projects

Maniposynth

(paper) (11min video figure ⬇) (33min presentation ⬇) (slides) (artifact)

The capstone project of my thesis is Maniposynth, a bimodal environment for functional programming. The key question is how to offer non-linear graphical interactions while still representing the program as ordinary, linear code.

Tiny Structure Editors

(paper). (technical supplement) (5min presentation ⬇) (slides)

If bimodal programming becomes more widespread, how can users easily specify a GUI for their custom data types? Tiny Structure Editors automatically transforms the output of toString functions into GUIs for manipulating the original object. The key technical idea is to track dependency provenance of substrings.

Sketch-n-Sketch

(paper) (30sec preview ⬇) (3min video figure ⬇) (19min presentation) (slides) (poster) (artifact)

Sketch-n-Sketch is a direct manipulation GUI for creating programs that output vector graphics, while simultaneously representing the program as always-editable textual code.

Deuce

(paper). (17min presentation) (slides) (artifact)

Instead of manipulation of output as in the other systems here, Deuce explores a streamlined interaction for quickly selecting multiple AST elements and invoking refactorings relevant to those elements.

Defense Presentation

January 2022. Most of the presentation focused on Maniposynth.

Research

Maniposynth: Bimodal Tangible Functional Programming. Brian Hempel, Ravi Chugh. ECOOP 2022. (11min video figure ⬇) (33min presentation ⬇) (slides) (artifact)

Tangible Values with Text: Explorations of Bimodal Programming. Brian Hempel. PhD Dissertation, 2022. (1hr defense presentation ⬇)

ManipML: Towards Bimodal Tangible Functional Programming. Brian Hempel, Ravi Chugh. LIVE 2020. (20min presentation)

Tiny Structure Editors for Low, Low Prices! (Generating GUIs from toString Functions). Brian Hempel, Ravi Chugh. VL/HCC 2020. (IEEE link) (technical supplement) (5min presentation ⬇) (slides)

Sketch-n-Sketch: Output-Directed Programming for SVG. Brian Hempel, Justin Lubin, Ravi Chugh. UIST 2019. (30sec preview ⬇) (3min video figure ⬇) (19min presentation) (slides) (poster) (artifact)

Output-Directed SVG Programming. Brian Hempel. MS Paper 2019.

Deuce: A Lightweight User Interface for Structured Editing. Brian Hempel, Justin Lubin, Grace Lu, Ravi Chugh. ICSE 2018. (17min presentation) (slides)

Semi-Automated SVG Programming via Direct Manipulation. Brian Hempel, Ravi Chugh. UIST 2016. (30sec preview) (5min video figure ⬇) (21min presentation) (slides) (artifact)

Programmatic and Direct Manipulation, Together at Last. Ravi Chugh, Brian Hempel, Mitchell Spradlin, Jacob Albers. PLDI 2016. (22min presentation) (slides) (artifact)

Service

Working Hypotheses

If you have any insight or reaction to any of these, feel free to shoot me an email so we can both be enriched!

Vision

The number of core, orthogonal computational concepts (types) that programmers use is fairly small, probably on the order of about 100-1000. Despite this, because UNIX mandated that everything is a list of bytes, none of these concepts have OS-level support. Instead, each application re-invents its version of those concepts, wasting programmer time and preventing users from composing their own workflows. Elevating these core concepts to have OS-level support for visualization and interactive operation could drastically improve the power of computers and the simplicity of software.

Observations

• In Sketch-n-Sketch, we explored manipulating the output to change the code, but the greater value is already in just making the computation visible. Once the computation is visible, further gains by adding direct manipulation interactions are good but may be comparably minimal. There's two phenomena limiting the usefulness of output-directed transforms. First, outputs calculated as the result of complicated expressions are not easily amenble to direct manipulation because the meaning of the manipulation in a large computation may be ambiguous. This observation suggests that direct manipulation interactions may be limited to more local transformations. The second limitation is related: local transformations are not what makes programming hard, so direct manipulation (at least as currently conceived) is unable to address the most difficult tasks of programming.
• The useful innovation in wire-drawing style visual programming languages (VPLs) is not the wires—they quickly become noisy—but the assumption that the toolbox of available components should be visible and explorable.
• Similarly, should they become successful, the primary benefit of structured editors (projectional editors) will be the co-location of code insights and code tools interspersed with the source code, rather than further away. There's no reason the same tooling can't be integrated in an ordinary (non-projectional) text buffer. But the projectional folks are more likely to discover these interface opportunities because they are reworking program display.
• The text-based shell with program interfaces explained in man pages was an okay idea for the 1970's. It's 2023. The UI could provide a lot more help. Though wires-based VPLs are not appropriate for general programming, such a VPL might make a good shell replacement.
• Interfaces between programs and between computers need strong types in order to promote (a) introspection, (b) composition, and thus (c) reusability. JSON is a slight improvement over an opaque stream of bits—with JSON there is structure to follow and names to explain—but JSON examples are still ambiguous about corner and failure cases.
• The C ABI's conception of type is too weak to facilitate easy and broad composition between programs and languages.
• Similarly, code interfaces could be part of the binary—not a separate header file!
• Heck, code interfaces could be part of a data file itself. (This is all very Smalltalk-like.)

Further thoughts.

Side Projects

• Nadocast.com Tornado prediction via ML post-processing of public weather models. Although, it's not just a side project anymore: it is now one of the experimental guidance products available to the forecasters at the Storm Prediction Center.

Class Projects

A mostly failed experiment attempting to use RNNs to predict a Haskell type signature given the function name and the three prior type signatures in the file. A BS/MS student built on this work—this report was placed online so there was something to cite. Though my experiment failed, I'm still optimistic about the motivation. Writing specs is hard—can we use semantic information in names to suggest specifications which could then be fed to a program synthesizer? Compared to regular code, specifications have the advantage that they are an unordered set of properties. You (the human or computer) can generate them in any order and may add and remove individual items freely (unlike additions and removal of code, which generally necessitates lots of reworking and cleanup). If I were to revisit this project, I'd reformulate the problem to predicting a set of properties rather than whole type signatures. To start, the return type would be a property in the set and the input argument types would each be a a property in the set (effectively, arguments would be treated as a set rather an ordered list). But by treating properties as elements of a set, you're not limited to just predicting types: you might predict properties such as "Is this function is recursive?" or "How long is this function?" or "Does this function preserve the input size?" and all of these other properties, along with the type properties, could form the specification of the function for static checkers or a synthesizer.

One useful bit of the project is a high quality regular expression for splitting variables into words (see identifier_segmentor.py). If you also apply a stemmer afterwards you obtain very useable results like the following:

$ pip2 install nltk
$ python2
>>> from identifier_segmentor import *
>>> print "  ".join(segment2("mkNameG_tcIdKey"))
mk  Name  G  tc  Id  Key
>>> print "  ".join(segment2("unsafeTExpCoerceIdKey"))
unsaf  -e  T  Exp  Coerc  -e  Id  Key
>>> print "  ".join(segment2("M.Map"))
M  -.  Map
>>> print "  ".join(segment2("unsafeShift32R"))
unsaf  -e  Shift  32r
>>> print "  ".join(segment2("castDoubleToWord64Array"))
cast  Doubl  -e  To  Word  64  Array
>>> print "  ".join(segment2("STUArray"))
Stu  Array
>>> print "  ".join(segment2("PrimMVar"))
Prim  M  Var
>>> print "  ".join(segment2("X86.Instr.JumpDest"))
X86  -.  Instr  -.  Jump  Dest
>>> print "  ".join(segment2("Typed.TypedDefinition'"))
Type  -d.  Type  -d  Definit  -ion  ‘

The file above with the stemmer is python but the regex alone should segment correctly in any number of languages.

Ruby:

$ irb
irb(main):004:0> "mkNameG_tcIdKey".scan(/\'|(?:^[^A-Za-z0-9\s\'])?(?:[^a-z\_\s\'\.]+$|[^a-z\_\s\'\.]+[0-9\.]|[^a-z\_\s\'\.]+(?![a-z])|[A-Z][^A-Z0-9\_\s\'\.]+\.?|[^A-Z0-9\_\s\'\.]+\.?)/)
=> ["mk", "Name", "G", "tc", "Id", "Key"]

Javascript:

$ node
> "mkNameG_tcIdKey".match(/\'|(?:^[^A-Za-z0-9\s\'])?(?:[^a-z\_\s\'\.]+$|[^a-z\_\s\'\.]+[0-9\.]|[^a-z\_\s\'\.]+(?![a-z])|[A-Z][^A-Z0-9\_\s\'\.]+\.?|[^A-Z0-9\_\s\'\.]+\.?)/g)
[ 'mk', 'Name', 'G', 'tc', 'Id', 'Key' ]

Just please give me credit if you use my segmentor regex!

Faster Elm 0.18 Compiler

The Elm 0.18 exhaustiveness checker has exponential complexity for certain case statements. The following patched versions of elm-make tame this explosion somewhat and reduce Sketch-n-Sketch build times by 5x (or more). More nominal cases might see some slowdown instead of a speedup.

Replace your elm-make executable with one of the following binaries:

• Mac
• Linux
• Windows

Humanity and the Limits of Computers

Computers can't pray.

Technical progress is not moral progress.

As your loved one dies, would you rather than a robot hold their hand?