Alan Kay’s interest in computers began during a two-week programming course by IBM as part of his Air Force training. Soon after, in 1966, he finished his Bachelor’s degree at the University of Colorado and entered into a Computer Science Masters program at the University of Utah, where he also undertook his doctorate[TTC]. During the Masters he was introduced to Ivan Sutherland’s 1963 PhD thesis on Sketchpad, a pioneering computational drawing system[HCIH5]. Two years later Sutherland joined the computer graphics lab at UoU and Kay worked under him.
Sketchpad was the first realisation of an interactive graphical computer system, a precursor to MacPaint, leading to technology such as Adobe’s Illustrator. It used a device called a light pen to control the screen, as the computer mouse had not yet been introduced by Englebart. Sutherland was the first person to investigate digital display beyond text, but his program dealt only with drawing on the screen and not with any symbolic or icon-like representations[IC13]. However, this exposure the graphical possibilities of computers stuck with Kay and became a cornerstone of his later work.
In 1968 he completed his doctoral project ‘the FLEX machine’ alongside Ed Cheadle, incrementing upon contemporary work with a high-resolution display for animated graphics and text. He later described it as having a user interface but ‘one that repelled end users instead of drawing them closer’[UIAPV123], he also said that that ‘much of the hardware and software was successful from the standpoint of computer science state-of-the-art research, but lacked sufficient expressive power to be useful to an ordinary user’[NMR394], but it was the seeds of what became the Dynabook.
Soon after, he read Marshal McLuhan’s seminal 1964 book Understanding Media and had an epiphany about the purpose of the computers he was designing[UIAPV124].
Marshal McLuhan popularised the field of Media Theory with Understanding Media and the 1967 experimental publication The Medium is the Massage. McLuhan postulates that is it the medium of communication that matters, and that the content is incidental. In the first part of Understanding Media he differentiates between ‘hot’ and ‘cold’ media by the amount of participation required – ‘hot’ media such as film demands the viewer’s attention and can be passively consumed, contrasted with a ‘cold’ medium like books, which require much more conscious effort on behalf of the reader to be understood[UM?]. He espoused that it was not the type of programming on the TV or the subject of novels that was important to track cultural change, it was the modes of consumption.
McLuhan is said to have predicted the internet[findRef]. He perceived the pervasive rise of electronic media as an era-shifting cultural event, describing a ‘global village’ – geography becomes irrelevant as information can travel instantly from anywhere to anywhere[findRef].
In an electric information environment, minority groups can no longer be contained—ignored. Too many people know too much about each other. Our new environment compels commitment and participation. We have become irrevocably involved with, and responsible for, each other.[MITM24]
In an essay Alan Kay wrote in 1989 he described his understanding of McLuhan’s thesis – ‘when he said “the medium is the message” he meant that you have to become the medium if you use it’[UIAPV124]. Kay was saying that to glean meaning from a medium you have to truly understand the medium and then be able to subtract the medium from the content. He allegorises this by saying that ‘it is in the nature of tools and man that learning to use tools reshapes us’[UIAPV124]. Kay had internalised that there is an inherent symbiosis between humans and anything humans use for purpose. It is with this revelation that he mentally re-classified the computer as a medium rather than a mere tool.
McLuhan theorised that the invention of the Guttenburg Press had such great cultural impact not due to its physical ability to produce books, but to its ability to change readers’ thought patterns[findRef]. Kay subscribed to this notion and believed that the same would be true for the rise of the personal computer. The paradigm of ‘computer as medium’ brings with it much greater social, political and cultural responsibility than ‘computer as tool’, and Kay recognised that decisions made in the early days of PC technology would massively propagate into the societies and cultures of the future, where computers would, inevitably, be ubiquitous.
By 1968, Seymour Papert had been working with renowned developmental psychologist Jean Piaget for five years[TTC], and had in 1967 become co-director of MIT’s Artificial Intelligence Laboratory alongside Marvin Minsky, with whom he went on to extensively collaborate[findRef]. Papert was the first person to explore how computers can be used for teaching, specifically for children.
As Papert explains in his 1980 book Mindstorms: Children, Computers, and Powerful Ideas, traditionally education has been based on providing information, exercises that challenge, and feedback. Papert recognised that the same model had been adopted for the use of teaching children about computers in classrooms, inasmuch as classrooms had computers at the time. He believed that computers had the ability to affect the way people think and learn, and that education using them should reflect that[MS3].
Piaget theorised that children develop different intellectual abilities in a specific order, for example simple reflexes are acquired in the first six weeks from birth, curiosity begins to develop at ages 4-7, abstract thought begins around age 11, and so on[findRef]. He simplified these results into three stages of development – kinesthetic, visual, and symbolic. Papart thought that the reason skills like calculation require deliberate teaching is due to our culture’s ‘relative poverty in materials from which the apparently “more advance” intellectual structures can be built’[MS20]; we almost exclusively use symbolic language to describe mathematical concepts where there are visual/kinesthetic equivalents, the abacus existed long before Descartes. Whose fondest memories in Maths class aren’t recalled by the ruler and compass? He believed that computers could provide the engaging context needed, spanning all three developmental mentalities. With that theory and Piaget’s studies showing receptivity to dynamic, passive learning throughout childhood, Papert and his team began working on the Logo programming language and environment.
The Logo language instructs a ‘turtle’ to move and draw. Turtles could be physical moving robots, but more commonly were ‘virtual’ turtles with position and direction on the screen[M11]. Any instruction would take place from wherever the turtle was currently positioned, thus REPEAT 4 [FORWARD 20 LEFT 90] would draw a square; this is now generally referred to as ‘turtle geometry’[M55]. Logo is an example of a high-level programming language – it is closer to human languages than machine languages, so abstracted as to be understandable by English-speakers but using specific phrases as instructions to the computer. The learning comes from the child editing the LEFT 90 to LEFT 100 and guessing how that will change the picture, before executing the code and finding the result. As well as the incidental comprehension of vector geometry, the child learns to teach themselves incrementally by self-referencing.
Papert coined the term ‘Constructionism’ for this theory of discovery learning, a progression of Piaget’s ‘Constructivism’ epistemological stance. Using existing knowledge to further their understanding by developing mental models, the children are guided by teachers rather than being taught information explicitly. With the Logo environment, Papert flipped the strategy of traditional teaching methods. ‘The child, even at preschool ages, is in control: The child programs the computer. And in teaching the computer how to think, children embark on an exploration of how they themselves think.’[MS19] Papert contributions to pedagogy cannot be understated; Logo and its descendants like Scratch (a language developed by Papert’s student Mitchel Resnick) are taught widely in schools half a century later.
In that Autumn of 1968, shortly after Kay’s initial exposure to McLuhan’s work, he visited Papert’s team at MIT’s AI Lab and observed one of the earliest Logo experiments within a school. He was struck by the radical ethos of Papert’s studies, which fitted precisely into what Kay considered the ‘computer as medium’ paradigm, because their purpose was to effect not what children learn, but how they learn. He writes, ‘if the computer is only a vehicle, perhaps you can wait until high school to give “driver’s ed” on it—but if it’s a medium, then it must be extended all the way into the world of the child.’[UIAPPV125] He attributed the success of Logo as a learning environment (far beyond that of other school-taught programming languages such as BASIC) to the decision of designing a language specifically with ‘the end user’s characteristics in mind’[UIAPV125]. Papert later praised Kay and his team at Xerox PARC for aligning with his vision for the future of technology education, saying they were ‘the only American workers on computers for children who made a clear decision that significant research could not be based on the primitive computers that were then becoming available in schools.’[MS210]
Kay’s visit to MIT made a big impression on him, it was clear that his work with computers should be grounded in psychological research. He found most of the psychologists he studied to have findings unsuitable to the progressive environment he was aiming to build around educational computing, until he came across Jerome Bruner and his 1966 book Toward a Theory of Instruction[UIAPV126]. Bruner built on Piaget’s studies, verifying much of his work concerning distinct developmental stages, but had alternative interpretations of how these mentalities influenced children at different ages. He distinguished them as ‘enactive’, ‘iconic’, and ‘symbolic’[TTI10]. Enactive learning is learning through action, iconic describes organising a system of visual (or otherwise sensory) representation , and symbolic involves organisation of words and language. The key difference between Bruner and Piaget’s conclusions was that Bruner believed all of these mentalities existed at all stages of childhood, but in each stage the corresponding Piagetian mentality was favoured. Therefore creating a rich learning environment involves all three types of learning but with particular emphasis to the mentality favoured by their developmental stage.
Papert was responsible for convincing Kay that learning was and should be inextricably linked to user interface design, and Bruner’s model of developmental mentalities gave Kay the framework on which to base his work. It was with this in mind Kay constructed the model of ‘Doing with Images makes Symbols’. Each element of the Dynabook was designed to facilitate enactive, iconic, and symbolic learning.
Doing – mouse, keyset and keyboard (knowing where you are, manipulate) with Images – icons, windows (recognize, compare, configure, concrete) makes Symbols – Smalltalk language (abstract, tie together long chains of reasoning, intuit)[UIAPV129]/[EHOS]
In 1969 Xerox launched the Palo Alto Research Center (commonly referred to as PARC) to investigate future computing technologies, poaching Alan Kay from the University of Utah in 1970[IG54]. Many of Doug Englebart’s colleagues from the Stanford Research Institute also joined the team in the early 1970s, bringing with them the aspirations of the NLS.
Englebart’s NLS and Kay’s FLEX both had the original idea of windows, which was clearly a revolutionary concept, but the informationscape was a blank canvas; beyond connecting our spacial understanding of the world with how we could interact with computers, it was unclear to what extent interfaces could be analogous to real life.
Kay’s team came about the concept of overlapping windows naturally due to the small real-estate of the screen but Bruner’s model suggested contrasting ideas[UIAPV129]. The iconic mentality would tend towards having as much information on the screen simultaneously as possible, allowing more room for cross-information problem solving and preventing the obstruction of having to navigate between material. Meanwhile the symbolic mentality encouraged parallels between interacting with the screen content and how we would interact with the same content in real life; an organised desk would have papers in stacks rather than scattered across it[AKTCC]. The is was the origin of the desktop metaphor[IG61], and as Stephen Johnson puts it, ‘Engelbart and Sutherland had endowed the digital computer with space; Kay’s overlapping windows gave it depth’[IC47].
At the time of text-based interfaces computers had ‘modes’, where keys had different meanings according to which mode you were in. There was an ‘insert’ mode for writing text and an ‘edit’ mode for navigating through the text with the same keys (the most familiar interface we have now to a mode would be CapsLock). Modes were necessary because there were no other methods of navigation, but since the advent of the mouse that didn’t have to be a problem.
Kay defined modeless as ‘the user could always get to the next thing desired without any backing out’[UIAPV129], and since overlapping windows vastly improved ease of navigation it was a win for both symbolic and iconic mentalities. The text-based command syntax of existing systems seemed enormously cumbersome compared to the ease of using a mouse to cycle between windows. This led to Kay’s belief that modes, on the whole, were stifling interaction, and the team worked towards removing modes in as many areas as they could.
Inspired by Logo, Sketchpad and Simula (a language developed in 1965 for modelling manufacturing systems[AKTTC]), the Smalltalk language was built to use abstract data types. Smalltalk coined this as ‘object-oriented programming’[OHOST], a paradigm that is now commonplace – only one of the ten most popular languages on Github doesn’t use objects, and that’s because C was invented in 1972[Githut]. OOP is based around manipulating discrete objects with attributes (nominal data) and methods (functions that effect itself and/or other objects). In order to make objects modeless, a list of things that the object can do appear when it is clicked on, and thus the team at PARC invented drop-down menus.
In the efforts to banish modes from the computer Kay found text-editing to be the most challenging. He stumbled upon a solution while building a paragraph editor in response to the difficulties a group of beginner adult programmers were having when trying to implement one in Smalltalk. Instead of the highlighted character being a selection that is either to be replaced or have something inserted before it, he allowed the selector to be of variable width. This eliminated text modes because a zero-width selection always meant insert, and selecting one or more characters would mean replace (and therefore delete when replaced by nothing). Kay was surprised to find that his PARC coworker Larry Tesler had been working towards the same solution independently[AKAPV130], them both looking for ways to levy the mouse on the UI.
Modes were never actually removed from the interface, they were just disguised by an alternative action. Instead of ‘I want to do this so I will type in this command’ it became ‘I want to go there’; the spatiality of the screen presupposes thought with direct action. In reality we are constantly switching modes, every window vying for position at the top of our visual stack. Not to mention the quasimodes we dip into constantly of the modifier keys (shift, control and so forth)[THI].
HCI researcher Sherry Turkle compares our descent from the absolutism of text-based commands to the relative sub-surface anarchy of mode-switching window interactions, to the path of Western philosophy – from the unified truth of Kant to the obscurity and interpretive nature of Nietzsche[IC83]. She and theorists such as Lev Manovich attribute much of the seeming ‘advances’ of interface design to postmodernism[LONM78]. Manovich asserts that the random-access and ‘remixable’ fundamentalities of the modern desktop are not only a product of postmodernism but an enabler of it[LONM131]. The evolution of the computer as it pertains to postmodernism is a vast topic that I shall not cover in this essay as I am talking specifically about the influences on Alan Kay, but I think it’s important to assert that these interface decisions didn’t happen in a vacuum and had far-reaching cultural and philosophical consequences.