The history of the graphical user interface, understood as the use of graphic icons and a pointing device to control a computer, covers a five-decade span of incremental refinements, built on some constant core principles. Several vendors have created their own windowing systems based on independent code, but with basic elements in common that define the WIMP "window, icon, menu and pointing device" paradigm.

There have been important technological achievements, and enhancements to the general interaction in small steps over prior systems. There have been a few significant breakthroughs in use, but the same organizational interface metaphors and interaction idioms are still in use. Desktop computers are often controlled by computer mice and/or keyboards while laptops often have a pointing stick or touchpad, and smartphones and tablet computers have a touchscreen. The influence of game computers and joystick operation has been omitted.

Early research and developments

Early dynamic information devices such as radar displays, where input devices were used for direct control of computer-created data, set the basis for later improvements of graphical interfaces. Some early cathode ray tube (CRT) screens used a light pen, rather than a mouse, as the pointing device.

The concept of a multi-panel windowing system was introduced by the first real-time graphic display systems for computers: the SAGE Project and Ivan Sutherland's Sketchpad.

Augmentation of Human Intellect (NLS)

In the 1960s, Douglas Engelbart's Augmentation of Human Intellect project at the Augmentation Research Center at SRI International in Menlo Park, California developed the oN-Line System (NLS). This computer incorporated a mouse-driven cursor and multiple windows used to work on hypertext. Engelbart had been inspired, in part, by the memex desk-based information machine suggested by Vannevar Bush in 1945.

Much of the early research was based on how young children learn. So, the design was based on the childlike characteristics of hand–eye coordination, rather than use of command languages, user-defined macro procedures, or automated transformation of data as later used by adult professionals.The Xerox Alto was developed at Xerox PARC and incorporated a graphical user interface using windows, icons and a mouse. It also supported graphical applications, networking and WYSIWYG document production.

Engelbart publicly demonstrated this work at the Association for Computing Machinery / Institute of Electrical and Electronics Engineers (ACM/IEEE)—Computer Society's Fall Joint Computer Conference in San Francisco on December 9, 1968. It was so-called The Mother of All Demos.

The development of computers having multiple overlapping and resizable windows on a "desktop" is commonly, and incorrectly, attributed to Xerox PARC and its Alto. The Xerox Alto's windowing system was inspired by the DNLS (Display NLS)'s overlapping multi-windowing system, which was operational by early 1973 and used at several ARPA locations. In the DNLS, overlapping windows were referred to as "display areas" and could store multiple lines of strings.

In 1971, the screen could only be split into two display areas, vertically or horizontally; by early 1973, the full overlapping windowing system was implemented, and was capable of displaying on an Imlac PDS-1. The Xerox Alto greatly improved upon this system by adding the capability to display bitmapped images, buttons, and other graphics in these windows, as opposed to the DNLS's overlapping display areas which could only display strings of text.

Xerox PARC

led to the advances at Xerox PARC. Several people went from SRI to Xerox PARC in the early 1970s. In 1973, Xerox PARC developed the Alto personal computer. It had a bitmapped screen, and was the first computer to demonstrate the desktop metaphor and graphical user interface (GUI). Several thousand units were built and were heavily used at PARC, and at other XEROX offices, and at several universities for many years. The Alto greatly influenced the design of personal computers during the late 1970s and early 1980s, notably the Three Rivers PERQ, the Apple Lisa and Macintosh, and the first Sun workstations.

The modern WIMP GUI was first developed at Xerox PARC by Alan Kay, Larry Tesler, Dan Ingalls, David Smith, Clarence Ellis and several other researchers. This was introduced in the Smalltalk programming environment. It used windows, icons, and menus (including the first fixed drop-down menu) to support commands such as opening files, deleting files, moving files, etc. In 1974, work began at PARC on Gypsy, the first bitmap What-You-See-Is-What-You-Get (WYSIWYG) cut and paste editor. In 1975, Xerox engineers demonstrated a graphical user interface "including icons and the first use of pop-up menus".

In 1981 Xerox introduced a pioneering product, Star, a workstation incorporating many of PARC's innovations. Although not commercially successful, Star greatly influenced future developments, for example at Apple, Microsoft and Sun Microsystems.

Quantel Paintbox

Released by digital imaging company Quantel in 1981, the Paintbox was a color graphical workstation with support for mouse input, but more oriented for graphics tablets; this model also was notable as one of the first systems to implement pop-up menus.

Blit The Blit, a graphics terminal, was developed at Bell Labs in 1982.

Lisp machines, Symbolics Lisp machines originally developed at MIT and later commercialized by Symbolics and other manufacturers, were early high-end single user computer workstations with advanced graphical user interfaces, windowing, and mouse as an input device. First workstations from Symbolics came to market in 1981, with more advanced designs in the subsequent years.

Lisa, Macintosh, and Apple II

Beginning in 1979, started by Steve Jobs and led by Jef Raskin, the Apple Lisa and Macintosh teams at Apple Computer (which included former members of the Xerox PARC group) continued to develop such ideas. The Lisa, released in 1983, featured a document-centric graphical interface atop an advanced hard disk based OS that featured such things as preemptive multitasking and graphically oriented inter-process communication. The comparatively simplified Macintosh, released in 1984 and designed to be lower in cost, was the first commercially successful product to use a multi-panel window interface. A desktop metaphor was used, in which files looked like pieces of paper, file directories looked like file folders, there was a set of desk accessories like a calculator, notepad, and alarm clock that the user could place around the screen as desired, and the user could delete files and folders by dragging them to a trash-can icon on the screen. The Macintosh, in contrast to the Lisa, used a program-centric rather than document-centric design. Apple revisited the document-centric design, in a limited manner, much later with OpenDoc.

There is still some controversy over the amount of influence that Xerox's PARC work, as opposed to previous academic research, had on the GUIs of the Apple Lisa and Macintosh, but it is clear that the influence was extensive, because first versions of Lisa GUIs even lacked icons. These prototype GUIs are at least mouse-driven, but ignore the "window, icon, menu, pointing device" (WIMP) concept. Screenshots of first GUIs of Apple Lisa prototypes show the early designs. Apple engineers visited the PARC facilities (Apple secured the rights for the visit by compensating Xerox with a pre-IPO purchase of Apple stock) and several PARC employees subsequently moved to Apple to work on the Lisa and Macintosh GUI. However, the Apple work extended PARC's considerably, adding manipulatable icons, and drag and drop manipulation of objects in the file system (see Macintosh Finder) for example. A list of the improvements made by Apple, beyond the PARC interface, can be read at Folklore.org. Jef Raskin warns that many of the reported facts in the history of the PARC and Macintosh development are inaccurate, distorted or even fabricated, due to the lack of usage by historians of direct primary sources.

In 1984, Apple released a television commercial which introduced the Apple Macintosh during the telecast of Super Bowl XVIII by CBS, with allusions to George Orwell's noted novel, Nineteen Eighty-Four. The commercial was aimed at making people think about computers, identifying the user-friendly interface as a personal computer which departed from previous business-oriented systems, and becoming a signature representation of Apple products.

In 1986, the Apple II was launched with 16-bit CPU and significantly improved graphics and audio. It shipped with a new operating system, Apple GS/OS, with a Finder-like GUI similar to the Macintosh series.

Agat The Soviet Union-produced Agat PC featured a graphical interface and a mouse device and was released in 1983.

SGI 1000 series and MEX Founded 1982, SGI introduced the IRIS 1000 Series in 1983. The first graphical terminals (IRIS 1000) shipped in late 1983, and the corresponding workstation model (IRIS 1400) was released in mid-1984. The machines used an early version of the MEX windowing system on top of the GL2 Release 1 operating environment. Examples of the MEX user interface can be seen in a 1988 article in the journal "Computer Graphics", while earlier screenshots can not be found. The first commercial GUI-based systems, these did not find widespread use as to their (discounted) academic list price of $22,500 and $35,700 for the IRIS 1000 and IRIS 1400, respectively. However, these systems were commercially successful enough to start SGI's business as one of the main graphical workstation vendors. In later revisions of graphical workstations, SGI switched to the X Window System, which had been developed starting at MIT since 1984 and which became the standard for UNIX workstations.

Visi On VisiCorp's Visi On was a GUI designed to run on DOS for IBM PCs. It was released in December 1983. Visi On had many features of a modern GUI, and included a few that did not become common until many years later. It was fully mouse-driven, used a bit-mapped display for both text and graphics, included on-line help, and allowed the user to open several programs at once, each in its own window, and switch between them to multitask. Visi On did not, however, include a graphical file manager. Visi On also demanded a hard drive to implement its virtual memory system used for "fast switching", at a time when hard drives were very costly.

GEM (Graphics Environment Manager)

Digital Research (DRI) created GEM as an add-on program for personal computers. GEM was developed to work with existing CP/M and MS-DOS compatible operating systems on business computers such as IBM PC compatibles. It was developed from DRI software, known as GSX, designed by a former PARC employee. Its similarity to the Macintosh desktop led to a copyright lawsuit from Apple Computer, and a settlement which involved some changes to GEM. This was to be the first of a series of "look and feel" lawsuits related to GUI design in the 1980s.

GEM received widespread use in the consumer market from 1985, when it was made the default user interface built into the Atari TOS operating system of the Atari ST line of personal computers. It was also bundled by other computer manufacturers and distributors, such as Amstrad. Later, it was distributed with the best-sold Digital Research version of DOS for IBM PC compatibles, the DR-DOS 6.0. The GEM desktop faded from the market with the withdrawal of the Atari ST line in 1992 and with the popularity of the Microsoft Windows 3.0 in the PC front around the same period of time. The Falcon030, released in 1993 was the last computer from Atari to use GEM.

DeskMate