The Center for Supercomputing Research and Development (CSRD) at the University of Illinois (UIUC) was a research center funded from 1984 to 1993. It built the shared memory Cedar computer system, which included four hardware multiprocessor clusters, as well as parallel system and applications software. It was distinguished from the four earlier UIUC Illiac systems by starting with commercial shared memory subsystems that were based on an earlier paper published by the CSRD founders. Thus CSRD was able to avoid many of the hardware design issues that slowed the Illiac series work. Over its 9 years of major funding, plus follow-on work by many of its participants, CSRD pioneered many of the shared memory architectural and software technologies upon which all 21st century computation is based.
History UIUC began computer research in the 1950s, initially for civil engineering problems, and eventually succeeded by cooperative activities among the Math, Physics, and Electrical Engineering Departments to build the Illiac computer series. This led to founding the Computer Science Department in 1965.
By the early 1980s, a time of world-wide HPC expansion arrived, including the race with the Japanese 5th generation system targeting innovative parallel applications in AI. HPC/supercomputing had emerged as a field, commercial supercomputers were in use by industry and labs (but little by academia), and academic architecture and compiler research were expanding. This led to formation of the Lax committee. Report of the Panel on Large Scale Computing in Science and Engineering to study the academic needs of focused HPC research, and to provide commercial HPC systems for university research. When HPC practitioner Ken Wilson won the Nobel physics prize in 1982, he expanded his already strong advocacy of both, and soon several government agencies introduced HPC R&D programs.
As a result, the UIUC Center for Supercomputing R&D (CSRD) was formed in 1984 (with funding from DOE, NSF, and UIUC, as well as DoD Darpa and AFOSR), under the leadership of three CS professors who had worked together since the Illiac 4 project – David Kuck (Director), Duncan Lawrie (Assoc. Dir. for SW) and Ahmed Sameh (Assoc. Dir for applications), plus Ed Davidson (Assoc. Dir. for hardware/ architecture) who joined from ECE. Many graduate students and post-docs were already contributing to constituent efforts; full time academic professionals were hired, and other faculty cooperated. A total of up to 125 people were involved at the peak, over the nine years of full CSRD operationOtis Port. Superfast computers: you ain’t seen nothin’ yet, Parallel processing will leave today’s speediest machines in the dust, Business Week, Science and Technology section, pp. 91,92, August 26, 1985. David E. Sanger. BREAKING A COMPUTER BARRIER Three Ways to Speed Up Computers, Special To the New York Times, Sept. 9, 1985. Alexander Wolfe. Full speed ahead for software, Electronics, Mar. 10 1986. Donna K. H. Walters. A New Breed of Computers : Mini-Supers at Cutting Edge of Technology, Los Angeles Times, April 27, 1986 12 AM PT. Philip Elmer-DeWitt author, Thomas McCarroll, J. Madeline Nash, Charles Pelton reporters. Fast and Smart, Computers of the Future, Cover story, Time Magazine, pp. 54-58, March 28, 1988. William Allen, Sci Editor. UI team brings Cedar system up to speed, Illini Week, Oct. 27, 1988. John Markoff. Measuring how fast computers really are, . The New York Times Sep 22, 1991. Kevin Kelly, Joseph Weber, Janin Friend, Sandra Atchison, Gail DeGeorge, William J. Holstein. Hot Spots – America’s new growth regions, Business Week, Oct. 19, 1992. John Markoff. A New Standard to Govern PC's With Multiple Chips, New York Times, Oct. 28, 1997. Jon Van, Science writer. U. OF I. SUPERCOMPUTER FUNDING OKD, Chicago Tribune, Feb 04, 1985.
The UIUC administration responded to the computing and scientific times. CSRD was set up as a Graduate College unit, with space in Talbot Lab. UIUC President Stanley Ikenberry arranged to have Governor James Thompson directly endow CSRD with $1 million per year to guarantee personnel continuity. CSRD management helped write proposals that led to a gift from Arnold Beckman of a $50 million building, the establishment of NCSA, and a new CSRD building (now CSL).
The CSRD plan for success took a major departure from earlier Illiac machines by integrating four commercially built parallel machines using an innovative interconnection network and global shared memory. Cedar was based on designing and building a limited amount of innovative hardware, driven by SW that was built on top of emerging parallel applications and compiler technology. By breaking the tradition of building hardware first and then dealing with SW details later, this codesign approach led to the name Cedar instead of Illiac 5.
Earlier work by the CSRD founders had intensively studied a variety of new high-radix interconnection networks,Lawrie, Duncan H. (December 1975). "Access and Alignment of Data in an Array Processor". IEEE Transactions on Computers. C-24 (12): 1145–55.Pin-Yee Chen, Duncan H. Lawrie, David A. Padua, Pen-Chung Yew: Interconnection Networks Using Shuffles. Computer 14(12): 55-64 (1981) built tools to measure the parallelism in sequential programs, designed and built a restructuring compiler (Parafrase) to transform sequential programs into parallel forms, as well as inventing parallel numerical algorithms. During the Parafrase development of the 1970s, several papers were published proposing ideas for expressing and automatically optimizing parallelism.David A. Padua, Michael Wolfe: Advanced Compiler Optimizations for Supercomputers. Commun. ACM 29(12): 1184-1201 (1986)David J. Kuck, Robert H. Kuhn, David A. Padua, Bruce Leasure, Michael Wolfe: Dependence Graphs and Compiler Optimizations. POPL 1981: 207-218Michael Joseph Wolfe 1995. High Performance Compilers for Parallel Computing. Addison-Wesley Longman Publishing Co., Inc., USA.Ron Cytron: Doacross: Beyond Vectorization for Multiprocessors. ICPP 1986: 836-844 These ideas influenced later compiler work at IBM, Rice U. and elsewhere. Parafrase had been donated to Fran Allen's IBM PTRAN group in the late 1970s, Ken Kennedy had gone there on sabbatical and obtained a Parafrase copy, and Ron Cytron joined the IBM group from UIUC. Also, KAI was founded in 1979, by three Parafrase veterans who wrote KAP, a new source-source restructurer, (Kuck, Bruce Leasure, and Mike Wolfe).
The key Cedar idea was to exploit feasible-scale parallelism, by linking together a number of shared memory nodes through an interconnection network and memory hierarchy. Alliant Computers, Inc. Alliant Computer Systems had obtained venture capital funding (in Boston), based on an earlier architecture paper by the CSRD team D. A. Padua, D. J. Kuck and D. H. Lawrie. "High-Speed Multiprocessors and Compilation Techniques", Special Issue on Parallel Processing, IEEE Trans. on Computers, Vol. C-29, No. 9, pp. 763-776, Sept., 1980. and was then shipping systems. The Cedar team was thus immediately able to focus on designing hardware to link 4 Alliant systems and add a global shared memory to the Alliant 8-processor shared memory nodes. In distinction to this, other academic teams of the era pursued massively parallel systems (CalTech, later in cooperation with Intel), fetch-and-add combining networks (NYU), innovative caching (Stanford), dataflow systems (MIT), etc.
In sharp contrast, two decades earlier, the Illiac 4 team required years of work with state of the art industry hardware technology leaders to get the system designed and built. The 1966 industrial hardware proposals for Illiac 4 hardware technology even included a GE Josephson Junction proposal which John Bardeen helped evaluate while he was developing the theory that led to his superconductivity Nobel prize. After contracting with Burroughs Corp to build and integrate an all-transistor hardware system, lengthy discussions ensued about the semiconductor memory design (and schedule slips) with subcontractor Texas Instruments' Jack Kilby (IC inventor and later Nobelist), Morris Chang (later TSMC founder) and others. Earlier Illiac teams had pushed contemporary technologies, with similar implementation problems and delays.
Many attempts at parallel computing startups arose in the decades following Illiac 4, but nothing achieved success until adequate languages and software were developed in the 1970s and 80s. Parafrase veteran Steve Chen joined Cray and led development of the parallel/vector Cray-XMP, released in 1982. The 1990s were a turning point with many 1980s startups failing, the end of bipolar technology cost-effectiveness, and the general end of academic computer building. By the 2000s, with Intel and others manufacturing massive numbers of systems, shared memory parallelism had become ubiquitous.
CSRD and the Cedar system played key roles in advancing shared memory system effectiveness. Many CSRD innovations of the late 80s (Cedar and beyond) are in common use today, including hierarchical shared memory hardware. Cedar also had parallel Fortran extensions, a vectorizing and parallelizing compiler, and custom Linux-based OS, that were used to develop advanced parallel algorithms and applications. These will be detailed below.
Cedar design and construction One unusually productive aspect of the Cedar design effort was the ongoing cooperation among the R&D efforts of architects, compiler writers, and application developers. Another was the substantial legacy of ideas and people from the Parafrase project in the 1970s.Bruce Leasure: Parafrase. Encyclopedia of Parallel Computing 2011: 1407-1409. These enabled the team to focus on several design topics quickly:
Interconnection network and shared memory hierarchy Compiler algorithms, OS, and SW tools Applications and performance analysis
The architecture group had a decade of parallel interconnect and memory experience and high-radix shuffle network chosen, so after selecting Alliant as the node manufacturer, custom interfacing hardware was designed in conjunction with Alliant engineers. The compiler team started by designing Cedar Fortran for this architecture, and by modifying the Kuck & Assoc. (KAI) source-to-source translator with Cedar-specific transformations for the Alliant compiler. Having nearly two decades of parallel algorithm experience (starting from Illiac 4), the applications group chose several applications to study, based on emerging parallel algorithms. This was later extended to include some widely used applications that shared the need for the chosen algorithms David Kuck and Ahmed Sameh. “A Supercomputing Performance Evaluation Plan”. In: Lecture Notes in Computer Science No. 297: Proc. of First Int'l. Conf. on Supercomputing, Athens, Greece T.S. Papatheodorou E.N. Houstis C.D. Polychronopoulos, editors, Springer-Verlag, New York, NY, pp. 1--17, 1987. . Designing, building and integrating the system was then a multi-year effort, including architecture, hardware, compiler, OS and algorithm work.
System Architecture & Hardware The hardware design led to 3 different types of 24” printed circuit boards, with the network board using CSRD-designed crossbar gate array chips. The boards were assembled into three custom racks in a machine room in Talbot Lab using water-cooled heat exchangers. Cedar’s key architectural innovations and features included:
A hierarchical/cluster-based shared-memory multiprocessor (SMP) design using Alliant FX as the building block. This approach is still being followed in today’s parallel machines, where Alliant 8-processor systems have been shrunken to single chip multi-core nodes containing 16, 32 or more cores, depending on power and thermal limitations.