a a a

Interaction in 3D Graphics

Vol.32 No.4 November 1998

Bill Buxton
Alias|Wavefront, Inc. and Silicon Graphics, Inc.

There is a wonderful aspect to being asked to edit a special issue of a periodical such as Computer Graphics. Namely, one actually has the chance to exercise some editorial control — to express a point of view, to try and make some kind of statement. The sheer joy and supposed power that comes from that is short lived, however, since one also has to get the work done! But here I have been lucky. As anyone who knows me will tell you, I have no shortage of opinions. Knowing what I wanted to have articulated was not a problem. But getting it effectively said was, and came down to finding people who I respect, and who would be willing to take time out of their summer to try and share their perspective on interactive graphics.

I am lucky in that virtually everyone that I asked delivered, and did so in style. As a result, I feel that the current collection makes a contribution to the literature and to the community. Each of the contributors has done significant work in interaction. Each is an experienced practitioner and theoretician. And it shows.

The timing for such a special issue is opportune. On the one hand, systems are getting faster, making the possibilities for interaction ever richer while at the same time, systems are becoming ever more complex.

Surrounded by all the stunning graphics and animations created by the SIGGRAPH community, it is all too easy to forget about interaction and input. Yet, at SIGGRAPH’s inception, interaction was a key part of its charter. And the word “interaction” figured prominently in the titles (and spirit) of the classic texts of both Newman and Sproull, and Foley and van Dam. Yet, if we contrast the evolution of graphical output vs. input over the past 25 years of SIGGRAPH, I think that we will all have to admit that there is an imbalance — an imbalance that it is important to redress.

Hence this special issue. In it we have articles by six authors, each of whom brings a different background and perspective.

First, we begin with an application. Bob Zeleznik (website) received his M.S. in computer science from Brown University in 1989. As a Research Scientist in the Brown Computer Graphics Group, Zeleznik’s projects emphasized system architecture and programming language issues for interactive 3D graphics systems. More recently, as Director of User Interface Research at Brown, his research has focused on a range of post-WIMP user interface issues applied to 3D modeling, music notation and desktop productivity applications. Current themes of his research include the use of bimanual gestural interaction with 2D, 6D and force-feedback input devices.

His contribution, “Sketching in 3D,” introduces his work at Brown University in exploring how the fluency and immediacy of conventional sketching with a pen can be extended to 3D. His work demonstrates that design and modeling with 3D need not be more like drafting than sketching, and that through a change in approach, we can make significant gains without having to resort to exotic hardware, or waiting for the next generation of technology.

Shumin Zhai is a Research Staff Member at the IBM Almaden Research Center where he conducts research and innovative development in input devices and interaction techniques, theoretical modeling of human computer interaction (HCI), advanced graphical user interfaces and computer vision-based next generation multimodal interaction techniques. He received his Ph.D. degree from the University of Toronto where he worked on 3D interfaces and six degrees of freedom input control. Many of his publications can be found at his website.

Zhai’s contribution, “User Performance in Relation to 3D Input Device Design,” provides an overview of research into the design of high degree of freedom input devices, and how various aspects of their design affect human performance. His approach goes beyond the horse-race mentality which one often finds (my device is faster than yours) which often gives the impression that there exists a “best” device, independent of user, task or context. Rather, what one gets from his analysis is a sense of each device’s relative strengths and weaknesses, as well as an appreciation for the relevant dimensions of differentiation, and how they affect human performance.

W. Bradford Paley (website) is Principal, Digital Image Design Incorporated (DID) of New York. Paley is a long-time participant in the fields of interface design, scientific visualization and information presentation. After graduating with Phi Beta Kappa honors from U.C. Berkeley in 1981, he entered the financial data processing field as a consultant with Citibank in New York. He simultaneously pursued his interests in using the computer as a human communications medium by doing computer animation for the advertising industry. Finding the production tools awkward and almost non-existent he began writing his own, soon realizing that building a comfortable tool was more challenging and interesting than doing the animation itself.

Highlights of DID’s contributions to computer graphics are the invention and manufacture of the ID Magazine award-winning 3D mouse, the Cricket, and the award-winning animation input device, the Monkey. Current energies and resources are focused on the coming wave of tiny, inexpensive, special purpose computers, richly connected and distributed even throughout the environment. This is the natural outgrowth of concentration on visual communication, physical input device design, psychology, magic and simulation.

His contribution, “Designing Special-Purpose Input Devices,” is a “tale from the trenches” account of his experiences designing custom devices for specialized applications. What we too often forget is that it takes as much (or more) effort to design a great input device as it does a great image. Paley’s perspective nicely complements that of Shuman Zhai’s.

Mark Billinghurst is a fourth year Ph.D. student at the University of Washington’s Human Interface Technology Laboratory. His work focuses on conversational interfaces that combine multimodal input and output. He was co-organizer of the VRAIS 96, VRST 96, and Visual 98 tutorials on conversational interfaces and has published several papers and book chapters in the field. A native of New Zealand, Billinghurst graduated from Waikato University, Hamilton, New Zealand, with honors in 1990 and earned his master’s degrees in applied mathematics and physics in 1992.

His contribution, “Put That Where? Voice and Gesture at the Graphics Interface,” gives a good introduction to multimodal interfaces that employ both manual gesture and speech. It is about 20 years since Richard Bolt and his colleagues at the Architecture Machine Group at MIT demonstrated their classic “Put-That-There” system — a system that was mainly made up using (albeit expensive) “over the counter” hardware. The hardware is nowadays neither expensive nor rare. Yet since these earlier days, we seem to have made little visible progress. Perhaps what this says is that the underlying issues are conceptual and theoretical (mixed with a dose of technological inertia). Billinghurst’s contribution provides a good introduction to the current state of the art, and will hopefully provide a catalyst to renewed interest and activity in this domain.

Bill Buxton,
Chief Scientist
Alias|Wavefront, Inc. & Silicon Graphics, Inc.


The copyright of articles and images printed remains with the author unless otherwise indicated.

George Fitzmaurice is a Research Scientist at Alias|Wavefront, Inc. His research interests include computer augmented reality, physical-virtual interfaces, interactive 3D graphics and haptic input devices. He holds a Ph.D. in computer science from the University of Toronto, an M.Sc. in computer science from Brown University and a B.Sc. in mathematics with computer science from the Massachusetts Institute of Technology. Some of his publications can be found at his Web site.

Fitzmaurice is first author on the paper, “Compatibility and Interaction Style in Computer Graphics.” In one sense, this paper reinforces the classic notion that notation is a tool of thought, and so we must think about how we represent graphics systems in terms of what we want to accomplish with them. In particular, the paper investigates the relationship between how things are represented, and the resultant biases on usage and complexity. It is yet another illustration that there is more to user interfaces than simply determining how difficult it is to pick items from menus, etc.

Finally, I, Bill Buxton (website), am Chief Scientist of Alias|Wavefront Inc., and Silicon Graphics, Inc. I am also a Professor of computer science at the University of Toronto. My interest in interaction grew out of my previous life, where I was involved in designing and playing electro-acoustic musical instruments, and was refined through a long association with Xerox PARC. I am main author of “HMD’s, Caves & Chameleon,” written with George Fitzmaurice, which looks at various approaches to VR systems, and how each influences the ability of one to interact in different ways.

In conclusion, I want to take this opportunity to thank each of the authors for their effort and contribution, and express our collective desire for feedback. Finally, we hope that this collection helps provide some insights into this field that we care so much about, and helps you in your future work.

Editor’s note: Thank you, Bill, for your insightful coverage of input technology. I encourage readers to digest these focus articles as well as take a look back at previous issues’ coverage of virtual reality (Computer Graphics 30(4) November 1996) and next generation visual displays (Computer Graphics 31(2) May 1997).