Hyperspace as CyberSpace 163
Simnet 166
Cyberspace Limitations 167
Inside CyberSpace 169
Logosphere 173
Visualizing the Logosphere 174
Hyperspace as CyberSpace:
Perhaps the most intriguing new development that can move hypertext into even more interactive hyperspace, is the creation of systems using datagloves and helmet mounted displays and heads-up goggles, for creating virtual realities. Cyberspace augments these more or less realistic notions of hyperspace with assumptions that are even more fantastic, of communication through neural implants, and other techniques. This is of course pure fantasy, but this fantasy has a liberating effect on imaginings about how to use this new technology and where it is going. As Michael Benedikt, an architect and visionary, put it during the first conference on cyberspace, "In cyberspace, information intensive institutions and businesses take on a form, identity, and working reality --- quite literally, an architecture --- that is both counterpart and different to the form, identity, and reality they have in the everyday physical world." Although the more esoteric techniques are not (should I say "Yet"?) available, computer-based systems can already create artificial realities of amazing impact. Through the awesome power of computers and these interfaces, geographic space can be distorted, compressed, and reshaped at will. People can literally walk and peruse artificial space as if it were real, experiencing feedback from their every action in ways that mimic reality. Vibrotactile bodysuits can respond to human actions to give the kinaesthetic impression of flipping switches, touching things lightly, catching small objects, and encountering physical resistance. And the fantastic can be made to be part of that reality. Touching an object can literally explode it into a new world: any object can be made the doorway of yet another reality. A physical - looking object can be turned into a hotspot that links immediately to another time and space. Only imagination can tell us where this development will lead. Already architects use it to explore building designs; businesses are exploring the techniques for teleconferencing; and others are beginning to use it for data analysis of complex relationships. As a natural expansion of the powers of hypertext, cyberspace interfaces and technologies will let people experience the ultimate abstraction of navigating pure information, the logosphere of information that only exists within a computer and someone's imagination.
The most imaginative notions about cyberspace stem from the first book to hold the word, "Neuromancer", by William Gibson (Ace Books, 1984).
Figure 8.1 A depiction of an interactive CyberSpace of virtual reality.
All images and icons are seen in goggles containing computer displays. Touching any image can literally explode the space into new dimensions.
Cyberspace is a combination of all the modern interface technologies applied to direct manipulation (Shneiderman, 1983) systems. Only, direct manipulation is taken to an incredible extreme to represent reality and its interactivity as completely as possible. The interactivity of a mouse, embedded menus, touchscreen, computer graphics, multimedia, database storage and retrieval, reading, writing, etc. are all there, but added to that are the freedom to move and to get feedback for each action. A sensory net of mechanical, electrical, and light components monitor as many actions as possible. Interactivity is thus extended to encompass a much broader bandwidth of human communication with a computer. Through the sensory net it is possible to wave one's hands through a magnetic field that senses every motion, or to sense eye motion and eye tracking so that the computer knows what one is reading, to shift one's head and have the scene seamlessly altered , either gradually or abruptly, into a new context, and there are perhaps many more interactive gestures (sign languages?) we are only beginning to mold into human computer communication.
All the constraints that we have learned as children to deal with physical reality can be turned into abstractions for symbolic communication through cyberspace implementations. As a communication system between human and computer, virtual realities encourage us to use our understanding of physical reality to set the conventions for guiding human-computer communication. The rules governing our interaction with physical reality can be used to facilitate expectations and guide expectations and the goals and plans we have. Instead of confusing users with fantastic flights of fantasy, virtual realities can ground people in the common sense world of the everyday while they work with the abstractions of numbers and words. The strength of cyberspace systems may be not so much for breaking the conventions of physical reality, but to turn the conventions of physical reality into a constraint system, a guiding metaphor, a mental model, for helping us to deal with abstractions.
By making the content of abstractions in some sense manipulable, cyberspace may help us create a logosphere of mental models that simplify and reduce our mental burden in dealing with complexities of a theoretical and abstract kind. Scientific visualization systems could let us interact with models of molecules, atoms, quarks, strings, space-time, chaos, etc. in personally dramatic and novel ways. Telerobotics could help us explore the inaccessible mysteries of the outer planets and space. Businesses, especially knowledge workers, can create new database metaphors that turns mounds of data into trees in a forest, houses in a city, storage rooms in a building, or any other convenient metaphor for an interactive knowledge space or dataspace. And of course, cyberspace concepts, can be added to the rich texture of hypermedia and hypertext to improve education and training.
Simnet
The cyberspace concept, in its virtual realities guise, is already in use in Army training in an innovative project called Simnet. Simnet is a networked simulation system of tank crews warfighting each other. Instead of using real tanks, rockets, and oil, computer -based part task simulators with display screens and instrumented controls are bolted to the floor of normal schoolrooms. The crews can be in neighboring workstations in the same room or in different states, or even spread around the globe. Each station is in the same virtual battlefield with a display that mimics a tank's line of sight, or infrared and radar sensors to his opponents and to friendly forces. Each station is a manned simulator of an entire tank crew -- commander, driver, gunner, and loader. Everyone performs in a conventional way on this virtual battlefield, except when they are hit, disabled or destroyed, no one actually gets hurt. And after the fighting stops, everyone can replay selected portions of the action, and evaluate both the level of coordination and training, and the actual value of the plans and doctrine that were used. Only the upper echelons are allowed to engage in the power of the fantastic, and view the entire battlefield as if they were omniscient, moving from one tank to another (and even the opponents') or obtaining dramatic overviews of the engagement while it is in progress. Simnet is still a relatively small scale implementation, not a widely fielded system, and it is undergoing constant development and improvement at BBN in Cambridge, Ft. Leavenworth, Ft. Rucker, and Ft. Knox.
One of the most controversial and dramatic additions to Simnet being developed is the SAFOR ( Semi-Automated Force), an attempt to replace some nodes in this network of manned simulators with intelligent virtual opponents entirely simulated by a computer. Progress toward this goal has been stunning over the past five years. SATFOR vehicles and weapons now appear to be indistinguishable from manned simulators on the network. They exhibit proper tactical actions, route following and navigation, and damage reactions. Their actions coordinate not only at the level of an individual tank, but also at battalion and higher planning levels. Because they are all pure software, it is much easier to adapt them to other tactical components, such as helicopters and fixed wing aircraft. This adds a greater amount of realism to the battlefield for the manned tank simulators, without the prohibitive cost of building manned helicopter or aircraft simulators.
Adding this kind of intelligence and realism to the training simulation has led some to dream of getting rid entirely of the simulator, and replacing it with a computerized datasuit and datagloves. This in turn has led to a dream of a walking exoskeleton of sensors and activators to enclose a human in a portable, movable cyberspace of virtual reality for training and for actual engagements. While this is a questionable prospect for the near future, the Simnet concept has attracted widespread interest and support, and will undoubtedly reshape training throughout the armed forces.
For an update of cyberspace and virtual reality applications to training, see a paper I wrote 5 years later: Immersive Training Systems Psotka, (1994) Instructional Science.
A longer review paper on the VR for education and
training. (Aug 8, 1994)
Cyberspace Limitations
The first commercial implementations of a virtual reality system, and the beginnings of a cyberspace system, belong to Autodesk, the creators of a computer-aided design system favored by designers and architects, AutoCad. Interestingly, this company has backed another visionary concept, the Xanadu hypertext system conceived by Nelson. Another prominent participant in this activity is VPL Research, makes of the DataGlove and DataSuit. If the hype surrounding these products even begins to approximate their real uses, many more companies are certain to jump on the bandwagon shortly.
For all of their phenomenal assets, cyberspace implementations are not without their peril. A nagging question to anyone familiar with the history of computer games and adolescents, is the issue of isolation versus cooperation. Cyberspace could lead to both. Individuals in a cyberspace cocoon could be isolated from everyone else and reality, or they could share and cooperate with each other in a broad bandwidth of communication that is unparalleled in our history. Which will occur? Undoubtedly both, for different purposes and by different individuals. It is an issue whose first halting telltale signs have yet really to appear, but whose long aftermath of discussion and argumentation will be enmeshed in many other debates on technology and social conventions and morality.
Long before that stage of sophisticated moral entanglement becomes pronounced, there are many limitations of "reality manipulation" that come to the fore in surprising ways. Consider for instance trying to introduce someone to the complex abstractions of hypertext, as we are doing in this book, in a cyberspace environment. It would probably be very difficult to take advantage of the special strengths of cyberspace in anything but a peripheral way. On the one hand, creating hands-on interaction with hypertext is extraordinarily important and effective for conveying the sense of what hypertext is all about, and cyberspace could be an effective vehicle for that. But how do you get across the abstractions of navigation problems, knowledge representation limitations, design and authoring concerns, and all the other issues that this book is all about without confronting the real limitations of concrete images: they are too opaque to convey abstract generalities. Inevitably, even in cyberspace, one has to deal with numbers and text.
Every interaction with a virtual manipulative in cyberspace is fraught with misleading ambiguities when the goal of the interaction is to induce generalizations, or to encourage the formation of general principles and rules. The person who might use cyberspace to ride the roller coaster of stock-market data, could literally sit in the car and feel the violent up and down swings of the market as he gazed at a neighboring train that ran on a track of a stock of his choosing. Would that make it easier to predict and anticipate the next change in the stock's value? Would that make it any easier to grasp general and intuitive factors that affect the specific stock or the whole market? The problem of course is to match reality with the symbolic realm of interest in an appropriate metaphor that is revealing and enlightening. The problem is one of creating a virtual model of reality that is semantically coherent and relevant so that it has comparable associations and constraints from key factors: a first-order isomorphism. The problem is one of finding first- or second-order isomorph or correlations to the phenomenon of interest. This is not an easy problem to solve. In many cases, shorthand metaphors of this kind simply are unavailable. A roller coaster stock market ride may leave the analyst light headed and dizzy rather than enlightened. Yet how would one choose a better metaphor?
Light headedness and confusion are also feelings that many experience with the best implementations of cyberspace today. When one straps on the goggles, or "heads-up" display, and gloves and begins to move in this eerie realm, the first reaction tends to be profound disorientation rather than pleasure. Sometimes the most profound pleasure and shock comes when you take the goggles off and see a world of resplendent color and exquisite detail. Much of this stems from current technological limitations rather than a fundamental failing of the concept. The miniature displays in the goggles do not provide enough texture to create realistic depth gradients. The refresh rate of the screens is too slow to keep up with rapid head movements. The computers cannot create fully textured objects fast enough, so usually only wire frame objects are rendered. Even then, taking account of sensors from a dataglove, and sequencing the visual object precisely to interact with a human wearing the glove requires such exquisite timing that invariably something goes wrong.
Humans may appear to be lazy and sloppy processors of reality in most everyday instances. For instance, we can walk or drive a car with no apparent memory of any control processes at all. Yet, when something does go wrong, even while distracted, we can also be extraordinarily sensitive to minute errors in timing and appearance.
Consider for instance, the very small changes in our view of the world that a pair of eyeglasses create. Not only do they reduce the blur, when they are well-fitted, but they also modify the apparent size of objects a small amount, and create same small distortions, reflections, and color aberrations. All of these changes are tiny compared to the gross changes that cyberspace provokes, yet most people need to adjust even to a new pair of glasses for a few days before they are entirely comfortable. Dealing with a poorly mapped version of reality inside a cyberspace may in the long run be as disconcerting as turning the world upside down with prisms.
Inside CyberSpace
Yet, cyberspace can be magical. Imagine , for a moment, donning a scuba mask containing small LCD (Liquid Crystal Display) panels to display computer-generated images at 30 frames per second. The mask also holds sensors that detect head motion. When your head moves, the sensors let the computer figure out how the display should change, and then the computer generates a new view on its screens to correspond to those changes. It also has earphones and a microphone to communicate with the computer. Next you can put on a dataglove, a futuristic, stiff glove wired with sensors and actuators. The basic technology for this device is so simple that Mattel markets a very effective version of something NASA took years to develop for under $100.00. The sensors tell the computer when you have moved your wrist, hand, and fingers, and the actuators can give your fingertips the feeling of pressing a button or flipping a switch, or they can also give the impression of a soft impact with small objects. Next you can mount a treadmill or apply sensors to your legs and ankles, to give the computer a full sense of your motion (See Figure 8.1). Now you are ready to go. At the University of North Carolina, exactly such a setup is used to give architects a direct vision of their designs. They can walk freely through the architectural space a get a much better appreciation of its strengths and weaknesses: feedback for reflection and redesign!
Imagine learning to juggle in this gear for cyberspace. Once you have put the equipment on, you have everything! You don't need any juggling balls at all. All you need to do is say to the computer, "Juggle!" Then the computer will begin to display balls floating in the right way in the air. If they drop to the ground, no problem, the computer will pick them up and continue to toss them. All you have to do to begin to learn to juggle is catch the virtual balls in the right sequence ( see Figure 8.2). The dataglove will even give you a gentle nudge in the proper spot of your hand whenever you make contact. Once that becomes an automatic skill, it wouldn't take much to have some motors geared up to your arm to make your arm move in the proper toss for the balls as the computer generates the images of their arc.
Even better, the sensors and actuators could be synchronized with a similar device on an expert's arms. Now the expert could tell exactly what you are doing or trying to do, and give you the kind of feedback you need to adjust correctly. Telerobotically, he could move your hands and arms so that you too could get the feel of expert performance. By switching control back and forth between the expert and the novice a kind of online model tracing could be experienced and expertise might be improved at new and unparalleled rates. How this technology could be used for training is in some ways immediately obvious, and in many other ways, still to be explored. Obviously the computer can begin with magically light balls that float ever so slowly in the air to make it easy for you to learn the right sequence of motions. As your skill improves, so can the speed of the balls. It would be intriguing to see just how fast someone could learn this way. You might be cascading four or five balls in a few hours of practice, with the expert in the cyberspace suit guiding you individually!
These capabilities of mapping expert and novice physical reactions together in a synchronic way, suggest that artificial realities might be appropriate for any kinds of motor skills particularly involving hand-eye coordination. Tennis, swimming, golf, many sports, including even fine grained skills like archery and sharpshooting, all seem to be on the verge of a new and improved training technology. But virtual realities are really only one-half of cyberspace, and to some extent, the simpler and less useful half. The real meat of cyberspace applications are in the logosphere, in realizing abstract symbolic systems.
Cyberspace and Mnemonics
When Giordano Bruno was burned at the stake on February 19, 1600 the world lost one of the last of its great mnemonic orators, someone who knew and practiced ancient skills of preparing complex arguments in great detail, and storing them in a systematic mental (mnemonic) system. Bruno's primary mental storage system was, conceptually, throughout his house, so that as he gave his speech he could walk through the house (in his mind) and pick up the important points of his arguments in the spots where he knew he kept them. Sometimes, of course, his speeches covered less familiar ground, and he had to walk outside his house to collect fresh arguments stored in less convenient places. It appears that hypertext and cyberspace may provide reasons for investigating these wonderful spatial mnemonic techniques once again, so that we may understand how best to make use, not only of the "desktop" metaphor, but the "home" metaphor, and perhaps even the "city" metaphor.
Gordon Bower (1965) has given us a good analysis of some of these important mnemonic techniques. However, his guidelines were for creating imaginary, mental systems to remember and retrieve facts and information from our memory stores. With cyberspace, retrieval will emphasize retrieval not from human memory, but from the vast external memory of the computerized memory store. In cyberspace we can begin to examine how to use some of these techniques for virtual, not imaginary, systems of storage and retrieval.
Consider the simple technique of using rhyme to create a visualizable, ordered set of numbers:
But what exactly is the power of this technique? Many studies have tried to uncover the exact mechanism that functions to create such lasting and effective retrieval with this mnemonic scheme. Many alternatives have been partially eliminated. It is not just the effort or time it takes to create a unique association between one - bun and income tax. It is not just the uniqueness or surprisingness of the image of ketchup spread over yourself inside a hot dog bun. It is not just the concreteness of the image. It is all of these things and it is not any of these things. It really comes down to the power of the visual system for creating orderly four dimensional representations of reality in which storage and retrieval is simply superb. In reality, we do not know how this visual retrieval system works. There are some vague ideas, based on the metaphor of holography or optical computer storage (Kohonen, 1988), but there are no implementations that begin to harness this power for closer analysis. We will undoubtedly have to wait for a more appropriate technology than linear or even parallel processing systems. But perhaps we can augment human capabilities and current computer technology even better by trying to understand this hidden system. Perhaps harnessed for similar use, icons and visual images can be used appropriately by humans to speed up and enhance retrieval and storage in hypertext systems.
However, the example also demonstrates the irrelevance of such a simplistic application of a visual mnemonic scheme in today's world. Instead of creating such mnemonics, even though they are simple and easy, it is much easier just to write a list. No one that I know uses mnemonics regularly anymore (except perhaps under the supreme pressure of a forced examination where notes are not allowed!). So the specific relevance of this particular mnemonic trick is not at all evident for cyberspace or hypertext. It is much easier to create a virtual "to do " list on a desktop calendar in cyberspace, and use a desktop metaphor to remember not what the list itself is, but just where to look to find the list. Other sophisticated applications based on our understanding of visual memory also need to be developed. Instead of artificially accessing the strengths of organized visual memory through these forced mnemonic associations, it seems much more reasonable to empower the strengths of visual memory by bringing into play organized images based on familiar and everyday environments: the desktop, office, home, and city. However, how to use these metaphors appropriately so that the right things can be found on demand is not at all self-evident. It may be time to look more closely at how Bruno and his illustrious predecessors used their mnemonic systems so that we too may better understand how to structure hyperspace for easy storage and retrieval.
Logosphere
We have already seen several ways in which the structure of the logosphere, the space of all our knowledge, can be charted in abstract hierarchical data structures but there may be more complex structures of knowledge that will be just as useful. Cyberspace offers us an invitation to realize hierarchical structures yet another way, by creating a visual logosphere, an empyrean projection of all our knowledge in a way that is reminiscent of the astrological structure of the starry heavens. Imagine having the structure of our knowledge available to us in a giant observatory and planetarium.
The celestial sphere, we know, has been divided up into the 30 degree sections of the zodiac, twelve zones dominated by the constellations near the ecliptic, an 18 degree band that holds the motion of all the planets except Pluto. Intriguingly, these divisions were formed long before the modern calendar on the basis of when the sun enters a particular section of the zodiac. Of course, the sun cannot be directly seen to enter a section of the zodiac, since the stars are invisible when the sun appears. Instead, the inference must be made that the heavens continue to revolve around the earth as the sun rises. The great insights of Copernicus, Kepler, and Newton seem small potatoes against the larger backdrop of wisdom that informed these "primitive" inferences and observations. How much did the early visualizations of the zodiac (as the Great Bear, the Little Dipper, the Hunter, and all the other possible starry cartoons) help make these temporal patterns clear? Would it be too extreme to conjecture that the modern order of space and the solar system could not have become evident with out this visualization? Without this carefully crafted mnemonic system that let early observers remember the structure of the stars and communicate it with their peers, would the irregularities of planetary motion and the regularities of the seasons been apparent? Would it have provoked the creation of Ptolemaic astronomy and everything that followed? Archeology and history only show that everywhere, in many civilizations, these patterns recurred spontaneously at the dawn of science, and that should be encouragement enough for us to begin to reconstruct the patterns of the logosphere in its celestial image.
If we were to lay out the image of our knowledge in a conceptual space would we too find patterns that resembled the strange creatures inhabiting the Zodiac? Or are there only orderly hierarchies of concepts such as those in WordNet? Of course, much depends on the kinds of nodes and links and other structure we choose to use to lay out this knowledge in a gigantic three- dimensional browser of conceptual space in the logosphere.
Imagine if you will, being able to see the patterns of conceptual space the way you see the star clusters in the night sky. What should it look like? We are free to create the first patterns in any way we like. Whether this is a lasting portrayal depends on how useful it will be to those logonauts who follow us into this space in the future. At the moment there are only very simplistic models: Semnet, WordNet, CYC. These are all straightforwardly hierarchical and network-like . Yet, from them may grow complex structures we may only be able to use if complex visual displays, planetaria - like, are used to organize them.
Visualizing the Logosphere
Databases of a simpler kind, such as numerical spreadsheets, turned into dataspaces through which people can navigate visually may provide access to many people who might otherwise feel quite alienated by numbers. Popular graphics systems already try to invoke images of reality to help people understand their semantic content. Popular weekly news magazines have pioneered this trend. A common example might involve using oil drums of increasing height to portray the rising import of foreign oil. Does this added dimensionality of the graph make it more memorable or aid its purpose in any demonstrable way? It is possible that this added realism has many positive effects over a simple bar graph or pie chart, yet if these benefits exist, they remain to be demonstrated (Psotka, 1982; Kosslyn, 1986).
Figure 8.3 Added realism in bar graphs --- do they improve the graph?