The Human Processor: changing the relation between human and computer

Transcription

1 The Human Processor: changing the relation between human and computer Joris Slob LIACS, Leiden University Niels Bohrweg 1, 2333CA Leiden Netherlands ABSTRACT In the Human-Computer Interaction (HCI) field it is common to choose the human as the task-giver and the computer as the obedient servant. HCI seeks to construct an interface between the human and the computer in such a way that the human needs to adapt as little as possible to facilitate this cooperation. In this paper we propose a role-reversal whilst maintaining the goal of minimizing the perceived disruption of normal human activities. The human will be coaxed to operate as a computer and perform basic logic operations. This setup is suggested to open up the discussion whether HCI would benefit by looking at the interaction from the other direction. It leads us to reconsider the extend to which HCI design can produce perfect interaction between fallible users and machines. Keywords User/Machine Systems, HCI, Unconventional Computation, Arithmetic and logic structures INTRODUCTION Doing calculations has long been seen as a faculty only reserved for man. We now have computers around us to prove otherwise. Computers excel at arithmetic and outperform us in speed since their first appearance. For simple arithmetic, people use pocket calculators, very limited versions of the computers we use. The computer can still do the simple arithmetic, but because of the logic structure under this ability, it can do much more. It is hard for people in general to understand the real essence of a computer, its ability to compute. When we are writing our s or creating graphics or music on the computer, it is difficult to understand what this has to do with simple additions and subtractions. Computers are just one way in which we can manifest computation. It is the most successful one in terms of its wide applicability and speed. We explore the possibility for humans to be part of the computation of a computer. In fiction questions like these have been asked before[2]. In this study, we look into the matter using experimental techniques and put this question into a historical perspective. Proceedings 13 th Computer-Human Interaction Netherlands Conference June , Leiden (The Netherlands) Maarten H. Lamers Media Technology MSc programme Leiden University Netherlands lamers@liacs.nl The Association for Computing Machinery website defines HCI as follows: Human-computer interaction is a discipline concerned with the design, evaluation and implementation of interactive computing systems for human use and with the study of major phenomena surrounding them. Traditionally, HCI explores different interfaces to transform the intentions of humans into computer tasks. We stretch the definition of HCI to include all interactions between humans and computers, not just those concerned with human use. This allows for a novel approach to studying their interaction: reversing the roles of computer and human. Such reversal of roles requires construction of an interface that transforms the computer s natural goals into tasks for humans. Staying true to goals of traditional HCI, the interface must be as intuitive for humans as possible. Although this paper describes the technical aspects to realize such an interface, we hope the reader will see this more as a thought experiment to show that any error or incompleteness is solely based on human flaw. CONTEXT The first programmable machine was envisioned by Charles Babbage. During the time of Charles Babbage, people used books to look up the values of certain functions. These functions were hand calculated by humans. A person that calculated these numbers was called a computer. Charles Babbage invented and constructed machines that could do the work of these computers and do them more accurately than any human. These machines were called Difference Engines, because they worked on the principle of finite differences to calculate polynomials without the need of multiplications. Contemporary Context The best definition we have of computability comes from Alan Turing[13], in which he describes an hypothetical minimal but sufficient machine that can deal with a broad range of problems. Machines that can do the same as this Turing machine are said to be Turing complete. All modern personal computers are based on the laws of electromagnetism and on the Von Neumann architecture[15]. This architecture has three important components: memory, control unit and an Arithmetic Logic Unit (also known as ALU). We use the Von Neumann architecture in our setup, but we examine whether it is possible to replace one of its 75

2 components with a human component. The field of unconventional computing deals with alternative implementations of existing computers or completely new computer designs. Notable subfields include: Conservative Logic[7] DNA computing[3] Quantum computing[6] Fluidics[10] Human-based computation This study deals with the latter form of alternative computation: Human-based computation. Human Computing The first reference to putting humans back in the computing equation was made by Richard Dawkins[5], where he describes using human evaluation as a fitness function in a genetic algorithm. Later Caldwell, Johnston[4] and Sims[11] used multiple people for the fitness function. A full Human-Based Genetic Algorithm was discussed by A. Kosorukoff [9]. Recent work uses humans as source of computation in domains where computers perform badly, e.g. image recognition and common sense tasks. A notable example is the Amazon Mechanical Turk, where people can do small tasks to help a larger project. Another important project is Peekaboom [14] and other works of Luis von Ahn. These projects place humans in an environment enticing them to use their computational capacity to do tasks where classical algorithms are suboptimal. A form of collective intelligence is introduced by James Surowiecki[12] by describing how the mean guess of uninformed people can be better than the informed guess of an expert. Other notable techniques in this field are collaborative filtering, tagging and verification games. There are two big differences between these projects and our approach: Awareness Completeness We try to research methods to let humans participate in a computation without them being aware of this fact. Also we will not restrict ourselves to computations that computers are bad at solving. We seek to create a universal computation machine from humans. THE ALU In this study we will focus on the ALU in the processor. To achieve an ALU capable to compute any binary function, we need to implement the sole sufficient operator (SSO) for this domain. With this SSO one can construct all finitary boolean functions. In other words: multiple application of this SSO makes any transformation from multiple input booleans to a single output boolean possible. Common SSOs are the NAND operator or the NOR operator. However, to construct a simple full adder one would need at least 9 NAND operations or 13 NOR operations. When combined, AND, OR and NOT form another set of sufficient operators. To show the structure of the ALU we decomposed a higher function in Figure 1. Note that the round nodes in this figure are the set of sufficient operators to do all binary operations. To clarify the process, let us walk through the whole computation, focussing on the actions of the ALU. Suppose someone wants to know the answer to seven added to five. We formulate this in symbolic language: = Then your software will rewrite this into machine language. This is a transformation which we do not consider part of solving the equation. It translates the + to an ADD operation and numbers in decimal system to unsigned binary numbers. Now we are at the processor level of communication. Computers store intermediate results in memory spaces called registers. Here AX stands for one of these registers. MOV stores the first value in memory and ADD adds the next value to the value in memory. For 4-bit integers this translates into the following instructions: MOV AX 0111 ADD AX 0101 MOV is a memory operation and we are only interested in steps that are ALU operations. In our equation the step is ADD , where we have substituted the value of AX with the value in its memory. Like decimal addition, the computer will now add the digits one by one and remember and deal with a carry when necessary. This operation is called a FULL ADDER. Our addition example with four bits now becomes four separate FULL ADDER instructions with three inputs: the carry, a bit from the first number and a bit from the second number: FULL ADDER FULL ADDER FULL ADDER FULL ADDER

3 A FULL ADDER is not a basic operator in our set, so we decompose that into smaller components. A full adder uses two HALF ADDERS and an OR operator. HALF ADDERS cannot handle carries as input and thus are more simple. A HALF ADDER can also be decomposed into AND, OR and NOT operations. Here we show a complete decomposition of FULL ADDER 0 1 1: FULL ADDER OR 0 1 HALF ADDER 0 1 XOR 0 1 HALF ADDER 1 1 XOR 1 1 OR 0 0 AND 0 0 AND 1 1 OR 0 1 AND 1 0 Thus this single FULL ADDER needed 13 basic operations to calculate. If an ALU supports 4-bits numbers it needs 4 * 13 = 52 operations for our example addition. THE HUMAN ALU To build a human-based computer we use the model introduced by Von Neumann [15]. We modelled the three components, the memory, the control unit and the ALU, in software. This software was designed such that the ALU component could be easily replaced with other systems. Using this setup we could easily study the effects of a human-based ALU on the total computer. The implementation of our ALU is highly influenced by the x86 assembly language. It should implement enough logic to be functionally complete. In our implementation we chose to implement the AND, OR and NOT functions. For further ease of use, we also define an extended ALU which also implements a half adder. Two different test ALU setups were made. Direct task model Hidden task model In the direct task model, humans were directly asked to solve the tasks that the computer needed. This meant that they were directly confronted with simple logical queries like What is the answer to 1 AND 0?. Although simple in nature, it is difficult for humans to quickly answer these queries. In the hidden task model the computer presented the task in a different form. In this case each input to the ALU was directly fed to a different colour channel (red, green or blue) and shown to a human. It is important to note that the computer has no notion of composite colours, but the human does. The human can quickly recognize different mixtures of colours as different output. The human was asked to click on the colour they saw. The buttons were interpreted as differently based on the operation the computer needed to do. For example, if the computer needed to do an AND operation on two inputs, it would put feed inputs directly on the red and blue LEDs. The green LED would be on to always make sure some colour was presented to the human. All LEDs on resulted in a whitish colour. All other possible combinations resulted in different perceived colours. For the AND operation the whitish button was interpreted as a 1 and the rest as a 0. EXPERIMENT To test our human-based computer, we programmed a simple calculator. In Figure 2 we see the number of primitive operators needed to do simple arithmetic calculations of different bit-sizes. The graph uses a loglog plot, so the slope defines the power with which the number of operations grows with respect to the number of bits. Because of the sharp increase of needed operations, we chose within the experiment to limit ourselves to addition calculations on four-bit numbers. This addition calculator was run on the simulated computer, that used the human processor to compute the results. Humans were asked to do whatever they thought was expected of them when confronted with the hardware. In the hardware setup, the test subjects were first allowed to get comfortable with the colours that corresponded to the different buttons using a simple program that showed the colour of the button they pressed. This was necessary because of the inevitable subtle difference between the colour on the button and the combined colours of the LEDs. After the necessary operations had been done, the participant was asked a few questions: 77

4 What do you believe is the purpose of this program/box? How many times do you think you pressed a button? Did you perceive any (familiar) pattern? RESULTS None of the test subjects believed the purpose of their actions was to do a computation. Most subjects answered believed their colour vision or hand eye coordination was being tested. The perceived number of times that the test subjects pressed buttons varied widely from 20 up to 100. The average was around 45, slightly less than the real number of presses needed for calculation. In mock-up tests on the computer people highly underestimated the number of operations needed. We believe it is due to the delay caused by the serial communication with the computer that people were more aware of the number of tasks. The test subjects didn t perceive any pattern apart from the fact that some colours appeared more often than others. This was due to the unequal appearance of the basic operators (AND, OR and NOT) in this application. The FULL ADDER uses AND operators significantly more than OR and NOT. Slightly more than 1 percent of all operations that humans had to perform were done incorrectly. Most of these errors can be attributed to errors in colour perception (red versus orange and blue versus cyan). Almost all of the errors altered the final outcome in our calculator application. Because the error can appear on any bit, the order of magnitude of the error on the final answer in our example application can be as large as the most significant bit. DISCUSSION We trust the reader to understand that this study is meant to be conceptual in nature, and not as a practical implementation of computer-human interaction. It is presented to initiate explorations of such interactions in novel ways. It is possible to include humans into a computation without their knowledge. For this experiment to work, it needed two carefully constructed components: Algorithm Contract of meaning An algorithm had to be constructed so the computation could be deconstructed in little tasks. Part of the computation can be said to lie in this very decomposition. The programmer makes use of something we call contract of meaning. The action of the human has to translate into a meaningful input for the computer. In the case of the hidden task model with the colors, there is circuitry in place how to interpret the buttons pushed by the human. This contract is based on world knowledge of the programmer. In conventional personal computers it is based on our knowledge of electromagnetism. In this hidden task model, it is the knowledge of color vision. This contract must be based on a process with fixed rules [1]. One of the unique features of this computer based on humans is its inherent fallibility. Where other computer models in the unconventional computing domain rely on systems with clear rules with low margins of error, the human processor relies on macroscopic effects for which we do not have closed mathematical models. One could argue this is a failing of this computer or a feature. We like to point out the parallel between the human processor and an over clocked normal processor. In both cases, resulting errors can lead to wrong results and in some cases to system crashes. We advise people using the human computer architecture that run into similar problems to replace their human with a more reliable one. This research could be extended by trying to implement the other two components of the Von Neumann architecture: the control unit and memory. This should be feasible since these components can be built from the same sufficient operators. It makes more sense however to implement memory in a very different way. Memory uses time delay as a fundamental principle. The most used system is a flip-flop architecture, but there are nice alternatives, for example the Hello World project by Yunchul Kim[8] used an audio signal in a closed feedback system. An interesting extension to this research would be to build a compiler for the human processor. Where traditional compilers optimize for speed or memory usage, the human compiler could try to minimize the amount of required primitive operations or try to avoid actions where humans make more mistakes. This could lead to atypical choices for compilation optimization. It might be possible to find a more natural basis to build a human processor on. We used the Boolean foundation, because it is easy to show that it is functionally complete. It is however not the most natural basis for a human processor and we hope that this paper will inspire others to explore these ideas. Traditionally imperfections in the interaction between users and machines are seen as flaws in the interface or the interpretation of the computer. While we do not contest the idea that perfecting the interface is a good idea, we wish to convey the idea that no amount of perfecting can eliminate all errors, because humans are not flawless entities that can be endlessly corrected by intelligent HCI. Part of the responsibility of the interaction between the human user and the computer should lie with the user. REFERENCES 1. Abbott, R. If a Tree casts a Shadow is it telling the Time? Journal of Unconventional Computation, 5(1):1 28, Adams, D. The Hitchhiker s Guide to the Galaxy. Harmony Books, Adleman, L.M. Molecular Computation of Solutions to Combinatorial Problems. Science, 266(5187): , Caldwell, C., and Johnston, V.S. Tracking a criminal suspect through face-space with a genetic algorithm. Proceedings of the Fourth International 78

5 Conference on Genetic Algorithms, pages Morgan Kaufmann, Dawkins, R. The Blind Watchmaker. W. W. Norton & Company, Inc., Feynman, R.P. Simulating Physics with Computers. Feynman and Computation: Exploring the Limits of Computers, Fredkin, E., and Toffoli, T. Conservative Logic. International Journal of Theoretical Physics, 21(3): , Kim, Y. Hello, world!, Kosorukoff, A. Human Based Genetic Algorithm. IEEE Transactions on Systems, Man and Cybernetics, SMC-2001, 5: , Prakash, M., and Gershenfeld, N. Microfluidic Bubble Logic. Science, 315(5813):832, Sims, K. Artificial Evolution for Computer Graphics. Computer Graphics, 25(4): , Surowiecki, J. The Wisdom of Crowds. Random House Large Print, Turing, A. On Computable Numbers, with an Application to the Entscheidungsproblem. Proceedings of the London Mathematical Society, 42(2): , Von Ahn, L., Liu R., and Blum, M. Peekaboom: a game for locating objects in images. Proceedings of the SIGCHI conference on Human Factors in computing systems, pages 55-64, Von Neumann, J. First Draft of a Report on the EDVAC. Technical report, University of Pennsylvania,