Every computer system requires human involvement, so the design has to start from what humans can perceive, remember, reason about and physically do. This chapter is the inventory of those limits and possibilities — and the vocabulary the examiner expects you to use on a case.
01The human information processor
Two claims to memorise:
- Human information processing differs from computation because perception, memory, problem solving and motor response are less stable but also more adaptive.
- All the visual, textual, sound and touch input becomes an integrated whole and recedes in memory — sometimes into a single cell (faces, places).
Input, by channel
| Channel | Physical reception | Interpretation | Design use |
|---|---|---|---|
| Visual — image | Light, retina, ganglion cells | 3D, colour | Images, icons, layout |
| Visual — text | Same eye, but coded | Semantics, pragmatics | Labels, instructions, messages |
| Auditory — sound | Pressure waves, cochlea | Semantics, pragmatics | Sonification of the interface, alerts |
| Auditory — speech | Same | Semantics, pragmatics | Spoken commands, conversational agents |
02Memory: three stores, one moving target
- Sensory memory (iconic for vision, echoic for sound) — very brief buffer; most of it never reaches awareness.
- Short-term / working memory — small capacity, holds the current task. This is why “reduce short-term memory load” is a design principle and why recognition beats recall.
- Long-term memory — effectively unlimited, but reconstructive rather than photographic.
The neuroscience the lecture actually wants you to quote
- The brain is ~1.5 kg with ~150,000 km of white-matter wiring (four times around the earth), about 70 billion brain cells. Complex, not chaotic.
- Connections are selective: each nerve cell contacts just one other cell, avoiding all others. Fried (2005) found the “Jennifer Aniston neuron”; Quian Quiroga found cells for Oprah Winfrey, the Golden Gate Bridge, the Eiffel Tower. Highly specific information (faces, places) can be stored in one or two cells.
- Memory physically changes the brain. When you remember, new contact points appear and others fade. Memory is therefore not fixed, it changes over time, and it is context-sensitive.
- Ramirez et al.'s false-memory experiment: a mouse explores box 1 with no shock; in box 2 it receives a shock; back in box 1, activating the neurons associated with the box-2 fear produces a fear response without any shock. The memory of the place was re-associated with fear.
- About 84% of all genes are active in the adult brain; each neuron type uses a specific range of genes (light receptors make photon-catching proteins; neurons in the substantia nigra make dopamine, central to reward and affective processing).
03Problem solving: the vocabulary
| Term | Definition to reproduce |
|---|---|
| Problem solving | The strategies, means and procedures used to achieve a goal; “how do I get what I want?” |
| Problem | The gap between the current state and the goal state. |
| Goal state | The desired end state or solution of the problem-solving activity. |
| Sub-goal | A middle-term goal along the way; goal decomposition breaks a complex problem into smaller ones by determining sub-goals. |
| Problem space | The short-, middle- and long-term goal states and the information needed at each step. Solving = “X should change so as to overlap Y”. |
| Well-defined problem | Full specification of the provided data, actions, strategies and goal states, which can be represented in a problem space. Example: find the missing semicolon. |
| Ill-defined problem | Partial or imprecise specification of data, actions, strategies and goal states, which cannot be entirely represented in a problem space. Example: a mission statement. |
Goal satisfaction is often incomplete: people accept a good-enough solution rather than the optimum. That is a fact about users, not a flaw in your design — design for satisficing.
The lecture's mission-statement example shows goal decomposition in action: “mission: online community” decomposes into business goals (aid e-commerce and edutainment, revitalise economic partnerships, publish enlightening websites, maintain personal and company ethics), each with targets such as “5% up in two years”.
04Seven strategies, and when each one fails
| Strategy | How it works | Where it breaks |
|---|---|---|
| Rule following | Apply a known procedure. | No rule for a novel situation; rules can be wrong or conflicting. |
| Difference reduction | Find the difference between current state and goal, and take the step that reduces it most visibly. | Local maxima. It never takes a step that increases the distance, so it cannot cross a valley. Also determines your technology: the lorry example forces a smaller lorry that makes more miles and more wear. |
| Means–end analysis | Compare current and goal state, and create sub-goals for the differences — even sub-goals that temporarily increase the distance to the goal. | Cognitively expensive; needs a model of the whole space. In the lorry example it justifies one big lorry that transports everything at once. |
| Forward reasoning | From data and rules to a conclusion: if A = 1 and B = 2 then C = 3; if C = 3 then D = 4. Cause known → effect predicted. | Only valid for known problems with a one-to-one relationship. |
| Backward reasoning | From symptom or goal back to the cause: A = 1 and D = 4, what is missing? | If more than one explanation fits the same phenomenon, neither direction is allowed without empirical validation excluding all causes but one. |
| Analogical reasoning | Find relations between features and categories that are similar to relations elsewhere. Clock signal : computer :: heartbeat : human being; case-based reasoning is a type of analogy. | Halfway intelligent, half associative: the analogy can be superficially appealing and functionally wrong (woodpecker → camp axe works; planetarium ↔ mosque is ambiguous). |
| Random trial and error | Low-level strategy, common with novices: follow the first association. | Violates the demand for systematic analysis and planned action. Classic evidence: novices look for Find under Edit because “find” is not semantically related to editing; they end up in Tools ▸ Goal Seek and start setting cells to values. |
| Creative problem solving | Give a different function to the standard means available — fuse two domains. | Unpredictable; hard to evaluate before it exists. |
Creative problem solving, in the lecture's examples
- OmniTouch — a wearable projector turns any surface (a table, your hand) into a touchscreen; the means (a projector) is given a new function (an input surface).
- Worm robot — locomotion of a nematode applied to a search-and-rescue robot.
- Crowd PC — use the back cover of a closed laptop as an interface.
- Deep learning — “signal detection in an advanced configuration”. Layers detect lines → edges → features → categories; weights encode relative importance and are tuned by training data. Its errors are interesting: an elephant face and Joseph Merrick share eyes, lumpy ears and a puffy face; a “car face”, an elephant chair. Hybrids exist, so error is only error in a rigid system — taxonomy should be fuzzy, and misclassification is at the basis of creative crossover.
05Representation: how you frame it decides the answer
How you represent a problem affects its solution — also in user interfaces. The lecture uses one problem (ice melting) at four levels of aggregation. Each level produces a different family of solutions, and changing one level changes the others: that is insight.
| Level | What it foregrounds | Ice-cream solutions it suggests |
|---|---|---|
| Surface / phenomenological | Colour changes, shapes, texture, deformation of solids, sun, stains | Walk in the shadow, use napkins, change detergent |
| Structural | Loss of structure, processes and mechanisms, work units and actions | Add emulsifiers or more sodium chloride; manage crystal formation so the ice does not turn gritty |
| Symbolic | Logic, reasoning, abstraction, formal transitions | H₂O(s) ⇌ H₂O(l); protein binding; derive the conditions from the equation |
| Molecular | Atoms and ions, changes in structure and process (animation) | Simulate the transition and intervene at the molecular level |
Text vs image. Text explains abstractions better (“infinity”, “intelligence”); images work better for spatial orientation and localisation — where is what. A designer should therefore make multiple representations, each addressing another aspect of the work to be done.
Two mindsets — name them and their habitat
| Logic rationalist | Associative empiricist | |
|---|---|---|
| Feels at home in | Closed systems — “like a fish in a tank” | Open systems — “like a bird in a tree” |
| Decision type | Structured decisions, usually well-defined problems with a precise problem state | Unstructured decisions, usually ill-defined problems with a vague problem state |
| Risk | Cannot cope when the problem space is not closed | First association may be wrong — but empirical association followed by verification can lead to better accuracy |
Neither is “stupid”. The lecture's punchline is that the product life cycle follows the innovation S-curve: exploration rewards associative breadth, exploitation rewards rational discipline.
06Motor output: Fitts' law
Fitts (1954), a US Air Force lieutenant-colonel and later professor, modelled pointing: the time to move a pointer to a target is a function of the distance divided by the target's size. The wider the widget and the closer it is to the pointer, the easier to hit it — particularly when the pointer is wide too (a fat finger).
T = a + b · log2(2D / W)
ID = log2(A/W + 1) · IP = 1 / b
T movement time · D (or A) distance to the target · W target width (the tolerance) · a y-intercept (start-up cost) · b slope in seconds per bit · ID Index of Difficulty in bits · IP Index of Performance, a.k.a. throughput. The original formulation is ID = log2(2A/W); the Shannon-style version uses log2(A/W + 1), or log2(A/W + 0.5) when ID < 3 bits.
Why a logarithm — four reasons, any of which earns a mark
- Speed–accuracy trade-off. Farther or smaller targets are harder, but difficulty does not grow linearly; it grows more slowly as you keep making the task harder.
- Information theory. Fitts was inspired by Shannon: pointing is “transmitting information” from brain to muscles, and information capacity is measured in bits.
- Empirical fit. A logarithmic relationship fitted the measured data better than a linear one.
- Doubling/halving effect. Double the distance → the log term rises by 1 bit; halve the width → also 1 bit. Twice as hard is not twice as long, only a little longer.
Reading the device off the constants
- ID depends on the task, not on the device: distance and width are properties of the target.
- a and b depend on the device and the user: the regression constants say how fast that modality completes the task. b is the slope in seconds/bit; IP = 1/b is the throughput.
- Fitts' law therefore measures the efficiency of system performance — one of the three performance criteria (effectiveness, efficiency, effort).
- Fitts models the experience side of performance (human movement), not the physical side of the hardware.
Fitts is also the explanation of the fat finger
A tiny on-screen target, a wide pointer (finger), high stress and no confirmation: the Mizuho trade is a Fitts problem, not a moral one. Reduce distance, enlarge the target, or add a confirmation step that costs one bit of attention instead of US$225 million.
07Worked example — do this once by hand
A button is 400 px from the current pointer position and 40 px wide. The device has a = 0.15 s and b = 0.12 s/bit.
- Shannon ID = log₂(400/40 + 1) = log₂(11) ≈ 3.46 bits.
- T = 0.15 + 0.12 × 3.46 ≈ 0.15 + 0.42 = 0.57 s.
- Move the button twice as far (800 px): ID = log₂(21) ≈ 4.39 bits → T ≈ 0.15 + 0.53 = 0.68 s. Difficulty rose by ~0.93 bit, time by ~0.11 s.
- Halve the width instead (20 px): ID = log₂(21) ≈ 4.39 bits — the same increment. Doubling distance and halving width cost the same one bit.
- If the original formula is used, ID = log₂(2×400/40) = log₂(20) ≈ 4.32 bits. State which formulation you used; the marker is checking the method, not the third decimal.
08Time scales, reaction time, and mental chronometry
Response time = reaction time (perceive and decide) + movement time (motor execution). Reaction time depends on stimulus type and is measured in milliseconds; movement time is what Fitts predicts.
| Band | Scale | Behaviour |
|---|---|---|
| Biological | 10⁻⁴ – 10⁻³ s | Neural and muscular events |
| Cognitive | 10⁰ – 10¹ s | Immediate behaviour: perceive, decide, act (reaction times sit here) |
| Rational | 10² – 10³ s (minutes–hours) | Adaptive behaviour, problem solving |
| Social | 10⁵ – 10⁷ s (days–months) | Social behaviour, routines, culture |
- Typical values quoted in the deck: visual ≈ 200 ms, pain ≈ 700 ms; auditory is the fastest channel. Movement time adds on top and depends on the subject's age and fitness.
- Mental chronometry measures the speed of cognitive processing — used to assess the complexity of mental operations, practice and learning effects, decision-making, attentional mechanisms, and to compare populations (age groups, clinical vs non-clinical).
- The gaming experiment: a life-size projection of a hotel lobby, two elevators, an opponent who may or may not appear after a warning; the participant fires an infrared rifle. Measured: reaction time of pulling the trigger and accuracy of the shot. The model behind it is temporal preparation — the effective preparation period (EPP) runs from warning signal (S1) to target (S2), and SOA is the stimulus onset asynchrony between them.
09Concept check
Question 1A target is 600 px away and 60 px wide. Using the Shannon formulation, what is the Index of Difficulty?
Question 2In T = a + b·ID, which statement is correct?
Question 3When is a problem “well-defined”?
Question 4A dispatcher always sends the van to the nearest town first, and runs out of battery. Which strategy did he use, and which would have worked?
Question 5A user reports a crash you have never seen, and three different causes would each explain it. What does the lecture allow?
Question 6“Solutions become focused on colour changes, shapes, texture, deformation of solids, sun and stains.” Which representation level is this?
Question 7Which claim about memory is false according to the lecture?
Question 8Response time in a pointing task is best described as…
10Exam-style questions
(a) Describe the parameters that participate in Fitts' law. (5 marks)
(b) A VR user selects an object and places it at a target location anywhere in 3D space. Can Fitts' law be used to assess this picking task? Justify yes or no. (5 marks)
Show model answer
(a) T = movement time; D (or A) = distance from start to target; W = width of the target, i.e. the tolerance for error; a = intercept, the start-up cost of the modality; b = slope in seconds per bit; ID = log₂(2D/W) or log₂(D/W + 1), the Index of Difficulty in bits; IP = 1/b, throughput. ID depends on the task; a and b are empirically determined per device and user. One mark each, up to five.
(b) Yes. The task is still pointing: the hand or controller travels a distance to a target with a finite tolerance, so the same information-theoretic model applies. What changes is that the target is a sphere (or box) in three dimensions, so the effective width is the projected tolerance along the approach direction, the distance includes the depth axis, and the constants must be re-estimated for the controller, the reachable volume and the visual feedback in stereo. Selection can also require a second action (grab, release), which must be modelled as a separate pointing subtask. A defensible “no” earns marks only if it argues that 3D pointing is not reducible to one D and one W (e.g. two-stage selection, occlusion, no fixed frame of reference) — but the lecture position is that the law still applies.
Marking note. In (b) the decisive phrase is “the form remains, the constants change”, the same answer the lecturer gave for Fitts in microgravity.
A dispatcher must deliver to towns X, Y and Z with one electric van and return with the highest possible charge. On screen, X is the closest (5 km) but uphill (3× energy); Y and Z are 15 km on flat roads. The dispatcher says: “X is right there, I'll get that one off the list first”, and strands the van.
(a) Characterise his problem-solving style and representation level, with evidence. (3)
(b) Name the strategy he used and explain why it failed. (4)
(c) Which strategy should the system have encouraged, and how does it differ? (3)
Show model answer
(a) He is an associative empiricist, not a logic rationalist: he reasons from the first salient association (“nearest = easiest”) in an open, under-specified situation rather than closing the problem formally. He works at the surface / phenomenological level — the map's apparent distance — and ignores the structural–symbolic information (energy per gradient, battery state as a constraint). Evidence: “right there”, “closest to the depot”, “get that one off the list”.
(b) Difference reduction. He repeatedly takes the action that most visibly reduces the difference between current state (3 undelivered towns) and goal state (all delivered, van home). It fails because the metric is wrong: the visible difference counts towns, not charge. Because the strategy never accepts a step that increases the distance, he cannot take the “detour” of doing the far, flat towns first — a local maximum.
(c) Means–end analysis. It decomposes the goal into sub-goals — “preserve enough charge to return” — and is willing to accept a sub-goal that temporarily increases the distance to the final goal (drive past X to Y and Z, come back to X last, or recharge mid-route). The system should therefore display energy, not only distance, and propose an ordering that satisfies the constraint.
Marking note. Naming the strategy without the current-state/goal-state mechanism caps (b) at 2. The “local maximum” phrase is worth a mark.
Explain how the representation of a problem changes its solution, using the ice-cream example at three levels. What should a designer do with this fact when a client presents a requirement?
Show model answer
- Surface: the problem is “melting looks messy” → solutions about colour, texture, deformation, stains: napkins, shadow, detergent.
- Structural: the problem is “loss of structure and gritty crystal formation” → emulsifiers, sodium chloride, process control.
- Symbolic: the problem is a phase transition H₂O(s) ⇌ H₂O(l) → solutions derived from logic and binding chemistry.
Each level makes a different family of solutions visible, and changing one level changes the others — that is where insight comes from. A designer should therefore make multiple representations, each addressing another aspect of the work, and should draw in more information than the representation the client offers. That is problem finding and opportunity seeking: the stated requirement is one level, the actual need may be another (the client's “legal issue” may really be a business-model question).
Marking note. Three levels correctly instantiated = 3; the designer's move (multiple representations + reframing) = 2; a concrete example of reframing = 1.