HCI has always been about conversation: an input language for the user, an output language for the machine, and a protocol for the exchange. Week 7 walks that ladder — bits and syntax, designed dialog, then the ambiguity of human language that makes the last rung hard.
The midterm covers material up to week 7, which is Lecture 04, Part 1. The second half of that lecture — large language models, transformers, speech acts, Grice's maxims, scripts — is week 8 and is not examinable. Neither is the Figma tutorial: you are expected to know what Figma is for (a high-fidelity prototyping tool), not how to operate it.
01Three kinds of language in HCI
| Language | Who adapts to whom | Example |
|---|---|---|
| Coding language | The human adapts to the machine — a compromise between grammar-oriented assembly and semantics-oriented human language | Machine code → assembly → FORTRAN → Python |
| Dialog language | The designer adapts the machine's turns to the user | An ATM: insert card → PIN → menu → amount |
| Natural language | The machine adapts to the human — or fails to | “There's a power reduction in section L-94. Explain.” |
Why bother with natural language at all? It is the most natural and inclusive interface: it lets non-experts drive complex systems, it exploits common ground instead of teaching a command vocabulary, and it is abstract and minimal — a sentence can carry a whole task. That is the promise; the rest of the chapter is the price.
FORTRAN (1957) was the first widely used high-level language, written so scientists could type formulae. Programmers derided it as “ease of use at the expense of optimal use of resources”. That trade — the machine gives up control, the human gains legibility — is the whole of HCI in miniature, and it is the only thing from the coding-language section worth carrying into an exam answer.
02Three levels of every dialog
| Level | Question it answers | Car infotainment example |
|---|---|---|
| Lexical | What does the control look like, where is it, how do I operate it? | The volume knob's shape, size, position, and whether it turns or slides |
| Syntactic | In what order must inputs and outputs occur? | You must select “Media” before “Bluetooth”; pairing is disabled while driving |
| Semantic | What effect does the action have on the application's data? | The paired phone replaces the previous one; the preset is overwritten |
Dialog itself is the structured exchange of input and output between user and system over time. Designing it is how you manage complexity: help menus, escape options, and a predictable next step. It is the same question as “how much state does the user have to hold in their head?” from chapter 2.
03Notations for dialog design
| Notation | Kind | Good at | Weak at |
|---|---|---|---|
| State transition network (STN) | Diagram | Sequential, state-dependent dialogs: ATMs, ticket machines, wizards | Combinatorial explosion when actions run concurrently; no natural hierarchy |
| Hierarchical STN | Diagram | Nesting a sub-dialog inside a super-state | Still no concurrency |
| Concurrent STN | Diagram | Two dialogues at once — music playing while navigating | States multiply; escapes become duplicated spaghetti |
| UML | Diagram | Use cases, activity, sequence, state machines; speaks to engineers | Heavyweight; the notation is not the design |
| Flowchart | Diagram | Familiar; algorithms and one-off procedures | Encourages thinking of the interface as a program, not a conversation |
| BNF / production rules | Textual | Grammar of a command language: legal sentences, recursion, ambiguity detection | No notion of state or user; purely syntactic |
| CSP (communicating sequential processes) | Textual | Formal reasoning about concurrency and deadlock | Mathematical overhead; rarely read by designers |
The trade-offs the lecture names, in pairs: event-based vs state-based; power vs clarity; model vs notation; sequential vs concurrent. An exam question that asks you to “choose a notation and justify it” is asking you to pick a side in one of these.
04STN, worked — the ATM
Rules of the notation: a circle is a state, an arc is an action or event, and an arc label reads action / condition. Every state should be reachable and every state should have a way out — including a way out that abandons the task.
How to draw one under exam pressure
- Underline the nouns that are states (idle, card inserted, PIN accepted) and the verbs that are arcs (insert, type, choose, confirm).
- Draw the happy path left to right. Do not decorate it yet.
- Add the failure branches: wrong PIN (retry ×3), insufficient funds, invalid amount.
- Add one escape mechanism — a cancel action from every non-committed state to a single abort state, and from there back to idle.
- Check reachability: is every state reachable? Is every terminal state escapable? Does any state have no outgoing arc? A state with no exit is a trap, and a trap is a design bug.
05Why natural language is hard: ambiguity
Natural language is ambiguous by design, and — this is the examinable point — we do not notice it. Four numbers carry the argument:
| Claim | Number | What it shows |
|---|---|---|
| English words for a person | > 5,000 | Synonyms are everywhere; “which noun describes this man?” has no unique answer |
| English words for move / sound / noise | 4,000 / 1,148 / 2,037 | Verbs of motion and sound are wildly over-specified |
| English words for smell and taste | a few dozen | Vocabulary is uneven across senses — yet humans discriminate > 1 trillion smells and ~half a million tones |
| Meanings of the 121 most frequent nouns | 7.8 each, ~20% of all word occurrences | Ambiguity is concentrated in the words you use most |
| Combinations for one ordinary sentence | ~14 quadrillion | “He gave a soft ball across the line…” — the machine must choose among 1.4 × 1016 parses |
That last number is why “computers are much better at chess than at understanding a shopping list”. Chess has rules and a bounded state space; language has neither.
06The NLP pipeline — five stages, five failure modes
| Stage | What it does | Where it breaks |
|---|---|---|
| Tokenisation | Splits a character string into words, numbers, operators; then maps each token to a vector (an integer id) so the machine can process it. | Whitespace is not universal. Chinese has no spaces, so word segmentation needs dictionaries and statistics. Turkish is agglutinative — yazıcılarımızdakilerden (“from those which our writers have”) is one word, and the longest Turkish word translates as a whole clause. |
| Part-of-speech tagging | Labels each token with its grammatical class: noun, verb, adjective, adverb, pronoun, preposition, determiner… | One word, many tags — “work” is a noun or a verb. Tagging is therefore probabilistic: choose the tag sequence that maximises the product of word-given-tag and tag-given-previous-tag probabilities. |
| Syntactic analysis (parsing) | Converts the linear sentence into a hierarchical parse tree, using a grammar — a formal specification of the structures the language allows. | Attachment ambiguity: “chased the cat with a stick” — is the stick an instrument of chasing or a possession of the cat? The tree is the ambiguity, drawn. |
| Semantic analysis | Marks the tree with meaning: proper names, gender, number, animacy, cause and effect, location, abstract vs concrete. | Quantifier scope, metaphor, negation — and it inherits every error from the stages above. |
| Coreference resolution | Finds all expressions that refer to the same entity — pronouns, names, noun groups — and links them. Usually heuristic: gender, number, and the arguments of the phrases. | “There's a power reduction in section L-94. Explain (it).” What is “it”? The power reduction, the section, or the event? A human resolves it instantly; a system must decide explicitly. |
Errors cascade. A tokenisation mistake corrupts the tags, the tags corrupt the parse, the parse corrupts the semantics. That is why accuracy decays with sentence length, and why “just add more data” does not fix a pipeline whose first stage is wrong.
Two design notes the lecture adds. Modern systems skip the hand-built pipeline: they predict the next token (“He ate his noodles with …” — “motor oil” has low likelihood), and predicting a masked word is called masked language modelling. And the highest-probability word is not always the best choice, so some randomness — temperature — is added to vary the wording.
07Parse trees and the compiler appendix
The parse tree is the output of syntactic analysis, the second stage of the compiler's analysis phase. It sits there because the compiler must know the structure of the program before it can attach meaning to it — the same order as the NLP pipeline.
S
┌───┴────────────┐
NP VP
┌───┴────┐ ┌──┴──────┐
Det N V NP
the dog chased ┌───┴───┐
Det N
the cat
# analysis phase: lexical → syntax (this tree) → semantic
# synthesis phase: intermediate code → optimise → code generation
The compiler architecture sits in Appendix 1 of the week-7 deck, and the deck closes it with “Appendices are not for the exam”. One exam sample item nevertheless asks where the parse tree fits. The safe position: know that a parse tree is the product of syntactic analysis and that it precedes semantic analysis — one sentence, no more. Do not memorise the seven-phase compiler diagram.
08Concept check
Question 1“You must enter the PIN before selecting an amount.” Which dialog level is this?
Question 2In a state transition network, what does an arc represent, and how is it labelled?
Question 3Why does a concurrent STN explode, and what is the standard mitigation?
Question 4“In Turkish, yazıcılarımızdakilerden is a single word meaning ‘from those which our writers have’.” Which NLP stage does this most directly complicate?
Question 5“There's a power reduction in section L-94. Explain it.” Deciding what “it” refers to is the job of…
Question 6Syntactic analysis (parsing) converts a linear sentence into…
Question 7Why is randomness (“temperature”) added when predicting the next word?
Question 8Which dialog notation is textual rather than diagrammatic?
09Exam-style questions
A campus library wants a self-service kiosk. A student scans an ID card; the system shows the borrowed items; the student may renew all items, renew one item, or pay a fine by card; after any action the student may print a receipt or finish; at any point before payment they may cancel and the session ends. A payment must not be cancellable once the card has been charged.
Draw a state transition network. Label all arcs and mark the escape behaviour. Then state one dialog property your diagram guarantees and one it does not.
Show model answer
Structure (marks are for topology, not artwork):
- idle —(scan ID / valid)—▶ item list
- item list —(renew all / items renewable)—▶ confirmation
- item list —(select item, renew / renewable)—▶ confirmation
- item list —(pay fine / fine > 0)—▶ amount due —(insert card / amount ok)—▶ processing —(approved)—▶ receipt
- confirmation —(print)—▶ receipt; —(finish)—▶ idle
- receipt —(take receipt)—▶ idle
- Escape: a single cancel/abort state reachable from item list, confirmation and amount due, returning to idle. No escape arc from processing — once the card is charged the transaction must complete, so the only exit is approved or declined → amount due.
- Failure arcs: invalid card → idle; declined → amount due; nothing to renew → item list with a message.
Guarantees: every non-terminal state is reachable from idle and has at least one exit; the payment commit point is provably unescapable, so the system cannot take money without producing a receipt state.
Does not guarantee: that the user knows where they are (no visibility of state), nor that concurrent sessions on other kiosks are consistent, nor timing or error recovery beyond the modelled arcs. An STN is a model of legal sequences, not of understanding.
Marking note. Unlabelled arcs −1 each. No escape at all caps at 6. Putting an escape on processing is a semantic error and loses the “property guaranteed” mark too.
Three complaints about the same banking app: (i) “I couldn't find where to change my address”; (ii) “It made me confirm the payee before it showed me the fee”; (iii) “It moved my savings into the current account when I meant to move the other way.” Assign each to a dialog level and give one fix per complaint.
Show model answer
| Complaint | Level | Reason | Fix |
|---|---|---|---|
| (i) cannot find address change | Lexical | The control exists but is not visible, badly labelled, or in the wrong place — the user cannot even form the action. | Put it under “Profile ▸ Personal details” with the word address in the label; make it searchable. |
| (ii) fee shown after confirmation | Syntactic | The order of inputs and outputs is wrong: a commitment is requested before the information needed to make it. | Move the fee display before the confirmation step; recompute and show it inside the confirmation dialog. |
| (iii) transfer direction reversed | Semantic | The action's effect on the application's data is the opposite of the user's intention — a “from/to” mapping error. | Label both ends explicitly (“From: Savings → To: Current”), show a preview of the resulting balances, and offer undo. |
Marking note. Level + mechanism + fix, roughly 2 + 1 per row; a fix that is only “make it clearer” earns nothing.
Commander Riker says to the computer: “There's a power reduction in section L-94. Explain.”
(a) Name the five stages of the NLP pipeline and say what each would have to produce for this utterance. (5)
(b) Identify two ambiguities the system must resolve, and name the stage responsible for each. (3)
(c) The ship's computer answers correctly. Explain why that response requires more than a parse tree. (2)
Show model answer
(a) Tokenisation: split into tokens and map to ids — There / 's / a / power / reduction / in / section / L / - / 94 / . / Explain, with decisions about “'s”, “L-94” and the full stop. POS tagging: power/NN reduction/NN in/Prep section/NN — note “power” could be a noun or a verb, and “explain” is an imperative. Parsing: build the tree — “power reduction” is a noun compound, “in section L-94” attaches to the reduction (location), “Explain” is the main verb with an elided object. Semantic analysis: mark power reduction as an event/phenomenon, section L-94 as a location, and the imperative as a request. Coreference resolution: link the elided object of “Explain” to the power reduction.
(b) Any two: “it”/elided object — what is to be explained, the reduction or the section (coreference); “power” — noun vs verb, and electrical vs computational power (tagging/semantics); “section L-94” — a compartment, a subsystem, or a document section (semantics); “Explain” — a request for a cause, a status report, or a justification (pragmatics, beyond the five stages).
(c) The parse tree only says what the sentence's structure is. To answer, the system needs a model of the ship — that L-94 is a section, that a power reduction there is an event with a cause, and what an “explanation” is for. That is knowledge and intent, not syntax: the lecture's point that understanding is reference to the world, not to the grammar. This is also why the answer must name the referent explicitly (“unoccupied sections are being closed to clear space”), rather than restating the question in different words.
Marking note. (a) 1 per stage, but only if the stage is applied to this sentence. (b) stage + ambiguity + why, 1.5 each. (c) must say “world knowledge / reference”, not “more data”.