Cite

This course has no released version yet: the text at this address may change. Cite a released version when one exists.

/courses/models-of-computation/programming-turing-machines
@misc{models-of-computation-programming-turing-machines,
  title = {{Programming Turing machines}},
  howpublished = {Lesson 3 of \emph{Models of computation}, Computability Theory course notes},
  url = {/courses/models-of-computation/programming-turing-machines},
  note = {Unreleased working version}
}

To cite a single statement, use the # link that appears next to its heading.

Lesson 3 of 13

Programming Turing machines

Auxiliary symbols, sequential composition of machines, translation invariance, multi-tape machines and their simulation by single-tape machines.

40 min5 questions
Before this lesson:Turing machines
Models of computation · lesson 3 of 13
  1. Algorithms and partial functions
  2. Turing machines
  3. Programming Turing machines
  4. Register machines
  5. Primitive recursive functions
  6. Coding finite sequences
  7. The Ackermann function
  8. Minimization and partial recursive functions
  9. LOOP and WHILE programs
  10. The λ-calculus
  11. Computing with λ-terms
  12. Equivalence of the models
  13. The Church–Turing thesis

Writing Turing machines directly is laborious, but a few techniques make it systematic. This lesson presents them and proves that several natural extensions of the model compute no new functions.

3.1 Auxiliary symbols: a doubling machine

The alphabet may contain symbols other than ⊔\sqcup and 11. They serve as markers recording which part of the input has already been processed. As an example, we compute x↦2xx \mapsto 2x. On input ⌜x⌝=1n\ulcorner x \urcorner = 1^{n} with n=x+1n = x + 1, the output must be ⌜2x⌝=12n−1\ulcorner 2x \urcorner = 1^{2n - 1}.

The machine works in two phases. In the first phase, it repeatedly replaces the leftmost unmarked 11 by XX and appends a YY to the right end of the tape contents. After nn rounds the tape contains XnYnX^n Y^n. In the second phase, it turns every YY into 11, erases the last symbol, and moves left turning every XX into 11. The alphabet is {⊔,1,X,Y}\{\sqcup, 1, X, Y\}:

11 XX YY ⊔\sqcup
q0q_0 (q1,X,R)(q_1, X, R) — (q3,1,R)(q_3, 1, R) —
q1q_1 (q1,1,R)(q_1, 1, R) — (q1,Y,R)(q_1, Y, R) (q2,Y,L)(q_2, Y, L)
q2q_2 (q2,1,L)(q_2, 1, L) (q0,X,R)(q_0, X, R) (q2,Y,L)(q_2, Y, L) —
q3q_3 — — (q3,1,R)(q_3, 1, R) (q4,⊔,L)(q_4, \sqcup, L)
q4q_4 (q5,⊔,L)(q_5, \sqcup, L) — — —
q5q_5 (q5,1,L)(q_5, 1, L) (q5,1,L)(q_5, 1, L) — (qh,⊔,R)(q_h, \sqcup, R)

Entries marked — are never used; to make δ\delta total, set them to (qh,⊔,R)(q_h, \sqcup, R), say.

Proposition 3.1.#

This machine computes x↦2xx \mapsto 2x.

Proof of Proposition 3.1.

Let n=x+1n = x + 1. We show by induction on j∈{0,…,n}j \in \{0, \dots, n\} that the machine reaches the configuration in state q0q_0 with tape Xj1n−jYjX^j 1^{n-j} Y^j written from cell 00 and the head on cell jj. For j=0j = 0 this is the initial configuration. Suppose it holds for some j<nj < n. Cell jj contains 11: the machine writes XX and enters q1q_1, which moves right across 1n−j−1Yj1^{n-j-1} Y^j to the first blank, at cell n+jn + j, writes YY there, and enters q2q_2. In state q2q_2 the head moves left across YYs and 11s until it reaches the XX on cell jj, and enters q0q_0 on cell j+1j + 1. The tape is now Xj+11n−j−1Yj+1X^{j+1} 1^{n-j-1} Y^{j+1}, as required.

For j=nj = n, the tape is XnYnX^n Y^n and the head scans the first YY, on cell nn, in state q0q_0. The machine writes 11 and enters q3q_3, which turns the remaining YYs into 11s and reaches the blank on cell 2n2n. It then erases cell 2n−12n - 1, which contains 11, and enters q5q_5 on cell 2n−22n - 2. Note that 2n−2≥02n - 2 \geq 0. State q5q_5 moves left, turning 11s and XXs into 11s, down to cell −1-1, and halts on cell 00. The final tape is 12n−11^{2n - 1} on cells 0,…,2n−20, \dots, 2n - 2, that is ⌜2x⌝\ulcorner 2x \urcorner.

3.2 Translation invariance

The behavior of a machine depends only on the relative position of the head and the symbols. Formally, for a tape τ\tau and d∈Zd \in \mathbb{Z}, let τ+d\tau^{+d} be the translated tape τ+d(i)=τ(i−d)\tau^{+d}(i) = \tau(i - d).

Lemma 3.2.#

If (q,τ,h)⊢(q′,τ′,h′)(q, \tau, h) \vdash (q', \tau', h'), then (q,τ+d,h+d)⊢(q′,τ′+d,h′+d)(q, \tau^{+d}, h + d) \vdash (q', \tau'^{+d}, h' + d) for every d∈Zd \in \mathbb{Z}. Consequently, the computation from a translated configuration is the translation of the original computation, and halts after the same number of steps.

Proof of Lemma 3.2.

The symbol scanned in the translated configuration is τ+d(h+d)=τ(h)\tau^{+d}(h + d) = \tau(h), so the same transition (q′,b,D)(q', b, D) is applied. The new tape coincides with τ+d\tau^{+d} except at h+dh + d where it contains bb, which is exactly τ′+d\tau'^{+d}, and the new head position is h′+dh' + d. The second statement follows by induction on the number of steps.

3.3 Sequential composition

The standard final configuration of a machine is, up to translation, an initial configuration on the output. This allows us to run machines one after the other.

Proposition 3.3.#

Let MM and NN be Turing machines computing f:N⇀Nf : \nat \rightharpoonup \nat and g:N⇀Ng : \nat \rightharpoonup \nat. There is a Turing machine M;NM ; N computing g∘fg \circ f.

Proof of Proposition 3.3.

By Exercise 3.4 below, we may assume that every halting computation of MM on an input ⌜x⌝\ulcorner x \urcorner ends in a standard final configuration.

Rename the states of NN so that they are disjoint from those of MM, and take the union of the alphabets. The machine M;NM ; N has the states of both machines, initial state q0Mq_0^M, halting state qhNq_h^N, and the transitions of both machines, except that every transition of MM entering qhMq_h^M is redirected to q0Nq_0^N. The state qhMq_h^M is no longer entered; its transitions, now required since it is not the halting state of M;NM ; N, are set arbitrarily. Entries for symbols of one machine absent from the alphabet of the other are set arbitrarily; they are never used, since MM only sees its own symbols, and NN is started on a tape containing only ⊔\sqcup and 11.

On input xx, the machine M;NM ; N first behaves like MM. If f(x) ⁣ ⁣↑f(x)\diverge, then MM does not halt, and neither does M;NM ; N. If f(x) ⁣ ⁣↓=yf(x)\converge = y, then M;NM ; N enters q0Nq_0^N with a tape containing exactly ⌜y⌝\ulcorner y \urcorner, starting at the head position hh. This is the initial configuration of NN on input yy, translated by hh. By Lemma 3.2, the remaining computation is the translation by hh of the computation of NN on input yy. Since translations preserve standard final configurations, M;NM ; N halts in a standard final configuration with output g(y)g(y) if g(y) ⁣ ⁣↓g(y)\converge, and otherwise it does not. Hence M;NM ; N computes g∘fg \circ f.

Exercise 3.4 ★★.#

Show that for every machine MM there is a machine M^\widehat M with φM^(1)=φM(1)\varphi^{(1)}_{\widehat M} = \varphi^{(1)}_M such that every halting computation of M^\widehat M on an input ⌜x⌝\ulcorner x \urcorner ends in a standard final configuration.

Hint

Let M^\widehat M maintain two end markers delimiting the part of the tape visited so far, and check, before halting, whether the tape between the markers has the form of an output.

Solution

Let M^\widehat M first write a marker ◃\triangleleft on cell −1-1 and a marker ▹\triangleright after the input, then simulate MM. Whenever the simulated head reaches a marker, the marker is moved one cell further (shifting it is easy since it is at the end of the visited zone), and the cell it occupied is treated as blank. When MM would enter qhq_h, M^\widehat M instead enters a checking phase: it records whether the scanned cell holds a 11 preceded by a blank, then scans the zone between the markers and verifies that it contains exactly one block of 11s, starting at the remembered position, and blanks elsewhere. If the check succeeds, it erases the markers, returns to the first 11, and halts. If it fails, it enters a state which moves right forever. Then M^\widehat M halts only in standard configurations, and does so exactly when MM halts in a standard configuration, with the same output.

Composition with functions of several arguments, such as h(f(x),g(x))h(f(x), g(x)), can be handled similarly, using a machine that copies its input. The details are tedious and teach little. We will obtain closure under composition much more easily from the equivalence with the recursive functions, proved later in this course.

3.4 Multi-tape machines

A natural extension gives the machine several tapes, each with its own head.

Definition 3.5.#

A kk-tape Turing machine is defined as in Definition 2.1, except that the transition function has the form

δ:(Q∖{qh})×Γk→Q×Γk×{L,S,R}k,\delta : (Q \setminus \{q_h\}) \times \Gamma^k \to Q \times \Gamma^k \times \{L, S, R\}^k,

where SS means that the head stays in place. A configuration consists of a state, kk tapes and kk head positions. By convention all heads start on cell 00, the input is written on tape 11 as in Definition 2.4, the other tapes are initially blank, and the output is read on tape 11: the machine computes yy if it halts with tape 11 and head 11 in a standard final configuration with output yy, whatever the contents of the other tapes.

Multi-tape machines are much easier to program. For instance, a 22-tape machine can copy its input to tape 22 in a single left-to-right pass.

tape 1abba⊔⊔tape 211⊔⊔⊔⊔tape 3⊔cccc⊔one tape, three tracksa1⊔b1cb⊔ca⊔c⊔⊔c⊔⊔⊔a shaded cell carries the head mark of its track

The simulation of a 33-tape machine: each cell of the single tape stores one cell of every tape, together with a mark telling whether the corresponding head is there.
Theorem 3.6.#

Every partial function computed by a kk-tape Turing machine is computed by a 11-tape Turing machine.

Proof of Theorem 3.6.

Let MM be a kk-tape machine with alphabet Γ\Gamma. The idea is to store the kk tapes on kk tracks of a single tape. Let the new alphabet be Γ∪(Γ×{0,1})k\Gamma \cup (\Gamma \times \{0, 1\})^k: a cell of the simulating machine NN contains either a symbol of Γ\Gamma, or a kk-tuple of pairs (ai,mi)(a_i, m_i) where aia_i is the content of the corresponding cell of tape ii and mi=1m_i = 1 indicates that the head of tape ii is at this cell. This alphabet is finite.

NN first converts cell 00 into this format: it replaces the symbol aa there by ((a,1),(⊔,1),…,(⊔,1))((a, 1), (\sqcup, 1), \dots, (\sqcup, 1)), since all heads of MM start on cell 00. Then NN simulates each step of MM as follows. The state of MM is stored in the state of NN. NN sweeps over the zone of the tape visited so far, from left to right, and collects in its state the symbols under the kk marked heads; there are finitely many possibilities, so they can be stored in the state. Knowing the state of MM and the scanned symbols, NN determines the transition of MM. It then sweeps back from right to left, updating the symbol under each marked head and moving each mark by one cell according to the transition. When a mark must move onto a cell containing a plain symbol aa of Γ\Gamma — a cell that no head has visited yet — NN first replaces it by ((a,0),(⊔,0),…,(⊔,0))((a, 0), (\sqcup, 0), \dots, (\sqcup, 0)); this is the correct content, since an unvisited cell of tape 11 still holds its input symbol and an unvisited cell of another tape is blank. Since all heads start on cell 00 and move by one cell at a time, the converted cells always form an interval containing 00, bordered by plain symbols, so NN recognizes the ends of the zone it must sweep. When MM halts, NN converts the tape back, keeping only track 11, moves the head to the position of head 11, and halts.

By induction on the number of steps, after the simulation of tt steps of MM, the tracks of NN contain the tapes of MM after tt steps, with the heads correctly marked, and the cells outside the converted zone hold the unvisited part of tape 11. Hence, if MM halts, the final tape and head position of NN are those of tape 11 of MM, so NN halts in a standard final configuration exactly when MM does, with the same output; and if MM does not halt, neither does NN.

3.5 Other variants

The same method — encode the richer structure on one tape, simulate step by step — shows that the following variants compute exactly the Turing-computable functions:

  • machines whose tape is infinite only to the right (fold the two-way tape on itself, using two tracks);
  • machines with a two-dimensional tape (store the finite non-blank region row by row);
  • machines with the alphabet {⊔,1}\{\sqcup, 1\} only (encode each symbol of a larger alphabet by a fixed-length block of ⊔\sqcup and 11);
  • non-deterministic machines, where δ\delta is a relation, and a function value is accepted if some computation produces it, provided the function is well defined (simulate all computations in breadth-first order, using a third tape).

Some care is needed with the input and output conventions in each case, but no new idea.

3.6 Exercises

Exercise 3.7 ★.

Design a 11-tape machine computing x↦x mod 2x \mapsto x \bmod 2, and prove it correct.

Solution

Erase the input from left to right, alternating between two states ee (even number of 11s erased so far) and oo (odd). Since ⌜x⌝\ulcorner x \urcorner has x+1x + 1 ones, at the first blank the state is oo if xx is even and ee if xx is odd. In state oo at the blank, write a single 11 (output 00); in state ee, write 11, move right, write 11, move left (output 11). Then halt with the head on the first 11. A formal proof proceeds by induction on the number of erased cells, as in Proposition 3.1.

Exercise 3.8 ★★.

Describe a 22-tape machine computing (x,y)↦x⋅y(x, y) \mapsto x \cdot y, and estimate its number of steps.

Solution

Use an auxiliary symbol #\# and a marker XX. First copy ⌜y⌝\ulcorner y \urcorner to tape 22 without its last 11, starting on cell 00, erasing it from tape 11, and replace the last 11 of ⌜x⌝\ulcorner x \urcorner (cell xx) by #\#. Now tape 11 holds 1x#1^x \# on cells 0,…,x0, \dots, x, tape 22 holds yy ones on cells 0,…,y−10, \dots, y - 1, and the output will be built on tape 11 from cell x+1x + 1 on. Each round starts with head 11 on the leftmost 11 of the first block and head 22 on cell 00: replace this 11 by XX, move head 11 right past #\# to the first blank after the output zone, then move both heads right together while head 22 reads 11, writing a 11 with head 11 at each step; finally return head 22 to cell 00, and head 11 left to the rightmost XX and one cell further right. When head 11 starts a round on #\#, all xx rounds are done and the output zone holds xyxy ones: append one more 11 (unary convention), go back to cell −1-1, and move right erasing the XXs and #\#, halting on cell x+1x + 1, the first cell of ⌜xy⌝\ulcorner xy \urcorner. The contents of tape 22 do not matter. The first phase takes O(x+y)O(x + y) steps, and each round O(x+y+xy)O(x + y + xy) steps, since head 11 crosses the output zone twice; the total is O(x2(y+1)+y)O(x^2(y + 1) + y) steps.

3.7 Notes and references

The simulation of multi-tape machines by single-tape machines, with a quadratic slowdown, is due to Hartmanis and Stearns, who used it in the foundational paper of complexity theory. The study of such simulations and of their cost belongs to complexity theory, where the robustness of the class of polynomial-time computable functions plays the role that the robustness of the computable functions plays here.

  • J. Hartmanis, R. E. Stearns, On the computational complexity of algorithms, Transactions of the American Mathematical Society 117 (1965), 285–306.
  • M. Sipser, Introduction to the Theory of Computation, 3rd edition, Cengage, 2012, chapter 3.