The maximum principle, Bellman's equation, and Carathéodory's work

One of the most important and deep results in optimal control theory is the maximum principle attributed to Hestenes (1950) and in particular to Boltyanskii, Gamkrelidze, and Pontryagin (1956). Another prominent result is known as the Bellman equation, which is associated with Isaacs' and Bellman's work (later than 1951). However, precursors of both the maximum principle and the Bellman equation can already be found in Carathéodory's book of 1935 (Ref. 1a), the first even in his earlier work of 1926 which is given in Ref. 2. This is not a widely acknowledged fact. The present tutorial paper traces Carathéodory's approach to the calculus of variations, once called the "royal road in the calculus of variations," and shows that famous results in optimal control theory, including the maximum principle and the Bellman equation, are consequences of Carathéodory's earlier results.