|
Modern day society relies on the operation of complex systems including aircraft, automobiles,
electric power systems, economic entities, business organizations, banking
and finance systems, computer networks, manufacturing systems, and industrial processes,
Decision and control are responsible for ensuring that these systems perform
properly and meet prescribed performance objectives, The safe, reliable, and efficient
control of these systems is essential for our society, Therefore, automatic decision
and control systems are ubiquitous in human engineered systems and have had an
enormous impact on our lives. As modern systems become more complex and performance
requirements more stringent, improved methods of decision and control
are required that deliver guaranteed performance and the satisfaction of prescribed
goals.
Feedback control works on the principle of observing the actual outputs of a system,
comparing them to desired trajectories, and computing a control Signal based
on that error, which is used to modify the performance of the system to make the
actual output follow the desired trajectory. The optimization of sequential decisions
or controls that are repeated over time arises in many fields, including artificial intelligence,
automatic control systems, power systems, economics, medicine, operations
research, resource allocation, collaboration and coalitions, business and finance, and
games including chess and backgammon. Optimal control theory provides methods
for computing feedback control systems that deliver optimal performance. Optimal
controllers optimize user-prescribed performance functions and are normally
designed offline by solving Hamilton-Jacobi-Bellman (HJB) design equations. This
requires knowledge of the full system dynamics model. However, it is often difficult
to determine an accurate dynamical model of practical systems. Moreover, determining
optimal control policies for nonlinear systems requires the offline solution of
nonlinear HJB equations, which are often difficult or impossible to solve. Dynamic
programming (DP) is a sequential algorithmic method for finding optimal solutions in
sequential decision problems. DP was developed beginning in the 1960s with the work
of Bellman and Pontryagin. DP is fundamentally a backwards-in-time procedure that
does not offer methods for solving optimal decision problems in a forward manner in
real time. |