This book proposes, for the first time, a basic formulation for structural control that takes into account the stochastic dynamics induced by engineering excitations in the nature of non-stationary and non-Gaussian processes. Further, it establishes the theory of and methods for stochastic optimal control of randomly-excited engineering structures in the context of probability density evolution methods, such as physically-based stochastic optimal (PSO) control. By logically integrating randomness into control gain, the book helps readers design elegant control systems, mitigate risks in civil engineering structures, and avoid the dilemmas posed by the methods predominantly applied in current practice, such as deterministic control and classical linear quadratic Gaussian (LQG) control associated with nominal white noises.