Advanced Kalman Filter Theory and Practice

This intensive course is designed for engineers and scientists who need to deepen their understanding of the mathematical foundations of Kalman filtering and learn new practical techniques to build and maintain real world Kalman filters.

Prerequisites:An undergraduate degree in engineering, natural science (physics, chemistry, etc.) or mathematics. An advanced degree in one of these fields will make it easier to grasp a few of the concepts presented, but nothing in the course presupposes such advanced work. Some practical acquaintance and/or experience with applications of Kalman filters in real world situation will certainly enrich the learning experience in this course but, again, are not required.

Course Style:  This is an intensive, highly interactive seminar.  The participants will not be mere passive spectators at a series of lectures.  Class participants will have ample opportunity to exercise what they learn.  The course is divided up into segments with each segment lasting approximately 2 hours.  There are 4 segments each day.  Each segment is 2 hours long and is structured as follows:

q       1-hour interactive presentation of new material

q       30-minute exercise session

q       15-minute exercise review period

q       15-minute break 

  The 30-minute exercise session usually consists of two parts: an individual part (15 minutes) and a workgroup part (15 minutes). The student will have the opportuntity to work both alone and as part of a workgroup. The workgroups are arranged so that the level of skill and experience is balanced across workgroups.

 Sample content from Day 1, Segment 1, including the exercise session at the end, are provided here in a PDF file: Set Theory (PDF). (Note this segment is only about the half the length of other segments, because it is preceded by an introductory session on Day 1.)

Most of the segments also have Matlab m-files and/or Simulink models associated with them. For example, the segment on linear systems theory uses the following pair of files (among others): State Space Generation m-File and Generic State Space Model.

Course Content:

Stochastic dynamic systems (both linear and non-linear) play a central role in Kalman filter theory. Thus the course will provide a solid grounding in the mathematical concepts and techniques underlying such systems. Also, a brief introduction to Hilbert Space theory will be provided, since optimal estimation techniques such as the Kalman filter can be briefly characterized a orthogonal projection in Hilbert space. Many fields of modern science and engineering employ Hilbert Space concepts and techniques; for example, in engineering they play a key role in digital signal processing and in physics they are used in quantum mechanics. Hilbert Space is not as scary as it sounds. It is really only a generalization of 3-dimensional Euclidean space to an infinite number of dimensions. The concepts of inner product (aka dot product), orthogonality, norm (aka vector magnitude), and linear transformations are already familiar from 3-space. In addition to these more mathematical topics, useful practical techniques for identifying appropriate state models based on empirical input-output data will be taught. Hands-on problem solving using these techniques will help class participants acquire solid engineering tools.

Day 1:  Foundational Math I: 

q       Segment 1 – Basic Set Theory: sets, set operations, Cartesian products, functions, n-ary operations, relations, equivalence relations. Practical applications: probability event algebras.

q       Segment 2 – Algebra:  algebraic structures, homomorphisms, groups, rings, fields, vector spaces, bases, inner products, norms, matrices

q       Segment 3 – Topological spaces, open / closed / compact sets, continuous functions; s-algebras, measures, probability measures; Borel sets; integration

q       Segment 4 – Infinite dimensional vector spaces; Banach space; Hilbert space; orthogonal projection, least squares approximation, recursive formulations

Day 2:  Foundational Math II:

q       Segment 1 – Probability theory, random variables, distributions, density functions, expectation, moments, conditional expectation

q       Segment 2 – Stochastic processes, second-order processes, Gaussian processes, Wiener-Levy processes, white noise, random walk, random constants, random ramps; Allan variance

q       Segment 3 – Correlation, autocorrelation, cross-correlation, power spectral density. Practical applicatons: system identification techniques (SI/SO -> MI/SO)

q       Segment 4 – Mean square calculus:  convergence, derivatives, integration, stochastic differential equations

Day 3:  Dynamic Systems (deterministic):

q       Segment 1 – Ordinary differential operators, differential equations, dynamic systems, state space representation

q       Segment 2 – Distributions: linear functionals, dual space, delta functional, Heaviside operator

q       Segment 3 – Transforms:  Laplace, Fourier, Hilbert, Mellin

q       Segment 4 – Solutions to differential equations:  state transition matrix; transient / steady-state response; frequency domain analysis: stability criteria

Day 4:  Dynamic Systems (stochastic) and Kalman Filtering

q       Segment 1 – Stochastic differential equations, stochastic dynamic systems

q       Segment 2 – Error and Uncertainty:  The error covariance matrix and covariance analysis; matrix Riccatti equation

q       Segment 3 – Measurement model; observability criteria;

q       Segment 4 – Continuous formulation of the filtering problem and derivation of solutions: static and dynamic estimation problems; alternative derivations

Day 5:  Advanced Topics

q       Segment 1 – System identification: state vector augmentation; model identification and evaluation; innovations sequence monitoring.

q       Segment 2 – Discrete formulation:  difference equations

q       Segment 3 – Nonlinear problems:  Linearization techniques and the extended Kalman filter

q       Segment 4 – Adaptive Kalman filtering