Structuring a Neural Lyapunov function

Three simple neural Lyapunov function structures are provided, but users can always specify a custom structure using the NeuralLyapunovStructure struct.

Pre-defined structures

The simplest structure is to train the neural network directly to be the Lyapunov function, which can be accomplished using an NoAdditionalStructure.

NeuralLyapunov.NoAdditionalStructure — Function

NoAdditionalStructure()

Create a NeuralLyapunovStructure where the Lyapunov function is the neural network evaluated at the state. This does impose any additional structure to enforce any Lyapunov conditions.

Corresponds to $V(x) = ϕ(x)$, where $ϕ$ is the neural network.

Dynamics are assumed to be in f(state, p, t) form, as in an ODEFunction. For f(state, input, p, t), consider using add_policy_search.

source

The condition that the Lyapunov function $V(x)$ must be minimized uniquely at the fixed point $x_0$ is often represented as a requirement for $V(x)$ to be positive away from the fixed point and zero at the fixed point. Put mathematically, it is sufficient to require $V(x) > 0 \, \forall x \ne x_0$ and $V(x_0) = 0$. We call such functions positive definite (around the fixed point $x_0$).

Two structures are provided which partially or fully enforce the minimization condition: NonnegativeStructure, which structurally enforces $V(x) \ge 0$ everywhere, and PositiveSemiDefiniteStructure, which additionally enforces $V(x_0) = 0$.

NeuralLyapunov.NonnegativeStructure — Function

NonnegativeStructure(network_dim; <keyword_arguments>)

Create a NeuralLyapunovStructure where the Lyapunov function is the L2 norm of the neural network output plus a constant δ times a function pos_def.

Corresponds to $V(x) = \lVert ϕ(x) \rVert^2 + δ \, \texttt{pos\_def}(x, x_0)$, where $ϕ$ is the neural network and $x_0$ is the equilibrium point.

This structure ensures $V(x) ≥ 0 \, ∀ x$ when $δ ≥ 0$ and pos_def is always nonnegative. Further, if $δ > 0$ and pos_def is strictly positive definite around fixed_point, the structure ensures that $V(x)$ is strictly positive away from fixed_point. In such cases, the minimization condition reduces to ensuring $V(x_0) = 0$, and so DontCheckNonnegativity(true) should be used.

Arguments

network_dim: output dimensionality of the neural network.

Keyword Arguments

δ: weight of pos_def, as above; defaults to 0.
pos_def(state, fixed_point): a function that is positive (semi-)definite in state around fixed_point; defaults to $\log(1 + \lVert x - x_0 \rVert^2)$.
grad_pos_def(state, fixed_point): the gradient of pos_def with respect to state at state. If isnothing(grad_pos_def) (as is the default), the gradient of pos_def will be evaluated using grad.
grad: a function for evaluating gradients to be used when isnothing(grad_pos_def); defaults to, and expects the same arguments as, ForwardDiff.gradient.

Dynamics are assumed to be in f(state, p, t) form, as in an ODEFunction. For f(state, input, p, t), consider using add_policy_search.

See also: DontCheckNonnegativity

source

NeuralLyapunov.PositiveSemiDefiniteStructure — Function

PositiveSemiDefiniteStructure(network_dim; <keyword_arguments>)

Create a NeuralLyapunovStructure where the Lyapunov function is the product of a positive (semi-)definite function pos_def which does not depend on the network and a nonnegative function non_neg which does depend the network.

Corresponds to $V(x) = \texttt{pos\_def}(x, x_0) * \texttt{non\_neg}(ϕ, x, x_0)$, where $ϕ$ is the neural network and $x_0$ is the equilibrium point.

This structure ensures $V(x) ≥ 0$. Further, if pos_def is strictly positive definite fixed_point and non_neg is strictly positive (as is the case for the default values of pos_def and non_neg), then this structure ensures $V(x)$ is strictly positive definite around fixed_point. In such cases, the minimization condition is satisfied structurally, so DontCheckNonnegativity(false) should be used.

Arguments

network_dim: output dimensionality of the neural network.

Keyword Arguments

pos_def(state, fixed_point): a function that is positive (semi-)definite in state around fixed_point; defaults to $\log(1 + \lVert x - x_0 \rVert^2)$.
non_neg(net, state, fixed_point): a nonnegative function of the neural network; note that net is the neural network $ϕ$, and net(state) is the value of the neural network at a point $ϕ(x)$; defaults to $1 + \lVert ϕ(x) \rVert^2$.
grad_pos_def(state, fixed_point): the gradient of pos_def with respect to state at state. If isnothing(grad_pos_def) (as is the default), the gradient of pos_def will be evaluated using grad.
grad_non_neg(net, J_net, state, fixed_point): the gradient of non_neg with respect to state at state; J_net is a function outputting the Jacobian of net at the input. If isnothing(grad_non_neg) (as is the default), the gradient of non_neg will be evaluated using grad.
grad: a function for evaluating gradients to be used when isnothing(grad_pos_def) || isnothing(grad_non_neg); defaults to, and expects the same arguments as, ForwardDiff.gradient.

Dynamics are assumed to be in f(state, p, t) form, as in an ODEFunction. For f(state, input, p, t), consider using add_policy_search.

Defining your own neural Lyapunov function structure

To define a new structure for a neural Lyapunov function, one must specify the form of the Lyapunov candidate $V$ and its time derivative along a trajectory $\dot{V}$, as well as how to call the dynamics $f$. Additionally, the dimensionality of the output of the neural network must be known in advance.

NeuralLyapunov.NeuralLyapunovStructure — Type

NeuralLyapunovStructure(V, V̇, f_call, network_dim)

Specifies the structure of the neural Lyapunov function and its derivative.

Allows the user to define the Lyapunov in terms of the neural network, potentially structurally enforcing some Lyapunov conditions.

Fields

V(phi::Function, state, fixed_point): outputs the value of the Lyapunov function at state.
V̇(phi::Function, J_phi::Function, dynamics::Function, state, params, t, fixed_point): outputs the time derivative of the Lyapunov function at state.
f_call(dynamics::Function, phi::Function, state, params, t): outputs the derivative of the state; this is useful for making closed-loop dynamics which depend on the neural network, such as in the policy search case.
network_dim: the dimension of the output of the neural network.

phi and J_phi above are both functions of state alone.

source

Calling the dynamics

Very generally, the dynamical system can be a system of ODEs $\dot{x} = f(x, u, p, t)$, where $u$ is a control input, $p$ contains parameters, and $f$ depends on the neural network in some way. To capture this variety, users must supply the function f_call(dynamics::Function, phi::Function, state, p, t).

The most common example is when dynamics takes the same form as an ODEFunction. i.e., $\dot{x} = \texttt{dynamics}(x, p, t)$. In that case, f_call(dynamics, phi, state, p, t) = dynamics(state, p, t).

Suppose instead, the dynamics were supplied as a function of state alone: $\dot{x} = \texttt{dynamics}(x)$. Then, f_call(dynamics, phi, state, p, t) = dynamics(state).

Finally, suppose $\dot{x} = \texttt{dynamics}(x, u, p, t)$ has a unidimensional control input that is being trained (via policy search) to be the second output of the neural network. Then f_call(dynamics, phi, state, p, t) = dynamics(state, phi(state)[2], p, t).

Note that, despite the inclusion of the time variable $t$, NeuralLyapunov.jl currently only supports time-invariant systems, so only t = 0.0 is used.

Structuring $V$ and $\dot{V}$

The Lyapunov candidate function $V$ gets specified as a function V(phi, state, fixed_point), where phi is the neural network as a function phi(state). Note that this form allows $V(x)$ to depend on the neural network evaluated at points other than just the input $x$.

The time derivative $\dot{V}$ is similarly defined by a function V̇(phi, J_phi, dynamics, state, params, t, fixed_point). The function J_phi(state) gives the Jacobian of the neural network phi at state. The function dynamics is as above (with parameters params).