API

ModelingToolkitNeuralNets.NeuralNetworkBlock — Function

NeuralNetworkBlock(; n_input = 1, n_output = 1,
    chain = multi_layer_feed_forward(n_input, n_output),
    rng = Xoshiro(0),
    init_params = Lux.initialparameters(rng, chain),
    eltype = Float64,
    name)
NeuralNetworkBlock(n_input, n_output = 1; kwargs...)

Create a ModelingToolkit component system that evaluates a Lux neural network.

Arguments

n_input: Number of scalar inputs accepted by the network.
n_output: Number of scalar outputs produced by the network.

Keyword Arguments

chain: Lux model to call from the generated symbolic equations.
rng: Random number generator used to initialize parameters and query the Lux output size.
init_params: Initial Lux parameter container. It is flattened into the tunable parameter vector p.
eltype: Element type used for the stored ComponentArray parameter values.
name: Required ModelingToolkit component name.

Returns

A System with input variables inputs, output variables outputs, tunable network parameters p, and non-tunable parameters storing the Lux model and parameter-container type.

Examples

using Lux, ModelingToolkitBase, ModelingToolkitNeuralNets, Random

chain = multi_layer_feed_forward(2, 1; width = 8, depth = 2)
@named nn = NeuralNetworkBlock(2, 1; chain, rng = Xoshiro(0))

length(nn.inputs) == 2
length(nn.outputs) == 1

source

ModelingToolkitNeuralNets.SymbolicNeuralNetwork — Function

SymbolicNeuralNetwork(; n_input = 1, n_output = 1,
    chain = multi_layer_feed_forward(n_input, n_output),
    rng = Xoshiro(0),
    init_params = Lux.initialparameters(rng, chain),
    nn_name =  :NN,
    nn_p_name = :p,
    eltype = Float64)

Create a callable symbolic neural-network parameter and a symbolic parameter vector.

Keyword Arguments

n_input: Number of scalar entries expected in the network input vector.
n_output: Number of scalar entries returned by the network.
chain: Lux model represented by the returned callable parameter.
rng: Random number generator used to initialize Lux parameters.
init_params: Initial Lux parameter container. It is flattened into the returned symbolic parameter vector.
nn_name: Symbol used as the callable neural-network parameter name.
nn_p_name: Symbol used as the neural-network parameter-vector name.
eltype: Element type used for the stored ComponentArray parameter values.

Returns

A tuple (NN, p) where NN(input, p) is a symbolic callable parameter and p is a symbolic vector containing the flattened neural-network parameters.

Interface

NN(input, p) expects input to have length n_input and p to have the same length as the flattened init_params. The returned symbolic expression has length n_output. Use get_network on the default value of NN to recover the underlying Lux model.

Examples

using Lux, ModelingToolkitBase, ModelingToolkitNeuralNets, Random

chain = multi_layer_feed_forward(2, 2; width = 4)
NN, p = SymbolicNeuralNetwork(; chain, n_input = 2, n_output = 2, rng = Xoshiro(0))

get_network(ModelingToolkitBase.getdefault(NN)) === chain

The returned values can be used in equations as symbolic parameters:

using ModelingToolkitBase

@variables x(t_nounits)[1:2] y(t_nounits)[1:2]
eqs = [y ~ NN(x, p)]

source

ModelingToolkitNeuralNets.@SymbolicNeuralNetwork — Macro

@SymbolicNeuralNetwork
@SymbolicNeuralNetwork NN, p = chain
@SymbolicNeuralNetwork NN, p = chain rng

Construct a symbolic neural network while inferring names and dimensions from the assignment.

Arguments

NN: Left-hand-side name for the callable neural-network parameter.
p: Left-hand-side name for the flattened neural-network parameter vector.
chain: Lux chain whose first and last layers determine n_input and n_output.
rng: Optional random number generator passed to SymbolicNeuralNetwork.

Interface

The macro supports Lux chains whose first and last layers are Lux.Dense, because those layers expose the input and output dimensions needed for automatic size inference. For other layer types, call SymbolicNeuralNetwork directly with explicit n_input and n_output.

Examples

using Lux, ModelingToolkitNeuralNets, Random

chain = Lux.Chain(
    Lux.Dense(1 => 3, Lux.softplus; use_bias = false),
    Lux.Dense(3 => 1, Lux.softplus; use_bias = false),
)
rng = Xoshiro(0)
@SymbolicNeuralNetwork NN, p = chain rng

source

ModelingToolkitNeuralNets.multi_layer_feed_forward — Function

multi_layer_feed_forward(; n_input, n_output, width::Int = 4,
    depth::Int = 1, activation = tanh, use_bias = true, initial_scaling_factor = 1e-8)
multi_layer_feed_forward(n_input, n_output; kwargs...)

Create a fully connected Lux chain for symbolic neural-network models.

Arguments

n_input: Number of scalar inputs to the first dense layer.
n_output: Number of scalar outputs from the final dense layer.

Keyword Arguments

width: Number of hidden units in each hidden dense layer.
depth: Number of hidden dense layers after the first hidden layer.
activation: Activation function used by the hidden dense layers.
use_bias: Whether each dense layer includes a bias vector.
initial_scaling_factor: Multiplicative factor applied to the final layer's initial weights.

Returns

A Lux.Chain compatible with NeuralNetworkBlock, SymbolicNeuralNetwork, and @SymbolicNeuralNetwork.

Examples

using ModelingToolkitNeuralNets

chain = multi_layer_feed_forward(2, 1; width = 8, depth = 2, activation = tanh)

source

ModelingToolkitNeuralNets.get_network — Function

get_network(wrapper)

Return the Lux model stored by a symbolic neural-network callable.

Arguments

wrapper: A callable wrapper obtained from the default value of a neural-network parameter created by SymbolicNeuralNetwork.

Returns

The Lux model passed as the chain keyword when constructing the symbolic neural network.

Examples

using Lux, ModelingToolkitBase, ModelingToolkitNeuralNets

chain = Lux.Chain(
    Lux.Dense(1 => 3, Lux.softplus; use_bias = false),
    Lux.Dense(3 => 1, Lux.softplus; use_bias = false),
)
NN, p = SymbolicNeuralNetwork(; chain, n_input = 1, n_output = 1)

get_network(ModelingToolkitBase.getdefault(NN)) === chain

source

ModelingToolkitNeuralNets.isneuralnetwork — Function

ModelingToolkitNeuralNets.isneuralnetwork(p)

Return whether p is the symbolic callable for a neural network.

Arguments

p: Symbolic variable, parameter, symbolic array, or callable symbolic wrapper to inspect.

Returns

true when p has neural-network callable metadata and false otherwise.

Examples

using Lux, ModelingToolkitBase, ModelingToolkitNeuralNets

chain = multi_layer_feed_forward(1, 1)
@SymbolicNeuralNetwork NN, θ = chain

ModelingToolkitNeuralNets.isneuralnetwork(NN)
ModelingToolkitNeuralNets.isneuralnetwork(θ)

source

ModelingToolkitNeuralNets.isneuralnetworkps — Function

ModelingToolkitNeuralNets.isneuralnetworkps(p)

Return whether p is the symbolic parameter vector for a neural network.

Arguments

p: Symbolic variable, parameter, symbolic array, or callable symbolic wrapper to inspect.

Returns

true when p has neural-network parameter-vector metadata and false otherwise.

Examples

using Lux, ModelingToolkitBase, ModelingToolkitNeuralNets

chain = multi_layer_feed_forward(1, 1)
@SymbolicNeuralNetwork NN, θ = chain

ModelingToolkitNeuralNets.isneuralnetworkps(NN)
ModelingToolkitNeuralNets.isneuralnetworkps(θ)

source

ModelingToolkitNeuralNets.get_nn_chain — Function

ModelingToolkitNeuralNets.get_nn_chain(p)

Return the Lux chain associated with a symbolic neural-network callable.

Arguments

p: Symbolic callable parameter created by SymbolicNeuralNetwork or @SymbolicNeuralNetwork.

Returns

The Lux chain stored as the default value of p.

Throws

Throws an ErrorException when p is not a neural-network callable parameter.

Examples

using ModelingToolkitNeuralNets

chain = multi_layer_feed_forward(1, 1)
@SymbolicNeuralNetwork NN, θ = chain

ModelingToolkitNeuralNets.get_nn_chain(NN) === chain

source