ADTypes.jl

ADTypes.jl

ADTypes.jl is a multi-valued logic system to choose an automatic differentiation (AD) package and specify its parameters.

AbstractADType

Abstract supertype for all AD choices.

Extension contract

External packages may subtype AbstractADType to describe an AD backend. They must also implement mode for their concrete subtype and return an instance of an AbstractMode subtype. Consumers should dispatch on mode(ad), rather than on a downstream concrete AD type.

AutoForwardDiff{chunksize,T}

Struct used to select the ForwardDiff.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoForwardDiff(; chunksize=nothing, tag=nothing)

Type parameters

• chunksize: the preferred chunk size to evaluate several derivatives at once

Fields

• tag::T: a custom tag to handle nested differentiation calls (usually not necessary)

AutoPolyesterForwardDiff{chunksize,T}

Struct used to select the PolyesterForwardDiff.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoPolyesterForwardDiff(; chunksize=nothing, tag=nothing)

Type parameters

• chunksize: the preferred chunk size to evaluate several derivatives at once

Fields

• tag::T: a custom tag to handle nested differentiation calls (usually not necessary)

AutoMooncakeForward

Struct used to select the Mooncake.jl backend for automatic differentiation in forward mode.

Defined by ADTypes.jl.

Constructors

AutoMooncakeForward(; config=nothing)

Fields

• config: either nothing or an instance of Mooncake.Config – see the docstring of Mooncake.Config for more information. AutoMooncakeForward(; config=nothing) is equivalent to AutoMooncakeForward(; config=Mooncake.Config()), i.e. the default configuration.

AutoFiniteDiff{T1,T2,T3}

Struct used to select the FiniteDiff.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoFiniteDiff(;
    fdtype=Val(:forward), fdjtype=fdtype, fdhtype=Val(:hcentral),
    relstep=nothing, absstep=nothing, dir=true
)

Fields

• fdtype::T1: finite difference type
• fdjtype::T2: finite difference type for the Jacobian
• fdhtype::T3: finite difference type for the Hessian
• relstep: relative finite difference step size
• absstep: absolute finite difference step size
• dir: direction of the finite difference step

AutoFiniteDifferences{T}

Struct used to select the FiniteDifferences.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoFiniteDifferences(; fdm)

Fields

• fdm::T: a FiniteDifferenceMethod

AutoGTPSA{D}

Struct used to select the GTPSA.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoGTPSA(; descriptor=nothing)

Fields

• descriptor::D: can be either

  • a GTPSA Descriptor specifying the number of variables/parameters, parameter order, individual variable/parameter truncation orders, and maximum order. See the GTPSA.jl documentation for more details.
  • nothing to automatically use a Descriptor given the context.

AutoTaylorDiff{order}

Struct used to select the TaylorDiff.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoTaylorDiff(; order = 1)

Type parameters

• order: the order of the Taylor-mode automatic differentiation

AutoHyperHessians{chunksize}

Struct used to select the HyperHessians.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoHyperHessians(; chunksize=nothing)

Type parameters

• chunksize: the preferred chunk size to evaluate several derivatives at once. If nothing, HyperHessians chooses automatically.

AutoMooncake

Struct used to select the Mooncake.jl backend for automatic differentiation in reverse mode.

Defined by ADTypes.jl.

Constructors

AutoMooncake(; config=nothing)

Fields

• config: either nothing or an instance of Mooncake.Config – see the docstring of Mooncake.Config for more information. AutoMooncake(; config=nothing) is equivalent to AutoMooncake(; config=Mooncake.Config()), i.e. the default configuration.

AutoReverseDiff{compile}

Struct used to select the ReverseDiff.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoReverseDiff(; compile::Union{Val, Bool} = Val(false))

Fields

• compile::Union{Val, Bool}: whether to allow pre-recording and reusing a tape (which speeds up the differentiation process).

  • If compile=false or compile=Val(false), a new tape must be recorded at every call to the differentiation operator.
  • If compile=true or compile=Val(true), a tape can be pre-recorded on an example input and then reused at every differentiation call.

  The boolean version of this keyword argument is taken as the type parameter.

AutoTracker

Struct used to select the Tracker.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoTracker()

AutoZygote

Struct used to select the Zygote.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoZygote()

AutoEnzyme{M,A}

Struct used to select the Enzyme.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoEnzyme(; mode::M=nothing, function_annotation::Type{A}=Nothing)

Type parameters

• A determines how the function f to differentiate is passed to Enzyme. It can be:

  • a subtype of EnzymeCore.Annotation (like EnzymeCore.Const or EnzymeCore.Duplicated) to enforce a given annotation
  • Nothing to simply pass f and let Enzyme choose the most appropriate annotation

Fields

• mode::M determines the autodiff mode (forward or reverse). It can be:

  • an object subtyping EnzymeCore.Mode (like EnzymeCore.Forward or EnzymeCore.Reverse) if a specific mode is required
  • nothing to choose the best mode automatically

AutoChainRules{RC}

Struct used to select an automatic differentiation backend based on ChainRulesCore.jl (see the list here).

Defined by ADTypes.jl.

Constructors

AutoChainRules(; ruleconfig)

Fields

• ruleconfig::RC: a ChainRulesCore.RuleConfig object.

AutoDiffractor

Struct used to select the Diffractor.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoDiffractor()

AutoReactant{M<:AutoEnzyme}

Struct used to select the Reactant.jl compilation atop Enzyme for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoReactant(; mode::Union{AutoEnzyme,Nothing}=nothing)

Fields

• mode::M specifies the parameterization of differentiation. It can be:

  • an AutoEnzyme object if a specific mode is required
  • nothing to choose the best mode automatically

AutoFastDifferentiation

Struct used to select the FastDifferentiation.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoFastDifferentiation()

AutoSymbolics

Struct used to select the Symbolics.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoSymbolics()

AutoSparse{D,S,C}

Wraps an ADTypes.jl object to deal with sparse Jacobians and Hessians.

Fields

• dense_ad::D: the underlying AD package, subtyping AbstractADType
• sparsity_detector::S: the sparsity pattern detector, subtyping AbstractSparsityDetector
• coloring_algorithm::C: the coloring algorithm, subtyping AbstractColoringAlgorithm

Constructors

AutoSparse(
    dense_ad;
    sparsity_detector=ADTypes.NoSparsityDetector(),
    coloring_algorithm=ADTypes.NoColoringAlgorithm()
)

dense_ad(ad::AutoSparse)::AbstractADType
dense_ad(ad::AbstractADType)::AbstractADType

Return the underlying AD package for a sparse AD choice, act as the identity on a dense AD choice.

See also

• AutoSparse

sparsity_detector(ad::AutoSparse)::AbstractSparsityDetector

Return the sparsity pattern detector for a sparse AD choice.

See also

• AutoSparse
• AbstractSparsityDetector

AbstractSparsityDetector

Abstract supertype for sparsity pattern detectors.

Extension contract

External detectors implement the supported public forms of jacobian_sparsity and/or hessian_sparsity. A Jacobian pattern must be an AbstractMatrix{Bool} with shape (length(y), length(x)), where y is f(x) for the out-of-place form or the supplied output buffer for the in-place form. A Hessian pattern must be an AbstractMatrix{Bool} with shape (length(x), length(x)). Methods for unsupported operations must throw an error; they must not return an unrelated pattern.

New detectors should return Bool patterns, but consumers must not require Bool: they should treat every nonzero entry as a structural nonzero. Patterns given as integer or floating point matrices of ones and zeroes are common in the wild, are accepted unchanged by KnownJacobianSparsityDetector and KnownHessianSparsityDetector, and are handled correctly by the coloring and decompression implementations used downstream (SparseMatrixColorings.jl through DifferentiationInterface.jl). The element type of the pattern does not propagate to the differentiation result.

For a sparse matrix, the stored structure defines the pattern: an explicitly stored zero is treated as a structural nonzero, which yields a valid but more conservative coloring than the same matrix passed through dropzeros.

jacobian_sparsity(f, x, sd::AbstractSparsityDetector)::AbstractMatrix{Bool}
jacobian_sparsity(f!, y, x, sd::AbstractSparsityDetector)::AbstractMatrix{Bool}

Use detector sd to construct a (typically sparse) matrix S describing the pattern of nonzeroes in the Jacobian of f (resp. f!) applied at x (resp. (y, x)).

hessian_sparsity(f, x, sd::AbstractSparsityDetector)::AbstractMatrix{Bool}

Use detector sd to construct a (typically sparse) matrix S describing the pattern of nonzeroes in the Hessian of f applied at x.

NoSparsityDetector <: AbstractSparsityDetector

Trivial sparsity detector, which always returns a full sparsity pattern (only ones, no zeroes).

See also

• AbstractSparsityDetector

KnownJacobianSparsityDetector(jacobian_sparsity::AbstractMatrix) <: AbstractSparsityDetector

Trivial sparsity detector used to return a known Jacobian sparsity pattern.

AbstractMatrix{Bool} is the canonical pattern type, but the element type is neither converted nor checked: jacobian_sparsity hands the pattern back exactly as given, and consumers treat every nonzero entry as a structural nonzero, so integer or floating point matrices of ones and zeroes work as well (see the extension contract of AbstractSparsityDetector).

See also

• AbstractSparsityDetector
• KnownHessianSparsityDetector

KnownHessianSparsityDetector(hessian_sparsity::AbstractMatrix) <: AbstractSparsityDetector

Trivial sparsity detector used to return a known Hessian sparsity pattern.

AbstractMatrix{Bool} is the canonical pattern type, but the element type is neither converted nor checked: hessian_sparsity hands the pattern back exactly as given, and consumers treat every nonzero entry as a structural nonzero, so integer or floating point matrices of ones and zeroes work as well (see the extension contract of AbstractSparsityDetector).

See also

• AbstractSparsityDetector
• KnownJacobianSparsityDetector

coloring_algorithm(ad::AutoSparse)::AbstractColoringAlgorithm

Return the coloring algorithm for a sparse AD choice.

See also

• AutoSparse
• AbstractColoringAlgorithm

AbstractColoringAlgorithm

Abstract supertype for Jacobian/Hessian coloring algorithms.

Extension contract

External algorithms implement the supported public coloring functions. Each result must be an AbstractVector of integers: column colorings have length size(M, 2), row colorings have length size(M, 1), and symmetric colorings require a square matrix and have length size(M, 1). The assigned colors must satisfy the structural orthogonality condition documented by each coloring function. Unsupported coloring forms must throw an error.

Note

The terminology and definitions are taken from the following paper:

What Color Is Your Jacobian? Graph Coloring for Computing Derivatives, Assefaw Hadish Gebremedhin, Fredrik Manne, and Alex Pothen (2005)

column_coloring(M::AbstractMatrix, ca::AbstractColoringAlgorithm)::AbstractVector{<:Integer}

Use algorithm ca to construct a structurally orthogonal partition of the columns of M.

The result is a coloring vector c of length size(M, 2) such that for every non-zero coefficient M[i, j], column j is the only column of its color c[j] with a non-zero coefficient in row i.

row_coloring(M::AbstractMatrix, ca::AbstractColoringAlgorithm)::AbstractVector{<:Integer}

Use algorithm ca to construct a structurally orthogonal partition of the rows of M.

The result is a coloring vector c of length size(M, 1) such that for every non-zero coefficient M[i, j], row i is the only row of its color c[i] with a non-zero coefficient in column j.

symmetric_coloring(M::AbstractMatrix, ca::AbstractColoringAlgorithm)::AbstractVector{<:Integer}

Use algorithm ca to construct a symmetrically structurally orthogonal partition of the columns (or rows) of the symmetric matrix M.

The result is a coloring vector c of length size(M, 1) == size(M, 2) such that for every non-zero coefficient M[i, j], at least one of the following conditions holds:

• column j is the only column of its color c[j] with a non-zero coefficient in row i;
• column i is the only column of its color c[i] with a non-zero coefficient in row j.

NoColoringAlgorithm <: AbstractColoringAlgorithm

Trivial coloring algorithm, which always returns a different color for each matrix column/row.

See also

• AbstractColoringAlgorithm

NoAutoDiff

Struct used to select no automatic differentiation.

Defined by ADTypes.jl.

Constructors

NoAutoDiff()

NoAutoDiffSelectedError <: Exception

Signifies that code tried to use automatic differentiation, but NoAutoDiff was specified.

Constructors

NoAutoDiffSelectedError(msg::String)

mode(ad::AbstractADType)

Return the differentiation mode of ad, as a subtype of AbstractMode.

Extension contract

Every external concrete subtype of AbstractADType must provide this method. Return an instance of the most specific applicable mode trait; callers may use that trait for dispatch but must not require a particular backend implementation.

AbstractMode

Abstract supertype for the traits identifying differentiation modes.

Subtypes

• ForwardMode
• ReverseMode
• ForwardOrReverseMode
• SymbolicMode

ForwardMode

Trait for AD choices that rely on forward mode algorithmic differentiation or finite differences.

These two paradigms are classified together because they can both efficiently compute Jacobian-vector products.

ForwardOrReverseMode

Trait for AD choices that can work either in ForwardMode or ReverseMode, depending on their configuration.

ReverseMode

Trait for AD choices that rely on reverse mode algorithmic differentiation.

SymbolicMode

Trait for AD choices that rely on symbolic differentiation.

ADTypes.Auto(package::Symbol)
ADTypes.Auto(nothing)::NoAutoDiff

A shortcut that converts an AD package name into an instance of AbstractADType, with all parameters set to their default values.

Example

import ADTypes
backend = ADTypes.Auto(:Zygote)

# output

ADTypes.AutoZygote()

AutoModelingToolkit(obj_sparse::Bool, cons_sparse::Bool)
AutoModelingToolkit(; obj_sparse::Bool=false, cons_sparse::Bool=false)

Deprecated symbolic automatic differentiation selector. It returns AutoSparse(AutoSymbolics()) when either sparsity flag is true, and AutoSymbolics otherwise.

Defined by ADTypes.jl.

AutoSparseFastDifferentiation()

Deprecated constructor returning AutoSparse(AutoFastDifferentiation()).

Defined by ADTypes.jl.

AutoSparseFiniteDiff(; kwargs...)

Deprecated constructor returning AutoSparse(AutoFiniteDiff(; kwargs...)). All keyword arguments are forwarded to AutoFiniteDiff.

Defined by ADTypes.jl.

AutoSparseForwardDiff(; chunksize=nothing, tag=nothing)

Deprecated constructor returning AutoSparse(AutoForwardDiff(; kwargs...)). All keyword arguments are forwarded to AutoForwardDiff.

Defined by ADTypes.jl.

AutoSparsePolyesterForwardDiff(; chunksize=nothing, tag=nothing)

Deprecated constructor returning AutoSparse(AutoPolyesterForwardDiff(; kwargs...)). All keyword arguments are forwarded to AutoPolyesterForwardDiff.

Defined by ADTypes.jl.

AutoSparseReverseDiff(; compile::Union{Val, Bool}=Val(false))
AutoSparseReverseDiff(compile)

Deprecated constructor returning AutoSparse(AutoReverseDiff(; compile)). The compile setting is forwarded to AutoReverseDiff.

Defined by ADTypes.jl.

AutoSparseZygote()

Deprecated constructor returning AutoSparse(AutoZygote()).

Defined by ADTypes.jl.

AutoTapir

Struct used to select the Tapir.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoTapir(; safe_mode=true)

Fields

• safe_mode::Bool: whether to run additional checks to catch errors early.