ADTypes.jl
Documentation for ADTypes.jl.
ADTypes.ADTypes
— ModuleADTypes.jl
ADTypes.jl is a multi-valued logic system to choose an automatic differentiation (AD) package and specify its parameters.
ADTypes.AbstractADType
— TypeAbstractADType
Abstract supertype for all AD choices.
Dense AD
Forward mode
Algorithmic differentiation:
ADTypes.AutoForwardDiff
— TypeAutoForwardDiff{chunksize,T}
Struct used to select the ForwardDiff.jl backend for automatic differentiation.
Defined by ADTypes.jl.
Constructors
AutoForwardDiff(; chunksize=nothing, tag=nothing)
Type parameters
chunksize
: the preferred chunk size to evaluate several derivatives at once
Fields
tag::T
: a custom tag to handle nested differentiation calls (usually not necessary)
ADTypes.AutoPolyesterForwardDiff
— TypeAutoPolyesterForwardDiff{chunksize,T}
Struct used to select the PolyesterForwardDiff.jl backend for automatic differentiation.
Defined by ADTypes.jl.
Constructors
AutoPolyesterForwardDiff(; chunksize=nothing, tag=nothing)
Type parameters
chunksize
: the preferred chunk size to evaluate several derivatives at once
Fields
tag::T
: a custom tag to handle nested differentiation calls (usually not necessary)
Finite differences:
ADTypes.AutoFiniteDiff
— TypeAutoFiniteDiff{T1,T2,T3}
Struct used to select the FiniteDiff.jl backend for automatic differentiation.
Defined by ADTypes.jl.
Constructors
AutoFiniteDiff(; fdtype=Val(:forward), fdjtype=fdtype, fdhtype=Val(:hcentral))
Fields
fdtype::T1
: finite difference typefdjtype::T2
: finite difference type for the Jacobianfdhtype::T3
: finite difference type for the Hessian
ADTypes.AutoFiniteDifferences
— TypeAutoFiniteDifferences{T}
Struct used to select the FiniteDifferences.jl backend for automatic differentiation.
Defined by ADTypes.jl.
Constructors
AutoFiniteDifferences(; fdm)
Fields
fdm::T
: aFiniteDifferenceMethod
Taylor mode:
ADTypes.AutoGTPSA
— TypeAutoGTPSA{D}
Struct used to select the GTPSA.jl backend for automatic differentiation.
Defined by ADTypes.jl.
Constructors
AutoGTPSA(; descriptor=nothing)
Fields
descriptor::D
: can be either- a GTPSA
Descriptor
specifying the number of variables/parameters, parameter order, individual variable/parameter truncation orders, and maximum order. See the GTPSA.jl documentation for more details. nothing
to automatically use aDescriptor
given the context.
- a GTPSA
Reverse mode
ADTypes.AutoMooncake
— TypeAutoMooncake
Struct used to select the Mooncake.jl backend for automatic differentiation.
Defined by ADTypes.jl.
Constructors
AutoMooncake(; config)
Fields
config
: eithernothing
or an instance ofMooncake.Config
– see the docstring ofMooncake.Config
for more information.AutoMooncake(; config=nothing)
is equivalent toAutoMooncake(; config=Mooncake.Config())
, i.e. the default configuration.
ADTypes.AutoReverseDiff
— TypeAutoReverseDiff{compile}
Struct used to select the ReverseDiff.jl backend for automatic differentiation.
Defined by ADTypes.jl.
Constructors
AutoReverseDiff(; compile::Union{Val, Bool} = Val(false))
Fields
compile::Union{Val, Bool}
: whether to compile the tape prior to differentiation (the boolean version is also the type parameter)
ADTypes.AutoTracker
— TypeAutoTracker
Struct used to select the Tracker.jl backend for automatic differentiation.
Defined by ADTypes.jl.
Constructors
AutoTracker()
ADTypes.AutoZygote
— TypeAutoZygote
Struct used to select the Zygote.jl backend for automatic differentiation.
Defined by ADTypes.jl.
Constructors
AutoZygote()
Forward or reverse mode
ADTypes.AutoEnzyme
— TypeAutoEnzyme{M,A}
Struct used to select the Enzyme.jl backend for automatic differentiation.
Defined by ADTypes.jl.
Constructors
AutoEnzyme(; mode::M=nothing, function_annotation::Type{A}=Nothing)
Type parameters
A
determines how the functionf
to differentiate is passed to Enzyme. It can be:- a subtype of
EnzymeCore.Annotation
(likeEnzymeCore.Const
orEnzymeCore.Duplicated
) to enforce a given annotation Nothing
to simply passf
and let Enzyme choose the most appropriate annotation
- a subtype of
Fields
mode::M
determines the autodiff mode (forward or reverse). It can be:- an object subtyping
EnzymeCore.Mode
(likeEnzymeCore.Forward
orEnzymeCore.Reverse
) if a specific mode is required nothing
to choose the best mode automatically
- an object subtyping
ADTypes.AutoChainRules
— TypeAutoChainRules{RC}
Struct used to select an automatic differentiation backend based on ChainRulesCore.jl (see the list here).
Defined by ADTypes.jl.
Constructors
AutoChainRules(; ruleconfig)
Fields
ruleconfig::RC
: aChainRulesCore.RuleConfig
object.
ADTypes.AutoDiffractor
— TypeAutoDiffractor
Struct used to select the Diffractor.jl backend for automatic differentiation.
Defined by ADTypes.jl.
Constructors
AutoDiffractor()
Symbolic mode
ADTypes.AutoFastDifferentiation
— TypeAutoFastDifferentiation
Struct used to select the FastDifferentiation.jl backend for automatic differentiation.
Defined by ADTypes.jl.
Constructors
AutoFastDifferentiation()
ADTypes.AutoSymbolics
— TypeAutoSymbolics
Struct used to select the Symbolics.jl backend for automatic differentiation.
Defined by ADTypes.jl.
Constructors
AutoSymbolics()
Sparse AD
ADTypes.AutoSparse
— TypeAutoSparse{D,S,C}
Wraps an ADTypes.jl object to deal with sparse Jacobians and Hessians.
Fields
dense_ad::D
: the underlying AD package, subtypingAbstractADType
sparsity_detector::S
: the sparsity pattern detector, subtypingAbstractSparsityDetector
coloring_algorithm::C
: the coloring algorithm, subtypingAbstractColoringAlgorithm
Constructors
AutoSparse(
dense_ad;
sparsity_detector=ADTypes.NoSparsityDetector(),
coloring_algorithm=ADTypes.NoColoringAlgorithm()
)
ADTypes.dense_ad
— Functiondense_ad(ad::AutoSparse)::AbstractADType
Return the underlying AD package for a sparse AD choice.
See also
Sparsity detector
ADTypes.sparsity_detector
— Functionsparsity_detector(ad::AutoSparse)::AbstractSparsityDetector
Return the sparsity pattern detector for a sparse AD choice.
See also
ADTypes.AbstractSparsityDetector
— TypeAbstractSparsityDetector
Abstract supertype for sparsity pattern detectors.
Required methods
ADTypes.jacobian_sparsity
— Functionjacobian_sparsity(f, x, sd::AbstractSparsityDetector)::AbstractMatrix{Bool}
jacobian_sparsity(f!, y, x, sd::AbstractSparsityDetector)::AbstractMatrix{Bool}
Use detector sd
to construct a (typically sparse) matrix S
describing the pattern of nonzeroes in the Jacobian of f
(resp. f!
) applied at x
(resp. (y, x)
).
ADTypes.hessian_sparsity
— Functionhessian_sparsity(f, x, sd::AbstractSparsityDetector)::AbstractMatrix{Bool}
Use detector sd
to construct a (typically sparse) matrix S
describing the pattern of nonzeroes in the Hessian of f
applied at x
.
ADTypes.NoSparsityDetector
— TypeNoSparsityDetector <: AbstractSparsityDetector
Trivial sparsity detector, which always returns a full sparsity pattern (only ones, no zeroes).
See also
ADTypes.KnownJacobianSparsityDetector
— TypeKnownJacobianSparsityDetector(jacobian_sparsity::AbstractMatrix) <: AbstractSparsityDetector
Trivial sparsity detector used to return a known Jacobian sparsity pattern.
See also
ADTypes.KnownHessianSparsityDetector
— TypeKnownHessianSparsityDetector(hessian_sparsity::AbstractMatrix) <: AbstractSparsityDetector
Trivial sparsity detector used to return a known Hessian sparsity pattern.
See also
Coloring algorithm
ADTypes.coloring_algorithm
— Functioncoloring_algorithm(ad::AutoSparse)::AbstractColoringAlgorithm
Return the coloring algorithm for a sparse AD choice.
See also
ADTypes.AbstractColoringAlgorithm
— TypeAbstractColoringAlgorithm
Abstract supertype for Jacobian/Hessian coloring algorithms.
Required methods
Note
The terminology and definitions are taken from the following paper:
What Color Is Your Jacobian? Graph Coloring for Computing Derivatives, Assefaw Hadish Gebremedhin, Fredrik Manne, and Alex Pothen (2005)
ADTypes.column_coloring
— Functioncolumn_coloring(M::AbstractMatrix, ca::ColoringAlgorithm)::AbstractVector{<:Integer}
Use algorithm ca
to construct a structurally orthogonal partition of the columns of M
.
The result is a coloring vector c
of length size(M, 2)
such that for every non-zero coefficient M[i, j]
, column j
is the only column of its color c[j]
with a non-zero coefficient in row i
.
ADTypes.row_coloring
— Functionrow_coloring(M::AbstractMatrix, ca::ColoringAlgorithm)::AbstractVector{<:Integer}
Use algorithm ca
to construct a structurally orthogonal partition of the rows of M
.
The result is a coloring vector c
of length size(M, 1)
such that for every non-zero coefficient M[i, j]
, row i
is the only row of its color c[i]
with a non-zero coefficient in column j
.
ADTypes.symmetric_coloring
— Functionsymmetric_coloring(M::AbstractMatrix, ca::ColoringAlgorithm)::AbstractVector{<:Integer}
Use algorithm ca
to construct a symmetrically structurally orthogonal partition of the columns (or rows) of the symmetric matrix M
.
The result is a coloring vector c
of length size(M, 1) == size(M, 2)
such that for every non-zero coefficient M[i, j]
, at least one of the following conditions holds:
- column
j
is the only column of its colorc[j]
with a non-zero coefficient in rowi
; - column
i
is the only column of its colorc[i]
with a non-zero coefficient in rowj
.
ADTypes.NoColoringAlgorithm
— TypeNoColoringAlgorithm <: AbstractColoringAlgorithm
Trivial coloring algorithm, which always returns a different color for each matrix column/row.
See also
Modes
ADTypes.mode
— Functionmode(ad::AbstractADType)
Return the differentiation mode of ad
, as a subtype of AbstractMode
.
ADTypes.AbstractMode
— TypeAbstractMode
Abstract supertype for the traits identifying differentiation modes.
Subtypes
ADTypes.ForwardMode
— TypeForwardMode
Trait for AD choices that rely on forward mode algorithmic differentiation or finite differences.
These two paradigms are classified together because they can both efficiently compute Jacobian-vector products.
ADTypes.ForwardOrReverseMode
— TypeForwardOrReverseMode
Trait for AD choices that can work either in ForwardMode
or ReverseMode
, depending on their configuration.
This trait should rarely be used, because more precise dispatches to ForwardMode
or ReverseMode
should be defined.
ADTypes.ReverseMode
— TypeReverseMode
Trait for AD choices that rely on reverse mode algorithmic differentiation.
ADTypes.SymbolicMode
— TypeSymbolicMode
Trait for AD choices that rely on symbolic differentiation.
Miscellaneous
ADTypes.Auto
— FunctionADTypes.Auto(package::Symbol)
A shortcut that converts an AD package name into an instance of AbstractADType
, with all parameters set to their default values.
This function is type-unstable by design and might lead to suboptimal performance. In most cases, you should never need it: use the individual backend types directly.
Example
import ADTypes
backend = ADTypes.Auto(:Zygote)
# output
ADTypes.AutoZygote()
Deprecated
ADTypes.AutoTapir
— TypeAutoTapir
AutoTapir
is deprecated following a package renaming, please use AutoMooncake
instead.
Struct used to select the Tapir.jl backend for automatic differentiation.
Defined by ADTypes.jl.
Constructors
AutoTapir(; safe_mode=true)
Fields
safe_mode::Bool
: whether to run additional checks to catch errors early.