# ADTypes.jl

Documentation for ADTypes.jl.

`ADTypes.ADTypes`

— Module`ADTypes.jl`

ADTypes.jl is a multi-valued logic system to choose an automatic differentiation (AD) package and specify its parameters.

`ADTypes.AbstractADType`

— Type`AbstractADType`

Abstract supertype for all AD choices.

## Dense AD

### Forward mode

Algorithmic differentiation:

`ADTypes.AutoForwardDiff`

— Type`AutoForwardDiff{chunksize,T}`

Struct used to select the ForwardDiff.jl backend for automatic differentiation.

Defined by ADTypes.jl.

**Constructors**

`AutoForwardDiff(; chunksize=nothing, tag=nothing)`

**Type parameters**

`chunksize`

: the preferred chunk size to evaluate several derivatives at once

**Fields**

`tag::T`

: a custom tag to handle nested differentiation calls (usually not necessary)

`ADTypes.AutoPolyesterForwardDiff`

— Type`AutoPolyesterForwardDiff{chunksize,T}`

Struct used to select the PolyesterForwardDiff.jl backend for automatic differentiation.

Defined by ADTypes.jl.

**Constructors**

`AutoPolyesterForwardDiff(; chunksize=nothing, tag=nothing)`

**Type parameters**

`chunksize`

: the preferred chunk size to evaluate several derivatives at once

**Fields**

`tag::T`

: a custom tag to handle nested differentiation calls (usually not necessary)

Finite differences:

`ADTypes.AutoFiniteDiff`

— Type`AutoFiniteDiff{T1,T2,T3}`

Struct used to select the FiniteDiff.jl backend for automatic differentiation.

Defined by ADTypes.jl.

**Constructors**

`AutoFiniteDiff(; fdtype=Val(:forward), fdjtype=fdtype, fdhtype=Val(:hcentral))`

**Fields**

`fdtype::T1`

: finite difference type`fdjtype::T2`

: finite difference type for the Jacobian`fdhtype::T3`

: finite difference type for the Hessian

`ADTypes.AutoFiniteDifferences`

— Type`AutoFiniteDifferences{T}`

Struct used to select the FiniteDifferences.jl backend for automatic differentiation.

Defined by ADTypes.jl.

**Constructors**

`AutoFiniteDifferences(; fdm)`

**Fields**

`fdm::T`

: a`FiniteDifferenceMethod`

### Reverse mode

`ADTypes.AutoReverseDiff`

— Type`AutoReverseDiff{compile}`

Struct used to select the ReverseDiff.jl backend for automatic differentiation.

Defined by ADTypes.jl.

**Constructors**

`AutoReverseDiff(; compile::Union{Val, Bool} = Val(false))`

**Fields**

`compile::Union{Val, Bool}`

: whether to compile the tape prior to differentiation (the boolean version is also the type parameter)

`ADTypes.AutoTapir`

— Type`AutoTapir`

Struct used to select the Tapir.jl backend for automatic differentiation.

Defined by ADTypes.jl.

**Constructors**

`AutoTapir(; safe_mode=true)`

**Fields**

`safe_mode::Bool`

: whether to run additional checks to catch errors early. While this is on by default to ensure that users are aware of this option, you should generally turn it off for actual use, as it has substantial performance implications. If you encounter a problem with using Tapir (it fails to differentiate a function, or something truly nasty like a segfault occurs), then you should try switching`safe_mode`

on and look at what happens. Often errors are caught earlier and the error messages are more useful.

`ADTypes.AutoTracker`

— Type`AutoTracker`

Struct used to select the Tracker.jl backend for automatic differentiation.

Defined by ADTypes.jl.

**Constructors**

`AutoTracker()`

`ADTypes.AutoZygote`

— Type`AutoZygote`

Struct used to select the Zygote.jl backend for automatic differentiation.

Defined by ADTypes.jl.

**Constructors**

`AutoZygote()`

### Forward or reverse mode

`ADTypes.AutoEnzyme`

— Type`AutoEnzyme{M}`

Struct used to select the Enzyme.jl backend for automatic differentiation.

Defined by ADTypes.jl.

**Constructors**

`AutoEnzyme(; mode=nothing)`

**Fields**

`mode::M`

: can be either- an object subtyping
`EnzymeCore.Mode`

(like`EnzymeCore.Forward`

or`EnzymeCore.Reverse`

) if a specific mode is required `nothing`

to choose the best mode automatically

- an object subtyping

`ADTypes.AutoChainRules`

— Type`AutoChainRules{RC}`

Struct used to select an automatic differentiation backend based on ChainRulesCore.jl (see the list here).

Defined by ADTypes.jl.

**Constructors**

`AutoChainRules(; ruleconfig)`

**Fields**

`ruleconfig::RC`

: a`ChainRulesCore.RuleConfig`

object.

`ADTypes.AutoDiffractor`

— Type`AutoDiffractor`

Struct used to select the Diffractor.jl backend for automatic differentiation.

Defined by ADTypes.jl.

**Constructors**

`AutoDiffractor()`

### Symbolic mode

`ADTypes.AutoFastDifferentiation`

— Type`AutoFastDifferentiation`

Struct used to select the FastDifferentiation.jl backend for automatic differentiation.

Defined by ADTypes.jl.

**Constructors**

`AutoFastDifferentiation()`

`ADTypes.AutoSymbolics`

— Type`AutoSymbolics`

Struct used to select the Symbolics.jl backend for automatic differentiation.

Defined by ADTypes.jl.

**Constructors**

`AutoSymbolics()`

## Sparse AD

`ADTypes.AutoSparse`

— Type`AutoSparse{D,S,C}`

Wraps an ADTypes.jl object to deal with sparse Jacobians and Hessians.

**Fields**

`dense_ad::D`

: the underlying AD package, subtyping`AbstractADType`

`sparsity_detector::S`

: the sparsity pattern detector, subtyping`AbstractSparsityDetector`

`coloring_algorithm::C`

: the coloring algorithm, subtyping`AbstractColoringAlgorithm`

**Constructors**

```
AutoSparse(
dense_ad;
sparsity_detector=ADTypes.NoSparsityDetector(),
coloring_algorithm=ADTypes.NoColoringAlgorithm()
)
```

`ADTypes.dense_ad`

— Function`dense_ad(ad::AutoSparse)::AbstractADType`

Return the underlying AD package for a sparse AD choice.

**See also**

### Sparsity detector

`ADTypes.sparsity_detector`

— Function`sparsity_detector(ad::AutoSparse)::AbstractSparsityDetector`

Return the sparsity pattern detector for a sparse AD choice.

**See also**

`ADTypes.AbstractSparsityDetector`

— Type`AbstractSparsityDetector`

Abstract supertype for sparsity pattern detectors.

**Required methods**

`ADTypes.jacobian_sparsity`

— Function```
jacobian_sparsity(f, x, sd::AbstractSparsityDetector)::AbstractMatrix{Bool}
jacobian_sparsity(f!, y, x, sd::AbstractSparsityDetector)::AbstractMatrix{Bool}
```

Use detector `sd`

to construct a (typically sparse) matrix `S`

describing the pattern of nonzeroes in the Jacobian of `f`

(resp. `f!`

) applied at `x`

(resp. `(y, x)`

).

`ADTypes.hessian_sparsity`

— Function`hessian_sparsity(f, x, sd::AbstractSparsityDetector)::AbstractMatrix{Bool}`

Use detector `sd`

to construct a (typically sparse) matrix `S`

describing the pattern of nonzeroes in the Hessian of `f`

applied at `x`

.

`ADTypes.NoSparsityDetector`

— Type`NoSparsityDetector <: AbstractSparsityDetector`

Trivial sparsity detector, which always returns a full sparsity pattern (only ones, no zeroes).

**See also**

### Coloring algorithm

`ADTypes.coloring_algorithm`

— Function`coloring_algorithm(ad::AutoSparse)::AbstractColoringAlgorithm`

Return the coloring algorithm for a sparse AD choice.

**See also**

`ADTypes.AbstractColoringAlgorithm`

— Type`AbstractColoringAlgorithm`

Abstract supertype for Jacobian/Hessian coloring algorithms, defined for example in SparseDiffTools.jl.

**Required methods**

**Note**

The terminology and definitions are taken from the following paper:

"What Color Is Your Jacobian? Graph Coloring for Computing Derivatives"

Assefaw Hadish Gebremedhin, Fredrik Manne, and Alex Pothen (2005)

https://epubs.siam.org/doi/10.1137/S0036144504444711

`ADTypes.column_coloring`

— Function`column_coloring(M::AbstractMatrix, ca::ColoringAlgorithm)::AbstractVector{<:Integer}`

Use algorithm `ca`

to construct a structurally orthogonal partition of the columns of `M`

.

The result is a coloring vector `c`

of length `size(M, 2)`

such that for every non-zero coefficient `M[i, j]`

, column `j`

is the only column of its color `c[j]`

with a non-zero coefficient in row `i`

.

`ADTypes.row_coloring`

— Function`row_coloring(M::AbstractMatrix, ca::ColoringAlgorithm)::AbstractVector{<:Integer}`

Use algorithm `ca`

to construct a structurally orthogonal partition of the rows of `M`

.

The result is a coloring vector `c`

of length `size(M, 1)`

such that for every non-zero coefficient `M[i, j]`

, row `i`

is the only row of its color `c[i]`

with a non-zero coefficient in column `j`

.

`ADTypes.symmetric_coloring`

— Function`symmetric_coloring(M::AbstractMatrix, ca::ColoringAlgorithm)::AbstractVector{<:Integer}`

Use algorithm `ca`

to construct a symmetrically structurally orthogonal partition of the columns (or rows) of the symmetric matrix `M`

.

The result is a coloring vector `c`

of length `size(M, 1) == size(M, 2)`

such that for every non-zero coefficient `M[i, j]`

, at least one of the following conditions holds:

- column
`j`

is the only column of its color`c[j]`

with a non-zero coefficient in row`i`

; - column
`i`

is the only column of its color`c[i]`

with a non-zero coefficient in row`j`

.

`ADTypes.NoColoringAlgorithm`

— Type`NoColoringAlgorithm <: AbstractColoringAlgorithm`

Trivial coloring algorithm, which always returns a different color for each matrix column/row.

**See also**

## Modes

`ADTypes.mode`

— Function`mode(ad::AbstractADType)`

Return the differentiation mode of `ad`

, as a subtype of `AbstractMode`

.

`ADTypes.AbstractMode`

— Type`AbstractMode`

Abstract supertype for the traits identifying differentiation modes.

**Subtypes**

`ADTypes.ForwardMode`

— Type`ForwardMode`

Trait for AD choices that rely on forward mode algorithmic differentiation or finite differences.

These two paradigms are classified together because they can both efficiently compute Jacobian-vector products.

`ADTypes.ForwardOrReverseMode`

— Type`ForwardOrReverseMode`

Trait for AD choices that can work either in `ForwardMode`

or `ReverseMode`

, depending on their configuration.

This trait should rarely be used, because more precise dispatches to `ForwardMode`

or `ReverseMode`

should be defined.

`ADTypes.ReverseMode`

— Type`ReverseMode`

Trait for AD choices that rely on reverse mode algorithmic differentiation.

`ADTypes.SymbolicMode`

— Type`SymbolicMode`

Trait for AD choices that rely on symbolic differentiation.

## Miscellaneous

`ADTypes.Auto`

— Function`ADTypes.Auto(package::Symbol)`

A shortcut that converts an AD package name into an instance of `AbstractADType`

, with all parameters set to their default values.

This function is type-unstable by design and might lead to suboptimal performance. In most cases, you should never need it: use the individual backend types directly.

**Example**

```
import ADTypes
backend = ADTypes.Auto(:Zygote)
# output
ADTypes.AutoZygote()
```