Differentiable Swift is an experimental language feature for the Swift language that is currently in the pitch phase of the Swift Evolution process. The goal of this feature is to provide first-class, language-integrated support for differentiable programming, making Swift the first general-purpose, statically typed programming language to have automatic differentiation built in. Originally developed as part of the Swift for TensorFlow project, teams at PassiveLogic and elsewhere are currently working on it.
Differentiable Swift is purely a language feature and isn't tied to any specific machine learning framework or platform. It provides a means of building such frameworks in Swift, and works wherever Swift does: from Linux to macOS to WebAssembly.
The goal of this repository is to provide examples and documentation for differentiable Swift, to illustrate how it can be used, and to show the power of automatic differentiation in various applications. We hope to grow this over time with new examples and documentation, and welcome contributions to that end.
DocC-generated documentation can be found at https://passivelogic.github.io/differentiable-swift-examples
Differentiable Swift is present as an experimental feature in modern Swift toolchains. Due to the rapid speed at which it is evolving, for best results, we recommend using a Swift toolchain downloaded from Swift.org from either the Swift 5.9 development snapshots or the nightly development snapshots. The latter will more closely track the latest additions and fixes being upstreamed but may be slightly less stable overall.
It is possible to use differentiable Swift with the default Swift toolchains that ship inside Xcode, however only the
compiler additions are present in those toolchains. The standard library support needed to use the _Differentiation module
is not provided in those toolchains and needs to be added after the fact. One example of how to do this can be found
in this project.
No special compiler flags are needed to activate differentiable Swift, but you do need to place the following:
import _Differentiationin any file where differentiation will be used. The compiler will warn you about this if you do forget to add the above and try to use any differentiable Swift capabilities.
The following examples are present in the repository, and can be built and run via:
swift run [example]- BasicDifferentiation: A very simple example of using automatic differentiation with a few different functions and types.
- CustomDerivatives: Differentiable Swift lets you register custom derivatives for functions, and this shows how to do so.
- BasicGradientDescent: How to perform gradient descent optimization in Swift.
A motivating benchmark of a building thermal model, optimized via gradient descent, is implemented in several languages and frameworks to compare against differentiable Swift in the Benchmarks/BuildingSimulation directory.
If you want to learn more about differentiable Swift, there are a variety of resources out there. The API has changed over time, so some older documentation may provide great background on the feature but not fully reflect code as it is written today.
- Differentiable programming for gradient-based machine learning
- The Intro to Differentiable Swift series:
- Differentiable Programming Manifesto (note: slightly out of date)
- The Swift for TensorFlow project explored the use of differentiable Swift paired with machine learning frameworks: