NeuralLearnability

Map the learnability landscape of neural networks across function space.

Overview

NeuralLearnability trains a small neural network independently at each point of a 1D or 2D grid over the parameters of a target function, producing learnability landscapes. For example, sweeping a and b in f(x) = x³ + ax² + x + b reveals fractal boundaries between convergence and divergence — regions where small changes in the target function flip the training outcome.

Installation

PacletInstall["https://wolfr.am/1CUVgCFzU"]
Needs["NeuralLearnability`"]

Quick Start

(* 1D sweep *)
r = NeuralSweep[
  "Axis1" -> {0, 3, 200},
  "TargetFunction" -> Exp[-a x^2],
  "Steps" -> 500, "NSeeds" -> 5]

r["FinalLossPlot"]
r["LossCurvePlot"]
r["Properties"]

(* 2D fractal landscape *)
r = NeuralSweep[
  "Axis1" -> {-1, 2, 1000},
  "Axis2" -> {-2, 1, 1000},
  "TargetFunction" -> x^3 + a x^2 + x + b,
  "HiddenSize" -> 16, "Steps" -> 1000, "NSeeds" -> 1,
  "LearningRate" -> 0.25, "FixedSeed" -> True,
  "ReturnLossCurve" -> False, "ReturnAllCurves" -> False]
r["FinalLossPlot", "Log" -> True]

Features

Sweep axes: "Axis1" -> {min, max, n} sweeps parameter a in the target function. "Axis2" adds parameter b for 2D sweeps. Hyperparameter axes: "Axis1" -> {"LearningRate", min, max, n, "Log"}.

Target functions: Any expression using x, a, b — symbolic (Exp[-a x^2]) or string ("Sin[a x] + Cos[b x]"). Built-in functions: Sin, Cos, Exp, Log, Tanh, Sqrt, Abs, UnitBox, UnitStep, Sinc, Sign, and more.

Network: "HiddenSize", "NLayers", "Activation" ("Tanh", "ReLU", "Sigmoid", "GELU", "Swish"), "Residual" connections.

Optimizers: "SGD" (with optional "Momentum"), "RMSProp", "Adam". All with configurable parameters.

Loss functions: "MSE" (= "L2"), "MAE" (= "L1"), "Huber".

Regularization: "L2Reg", "L1Reg", "MomentReg" (penalizes weight distribution drift from initialization), "GradClip".

Training data: "NData", "Domain" -> {min, max}, "DataDist" ("uniform" or "random").

Seeding: "Seed" sets the base seed. "FixedSeed" -> True uses identical initial weights for all grid points (essential for clean fractal structure).

Diagnostics: "TrackWeightNorm", "TrackGradNorm", "TrackValidation", "TrackConvergenceStep" — all optional.

Precision: "Float64" for double precision. "FastMath" for ~2x speedup via Padé tanh approximation.

Output control: "ReturnLossCurve" -> False skips the per-step loss curve. "ReturnAllCurves" -> False skips per-seed curves. For large sweeps where you only need the final landscape, set both to False.

Output: NeuralSweepResult object with data and plot properties. Use r["Properties"] to list all.

Examples

See Notebooks/UsageExamples.nb for worked examples covering activation functions, optimizers, learning rate sweeps, depth comparisons, diagnostics, and more.

Building from Source

Requires CMake 3.14+ and a C++17 compiler.

./build.sh              # all 6 platforms
./build.sh --linux-only # just Linux

Cross-compilation dependencies:

• Linux ARM64: g++-aarch64-linux-gnu
• Windows x86-64: MinGW (g++-mingw-w64-x86-64)
• Windows ARM64: clang-21+
• macOS: OSXCross at ~/osxcross

Testing

wolframscript -file Testing/test_suite.wls

License

MIT