Pure Java Rumelhart's Multilayer Perceptron

No more fat JAR required for starting from scratch. JAR's total size is about 50 kB!

Implements pure java Multilayer Perceptron with one of the Activation Functions and Backpropagation training algorithm.

📰 See an example of use.

Activation Functions

This activation functions ([1] and [2]) are implemented currently:

• ArcTan
• Bent Identity
• ELU
• Gaussian
• Heaviside
• Identity
• ISRLU
• ISRU
• Leaky ReLU
• ReLU
• Sigmoid
• SiLU
• Sinc
• Sin
• Softmax
• Softplus
• Softsign
• Tanh

Backpropagation Algorithm

Perceptron is trained by Backpropagation algorithm. Key formulas are described below.

At each iteration weight between i-th and j-th node is changed by:

Δw_ij = - η * y_i * δ_j                (1)

where:
η - learning rate;
y_i - output of i-th node;
δ_j - delta coefficient for j-th node.

Delta coefficient for j-th node is calculation depends on location of the node and type of loss function.

For output layer node with Softmax activation function and Cross-entropy loss function delta is evaluated by:

δ_j = y_j - e_j                        (2)

where:
y_j - real output for j-th node;
e_j - expected output for j-th node.

For output layer node with other type of activation function, except Softmax, and Least Squares loss function delta is evaluated by:

δ_j = (y_j - e_j) * f'(S_j)            (3)

where:
f'(S_j) - derivative of activation function;
S_j - input signal for j-th node.

Node input signal is:

S_j = sum(y_i * w_ij)                  (4)

where:
y_i - output of i-th node (located closer to the input layer), which connected to j-th node (located closer to output layer);
w_ij - weight between i-th and j-th node.

For hidden layer node Delta coefficient evaluated by:

δ_j = f'(S_j) * sum(w_jk * δ_k)        (5)

where:
w_jk - weight between j-th node (located closer to input layer) and k-th node (located closer to output layer);
δ_k - delta coefficient for k-th node (located closer to output layer); the coefficient δ_k has already been calculated earlier by according to formula (3) or (4) if k-th node in output layer, and by formula (5) in previous iteration if k-th node in other layers.

How to Use The Jar Library

There are 2 cases for get JAR package (~50 Kb).

1. Get JAR from GitHub Packages. In this case you should use GitHub Personal Access Token (PAT). First, add repository to you maven project

<repositories>
  <repository>
    <id>central</id>
    <url>https://repo.maven.apache.org/maven2</url>
    <snapshots>
      <enabled>false</enabled>
    </snapshots>
  </repository>
  <repository>
    <id>github</id>
    <url>https://maven.pkg.github.com/NeuroMachinesLab/perceptron</url>
  </repository>
</repositories>

Secondly, configure GitHub Personal Access Token (PAT) by this Tutorial. After that you can add package to dependencies

<dependency>
    <groupId>ai.neuromachines</groupId>
    <artifactId>perceptron</artifactId>
    <version>3.1</version>
</dependency>

2. Or you can get JAR package from JitPack repository without PAT. Add repository to you maven project

<repositories>
    <repository>
        <id>central</id>
        <name>Central Repository</name>
        <url>https://repo.maven.apache.org/maven2</url>
        <snapshots>
            <enabled>false</enabled>
        </snapshots>
    </repository>
    <repository>
        <id>jitpack.io</id>
        <url>https://jitpack.io</url>
    </repository>
</repositories>

and add dependency

<dependency>
    <groupId>com.github.NeuroMachinesLab</groupId>
    <artifactId>perceptron</artifactId>
    <version>3.1</version>
</dependency>

Please note that the groupId is different from the package on GitHub Packages. This is expected, this is how JitPack works.