You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
DeepCpG consists of a DNA module to recognize features in the DNA sequence, a CpG module to recognize features in the methylation neighborhood of multiple cells, and joint module to combine the features from the DNA and CpG module.
4
+
5
+
DeepCpG provides different architectures for the DNA, CpG, and joint module. Architectures differ in the number of layers and neurons, and are hence more or less complex. More complex modules are usually more accurate, but more expensive to train. You can select a certain architecture using the `--dna_module`, `--cpg_model`, and `--joint_model` argument of `dcpg_train.py`, for example:
6
+
7
+
```
8
+
dcpg_train.py
9
+
--dna_module CnnL2h128
10
+
--cpg_module RnnL1
11
+
--joint_module JointL2h512
12
+
```
13
+
14
+
In the following, the following layer specifications will be used:
Th prefixes `Cnn`, `CnnRnn`, `ResNet`, `ResConv`, and `ResAtrous` denote the class of the DNA module.
46
+
47
+
Modules starting with `Cnn` are convolutional neural networks (CNNs). DeepCpG CNN architectures consist of a series of convolutional and max-pooling layers, which are followed by one fully-connected layer. Module `CnnLxhy` has `x` convolutional-pooling layers, and one fully-connected layer with `y` units. For example, `CnnL2h128` has two convolutional layers, and one fully-connected layer with 128 units. `CnnL3h256` has three convolutional layers and one fully-connected layer with 256 units. `CnnL1h128` is the fastest module, but modules with more layers and neurons usually perform better. In my experiments, `CnnL2h128` provided a good trade-off between performance and runtime, which I recommend as default.
48
+
49
+
`CnnRnn01` is a [convolutional-recurrent neural network](http://nar.oxfordjournals.org/content/44/11/e107). It consists of two convolutional-pooling layers, which are followed by a bidirectional recurrent neural network (RNN) with one layer and gated recurrent units (GRUs). `CnnRnn01` is slower than `Cnn` architectures and did not perform better in my experiments.
50
+
51
+
Modules starting with `ResNet` are [residual neural networks](https://arxiv.org/abs/1603.05027). ResNets are very deep networks with skip connections to improve the gradient flow and to allow learning how many layers to use. A residual network consists of multiple residual blocks, and each residual block consists of multiple residual units. Residual units have a bottleneck architecture with three convolutional layers to speed up computations. `ResNet01` and `ResNet02` have three residual blocks with two and three residual units, respectively. ResNets are slower than CNNs, but can perform better on large datasets.
52
+
53
+
Modules starting with `ResConv` are ResNets with modified residual units that have two convolutional layers instead of a bottleneck architecture. `ResConv` modules performed worse than `ResNet` modules in my experiments.
54
+
55
+
Modules starting with `ResAtrous` are ResNets with modified residual units that use [Atrous convolutional layers](http://arxiv.org/abs/1511.07122) instead of normal convolutional layers. Atrous convolutional layers have dilated filters, i.e. filters with 'holes', which allow scanning wider regions in the inputs sequence and thereby better capturing distant patters in the DNA sequence. However, `ResAtrous` modules performed worse than `ResNet` modules in my experiments
`FcAvg` is a lightweight module with only 54000 parameters, which first transforms observed neighboring CpG sites of all cells independently, and than averages the transformed features across cells. `FcAvg` is very fast, but performs worse than RNN modules.
67
+
68
+
`Rnn` modules consists of bidirectional recurrent neural networks (RNNs) with gated recurrent units (GRUs) to summarize the methylation neighborhood of cells in a more clever way than averaging. `RnnL1` consists of one fully-connected layer with 256 units to transform the methylation neighborhood of each cell independently, and one bidirectional GRU with 2x256 units to summarize the transformed methylation neighborhood of cells. `RnnL2` has two instead of one GRU layer. `RnnL1` is faster and performed as good as `RnnL2` in my experiments.
Joint modules join the feature from the DNA and CpG module. `JointL0` simply concatenates the features and has no learnable parameters (ultra fast). `JointLXh512` has `X` fully-connect layers with 512 neurons. Modules with more layers usually perform better, at the cost of a higher runtime. I recommend using `JointL2h512` or `JointL3h12`.
0 commit comments