DANN implementation for image classification

[zobapt]

Hello,
I am currently working on a domain adaptation project involving image datasets. I have successfully used the ADAPT package and especially DANN implementation in your tutorial, but I am unsure if the same techniques can be extended to image data.
I was wondering if you have examples of DANN implementation for computer vision tasks like multiclass classification, and if so what did you change in the parameters?
I am currently trying to apply your DANN implementation with a model trained on Mnist as source domain, to SVHN considered as target domain (or the opposite), would you happen to have tried it ?

Thank you for your time and for developing such a useful tool for domain adaptation. I look forward to your response and any guidance you can provide.
Best regards

[antoinedemathelin]

Hi @zobapt,

Thank you for asking! Yes, Adapt can be used for image classification tasks! Here is an example for the adaptation from SVHN to MNIST that should run on Google Colab:

Here is the link to the colab: https://colab.research.google.com/drive/1P9LUA9UCGlvCCJs9dg244NdyKdIiHSCJ?usp=sharing

Here is the code:

# Install adapt and come back to Tensorflow 2.15 (Adapt does not work with TF 2.16)
!pip install adapt
!pip install tensorflow==2.15

# Import packages
import tensorflow as tf
from tensorflow.keras import layers
import tensorflow_datasets as tfds
from sklearn.manifold import TSNE
import numpy as np
import matplotlib.pyplot as plt

from adapt.feature_based import DANN

# Load datasets
source_train, source_test = tfds.load('svhn_cropped', split=['train', 'test'], shuffle_files=False)
target_train, target_test = tfds.load('mnist', split=['train', 'test'], shuffle_files=False)

# Preprocessing
# IMPORTANT: Deep learning methods from Adapt require tuples (images, labels) instead of dictionaries
# SVHN images are cropped from (32, 32) to (28, 28) and MNIST are duplicated along the last axis to get 3 channels
source_train = source_train.map(lambda x: ((tf.cast(x['image'][2:-2, 2:-2, :], "float64") - 127.5) / 127.5,
                                           tf.one_hot(x['label'], depth=10)))
source_test = source_test.map(lambda x: ((tf.cast(x['image'][2:-2, 2:-2, :], "float64") - 127.5) / 127.5,
                                         tf.one_hot(x['label'], depth=10)))

target_train = target_train.map(lambda x: ((tf.cast(tf.concat([x['image']]*3, axis=-1), "float64") - 127.5) / 127.5,
                                           tf.one_hot(x['label'], depth=10)))
target_test = target_test.map(lambda x: ((tf.cast(tf.concat([x['image']]*3, axis=-1), "float64") - 127.5) / 127.5,
                                         tf.one_hot(x['label'], depth=10)))

input_target_train = target_train.map(lambda x, y: x)        # Extract inputs for the target (target labels are not given during training)
input_source_train = source_train.map(lambda x, y: x)

# Define networks
def make_encoder_model():
    model = tf.keras.Sequential()
    model.add(layers.Conv2D(64, (5, 5), strides=(1, 1), padding='same',
                                     input_shape=[28, 28, 3]))
    model.add(layers.ReLU())
    model.add(layers.MaxPool2D((3, 3), strides=(2, 2)))

    model.add(layers.Conv2D(64, (5, 5), strides=(1, 1), padding='same'))
    model.add(layers.ReLU())
    model.add(layers.MaxPool2D((3, 3), strides=(2, 2)))

    model.add(layers.Conv2D(128, (5, 5), strides=(1, 1), padding='valid'))
    model.add(layers.ReLU())

    model.add(layers.Flatten())
    return model

def make_discriminator_model():
    model = tf.keras.Sequential()
    model.add(layers.Dense(1024, input_shape=[512,]))
    model.add(layers.ReLU())

    model.add(layers.Dense(1024))
    model.add(layers.ReLU())

    model.add(layers.Dense(1))
    model.add(layers.Activation("sigmoid"))
    return model

def make_task_model():
    model = tf.keras.Sequential()
    model.add(layers.Dense(1024, input_shape=[512,]))
    model.add(layers.ReLU())

    model.add(layers.Dense(1024))
    model.add(layers.ReLU())

    model.add(layers.Dense(10))
    model.add(layers.Softmax())
    return model
    
# Instantiate DANN
encoder = make_encoder_model()
discriminator = make_discriminator_model()
task = make_task_model()

dann_model = DANN(encoder=encoder, discriminator=discriminator, task=task, lambda_=0.1, copy=False)
dann_model.compile(loss=tf.keras.losses.CategoricalCrossentropy(from_logits=False),
                   optimizer=tf.keras.optimizers.legacy.SGD(0.1),
                   metrics=["acc"])
                   
# Train DANN
dann_model.fit(X=source_train, Xt=input_target_train, batch_size=128, epochs=10,
               shuffle=True, validation_data=target_test, buffer_size=1000)

[antoinedemathelin]

Without adaptation (lambda_=0. in DANN)

Target test accuracy: ~0.53

With adaptation (lambda_=0.1 in DANN)

Target test accuracy: ~0.69

We are close to the results presented in the paper

[zobapt]

I'm very grateful for your complete answer. I managed to reproduce it without any problems.
Thank you very much !