Missing Batch Normalization in Downsample Class

[SeunghanYu]

Hi @johnnynunez and @ahatamiz !

Thank you for your excellent work on MambaVision!

I have been reviewing the architecture described in Section 3.1 ("Macro Architecture") of the paper, where the downsampler is described as a "batch normalized 3×3 CNN layer with stride 2" that reduces the image resolution by half.
However, in the Downsample class in mamba_vision.py (227-254), I noticed that the Batch Normalization step appears to be omitted, with only the 3×3 convolution (stride 2) implemented:

class Downsample(nn.Module):
    """
    Down-sampling block"
    """

    def __init__(self,
                 dim,
                 keep_dim=False,
                 ):
        """
        Args:
            dim: feature size dimension.
            norm_layer: normalization layer.
            keep_dim: bool argument for maintaining the resolution.
        """

        super().__init__()
        if keep_dim:
            dim_out = dim
        else:
            dim_out = 2 * dim
        self.reduction = nn.Sequential(
            nn.Conv2d(dim, dim_out, 3, 2, 1, bias=False),
        )

    def forward(self, x):
        x = self.reduction(x)
        return x

Could you please clarify whether the Batch Normalization is intended to be part of the downsampler, as described in the paper, or if the omission in the code is deliberate?

Thank you very much for your time and assistance!

Best regards,
SeunghanYu

[ahatamiz]

Hi @SeunghanYu

Thank you so much for your attention to the details. This is indeed a typo and we have already removed the word "batch normalized" to avoid any confusions in the next iteration (camera-ready version) of the manuscript.

Kind Regards,
Ali Hatamizadeh

[ahatamiz]

Hi @SeunghanYu please see the update manuscript that reflects this change.

Thanks again for your attention to details !

Regards

[SeunghanYu]

Hi @ahatamiz!

Thank you so much for the clarification.
I’m glad to hear the updated manuscript addresses this point, and I look forward to reading the newly updated version.
Thank you again for your prompt response and for sharing these updates!

Best regards,
SeunghanYu