Different prediction when changing args order

[shoegazerstella]

I noticed this strange behaviour on my model. The results of doing:
preds = model.predict([test_data_x1, test_data_x2, leaks_test])
are different from:
preds = model.predict([test_data_x2, test_data_x1, leaks_test])
when the only thing that changes is the order of the arguments.
Now I understand that the two networks can have different weights. But is there a way to understand when it's working properly and when it's not?

I have the same behaviour when using another embedding (sentence-bert), for this I slightly modified the network like this:

    # define inputs
    input1 = tf.keras.Input(shape=shape)
    input2 = tf.keras.Input(shape=shape)

    # Creating LSTM Encoder
    lstm_layer = Bidirectional(LSTM(number_lstm_units, 
                                    dropout=rate_drop_lstm, 
                                    recurrent_dropout=rate_drop_lstm))

    x1 = lstm_layer(input1)
    x2 = lstm_layer(input2)
    merged = concatenate([x1, x2])
    merged = BatchNormalization()(merged)
    merged = Dropout(rate_drop_dense)(merged)

    merged = Dense(number_dense_units, activation=activation_function)(merged)
    merged = BatchNormalization()(merged)
    merged = Dropout(rate_drop_dense)(merged)
    preds = Dense(1, activation='sigmoid')(merged)

    model = Model(inputs=[input1, input2], outputs=preds)
    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['acc']) #nadam

Do you have some suggestions on possible strategies to solve this?
Thanks a lot