Fix for flaky test AuthenticationTokenTest.equals #1338
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
The
AuthenticationTokenTest.Equals
fails because the token generated is not the same everytime. TheJSONObject.toString()
gives a non-deterministic output because becauseJSONObject
values are unordered. The solution is to sort the key-value pairs ofjsonHeader
andjsonPayload
before converting it to their string representation. I had to modifyXsuaaTokenCompTest.java
as it was affected by this change and had a failing test.Command to reproduce the failure:
mvn -pl spring-security edu.illinois:nondex-maven-plugin:2.1.7-SNAPSHOT:nondex -Dtest=com.sap.cloud.security.spring.token.authentication.AuthenticationTokenTest#equals
Error Output:
The test fails on 3/5 iterations of the NonDex tool
Reason:
The
org.json.JSONObject
class uses Maps as the data structure for theJSONObject
. And, according to HashMap documentation,"This class makes no guarantees as to the order of the map; in particular, it does not guarantee that the order will remain constant over time."
This causes the
jsonHeader.toString()
andjsonPayload.toString()
to give a non-deterministic output.About Flaky Tests:
Flaky tests are tests in software development that produce inconsistent or unreliable results, which can lead to non-deterministic outcomes of the test case, fixing them is important to both reliable testing and fixing vulnerabilities in the code.