a newly generated user API token is not recognised with Azure AD >= 340.vdef002cf6415

[michelgasser]

Jenkins and plugins versions report

Environment

Jenkins: 2.387.2
OS: Linux - 4.18.0-425.19.2.el8_7.x86_64
Java: 11.0.18 - Eclipse Adoptium (OpenJDK 64-Bit Server VM)
---
Office-365-Connector:4.18.0
Parameterized-Remote-Trigger:3.1.6.3
allure-jenkins-plugin:2.30.3
analysis-model-api:11.1.0
android-emulator:3.1.3
ansicolor:1.0.2
ant:487.vd79d090d4ea_e
antisamy-markup-formatter:159.v25b_c67cd35fb_
apache-httpcomponents-client-4-api:4.5.14-150.v7a_b_9d17134a_5
artifactory:3.18.1
audit-trail:333.vb_e1b_b_0f1238c
authentication-tokens:1.53.v1c90fd9191a_b_
avatar:1.2
aws-credentials:191.vcb_f183ce58b_9
aws-java-sdk:1.12.447-382.vda_68e2007233
aws-java-sdk-cloudformation:1.12.447-382.vda_68e2007233
aws-java-sdk-codebuild:1.12.447-382.vda_68e2007233
aws-java-sdk-ec2:1.12.447-382.vda_68e2007233
aws-java-sdk-ecr:1.12.447-382.vda_68e2007233
aws-java-sdk-ecs:1.12.447-382.vda_68e2007233
aws-java-sdk-efs:1.12.447-382.vda_68e2007233
aws-java-sdk-elasticbeanstalk:1.12.447-382.vda_68e2007233
aws-java-sdk-iam:1.12.447-382.vda_68e2007233
aws-java-sdk-kinesis:1.12.447-382.vda_68e2007233
aws-java-sdk-logs:1.12.447-382.vda_68e2007233
aws-java-sdk-minimal:1.12.447-382.vda_68e2007233
aws-java-sdk-sns:1.12.447-382.vda_68e2007233
aws-java-sdk-sqs:1.12.447-382.vda_68e2007233
aws-java-sdk-ssm:1.12.447-382.vda_68e2007233
azure-ad:340.vdef002cf6415
azure-sdk:132.v62b_48eb_6f32f
basic-branch-build-strategies:71.vc1421f89888e
bitbucket:223.vd12f2bca5430
bitbucket-scm-trait-commit-skip:0.4.0
blueocean:1.27.3
blueocean-autofavorite:1.2.5
blueocean-bitbucket-pipeline:1.27.3
blueocean-commons:1.27.3
blueocean-config:1.27.3
blueocean-core-js:1.27.3
blueocean-dashboard:1.27.3
blueocean-display-url:2.4.2
blueocean-events:1.27.3
blueocean-git-pipeline:1.27.3
blueocean-github-pipeline:1.27.3
blueocean-i18n:1.27.3
blueocean-jira:1.27.3
blueocean-jwt:1.27.3
blueocean-personalization:1.27.3
blueocean-pipeline-api-impl:1.27.3
blueocean-pipeline-editor:1.27.3
blueocean-pipeline-scm-api:1.27.3
blueocean-rest:1.27.3
blueocean-rest-impl:1.27.3
blueocean-web:1.27.3
bootstrap4-api:4.6.0-5
bootstrap5-api:5.2.2-2
bouncycastle-api:2.27
branch-api:2.1071.v1a_188a_562481
browserstack-integration:1.2.8
build-blocker-plugin:1.7.8
build-metrics:1.3
build-monitor-plugin:1.14-681.vd6817317a_2b_7
build-name-setter:2.2.0
build-pipeline-plugin:1.5.8
build-timeout:1.30
build-user-vars-plugin:1.9
bulk-builder:1.5
caffeine-api:3.1.6-115.vb_8b_b_328e59d8
checks-api:2.0.0
claim:516.v36293563731d
cloudbees-bitbucket-branch-source:800.va_b_b_9a_a_5035c1
cloudbees-disk-usage-simple:178.v1a_4d2f6359a_8
cloudbees-folder:6.815.v0dd5a_cb_40e0e
command-launcher:100.v2f6722292ee8
commons-httpclient3-api:3.1-3
commons-lang3-api:3.12.0-36.vd97de6465d5b_
commons-text-api:1.10.0-36.vc008c8fcda_7b_
conditional-buildstep:1.4.2
config-file-provider:3.11.1
configuration-as-code:1625.v27444588cc3d
console-column-plugin:197.vcf5a_ec1d7b_47
convert-to-pipeline:1.0
credentials:1236.v31e44e6060c0
credentials-binding:604.vb_64480b_c56ca_
cron_column:1.7
cucumber-reports:5.7.5
custom-tools-plugin:0.8
dashboard-view:2.472.v9ff2a_e6a_c529
data-tables-api:1.13.3-3
dependency-check-jenkins-plugin:5.4.0
dependency-track:4.3.1
display-url-api:2.3.7
docker-commons:419.v8e3cd84ef49c
docker-java-api:3.2.13-68.va_875df25a_b_45
docker-plugin:1.3.0
docker-workflow:563.vd5d2e5c4007f
dtkit-api:3.0.2
durable-task:504.vb10d1ae5ba2f
ec2:2.0.7
echarts-api:5.4.0-3
email-ext:2.96
embeddable-build-status:369.vb_a_68a_575a_b_11
envinject:2.901.v0038b_6471582
envinject-api:1.199.v3ce31253ed13
extended-choice-parameter:373.v1a_ecea_fdf2a_a_
extended-read-permission:3.2
external-monitor-job:203.v683c09d993b_9
favorite:2.4.1
flyway-runner:1.9
font-awesome-api:6.3.0-2
forensics-api:2.1.0
generic-webhook-trigger:1.86.3
git:5.0.1
git-client:4.2.0
git-parameter:0.9.18
git-server:99.va_0826a_b_cdfa_d
github:1.37.0
github-api:1.303-417.ve35d9dd78549
github-branch-source:1703.vd5a_2b_29c6cdc
global-build-stats:269.v214f74360b_3a_
google-oauth-plugin:1.0.8
gradle:2.5.1
groovy:453.vcdb_a_c5c99890
handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953
hashicorp-vault-pipeline:1.4
hashicorp-vault-plugin:360.v0a_1c04cf807d
htmlpublisher:1.31
hudson-wsclean-plugin:1.0.8
instance-identity:142.v04572ca_5b_265
ionicons-api:45.vf54fca_5d2154
ivy:2.4
jackson2-api:2.15.0-334.v317a_165f9b_7c
jakarta-activation-api:2.0.1-3
jakarta-mail-api:2.0.1-3
javadoc:233.vdc1a_ec702cff
javax-activation-api:1.2.0-6
javax-mail-api:1.6.2-9
jaxb:2.3.8-1
jdk-tool:66.vd8fa_64ee91b_d
jenkins-design-language:1.27.3
jersey2-api:2.39.1-1
jira:3.9
jira-steps:2.0.165.v8846cf59f3db
jjwt-api:0.11.5-77.v646c772fddb_0
jnr-posix-api:3.1.16-2
job-dsl:1.83
job-import-plugin:3.6
jquery:1.12.4-1
jquery3-api:3.6.4-1
jsch:0.1.55.61.va_e9ee26616e7
junit:1198.ve38db_d1b_c975
kubernetes:3923.v294a_d4250b_91
kubernetes-client-api:6.4.1-215.v2ed17097a_8e9
kubernetes-credentials:0.10.0
ldap:673.v034ec70ec2b_b_
lockable-resources:1150.v59db_2b_994618
log-parser:2.3.0
mailer:448.v5b_97805e3767
mapdb-api:1.0.9-28.vf251ce40855d
mask-passwords:150.vf80d33113e80
matrix-auth:3.1.7
matrix-project:789.v57a_725b_63c79
maven-plugin:3.22
mercurial:1260.vdfb_723cdcc81
metrics:4.2.13-420.vea_2f17932dd6
mina-sshd-api-common:2.9.2-62.v199162f0a_2f8
mina-sshd-api-core:2.9.2-62.v199162f0a_2f8
monitoring:1.94.0
msbuild:1.30
naginator:1.18.2
nested-view:1.31
node-iterator-api:49.v58a_8b_35f8363
nodelabelparameter:1.11.0
oauth-credentials:0.645.ve666a_c332668
okhttp-api:4.10.0-132.v7a_7b_91cef39c
ownership:0.13.0
pam-auth:1.10
parameterized-scheduler:1.2
parameterized-trigger:2.45
parasoft-findings:10.6.2
performance:918.v5511b_a_d40338
pipeline-aws:1.43
pipeline-build-step:491.v1fec530da_858
pipeline-config-history:1.6
pipeline-github-lib:42.v0739460cda_c4
pipeline-graph-analysis:202.va_d268e64deb_3
pipeline-groovy-lib:656.va_a_ceeb_6ffb_f7
pipeline-input-step:468.va_5db_051498a_4
pipeline-milestone-step:111.v449306f708b_7
pipeline-model-api:2.2125.vddb_a_44a_d605e
pipeline-model-definition:2.2125.vddb_a_44a_d605e
pipeline-model-extensions:2.2125.vddb_a_44a_d605e
pipeline-rest-api:2.32
pipeline-stage-step:305.ve96d0205c1c6
pipeline-stage-tags-metadata:2.2125.vddb_a_44a_d605e
pipeline-stage-view:2.32
pipeline-utility-steps:2.15.2
plain-credentials:143.v1b_df8b_d3b_e48
plugin-util-api:3.2.0
popper-api:1.16.1-3
popper2-api:2.11.6-2
port-allocator:1.10
prism-api:1.29.0-4
prometheus:2.2.2
publish-over:0.22
pubsub-light:1.17
purge-build-queue-plugin:88.v23b_97b_f2c7a_d
rebuild:320.v5a_0933a_e7d61
resource-disposer:0.22
run-condition:1.5
saml:4.403.v423b_3195a_9ec
scm-api:667.v8b_6e07cdc7f2
scoring-load-balancer:59.vf791549fa_989
script-security:1244.ve463715a_f89c
snakeyaml-api:1.33-95.va_b_a_e3e47b_fa_4
sonar:2.15
splunk-devops:1.10.1
splunk-devops-extend:1.10.1
sse-gateway:1.26
ssh:2.6.1
ssh-agent:333.v878b_53c89511
ssh-credentials:305.v8f4381501156
ssh-slaves:2.877.v365f5eb_a_b_eec
sshd:3.275.v9e17c10f2571
stashNotifier:1.28
structs:324.va_f5d6774f3a_d
subversion:2.17.2
support-core:1274.v097a_073e7733
swarm:3.40
timestamper:1.24
token-macro:359.vb_cde11682e0c
trilead-api:2.84.v72119de229b_7
uno-choice:2.6.5
variant:59.vf075fe829ccb
view-job-filters:364.v48a_33389553d
warnings-ng:10.1.0
workflow-aggregator:596.v8c21c963d92d
workflow-api:1208.v0cc7c6e0da_9e
workflow-basic-steps:1017.vb_45b_302f0cea_
workflow-cps:3659.v582dc37621d8
workflow-durable-task-step:1246.v5524618ea_097
workflow-job:1292.v27d8cc3e2602
workflow-multibranch:746.v05814d19c001
workflow-scm-step:408.v7d5b_135a_b_d49
workflow-step-api:639.v6eca_cd8c04a_a_
workflow-support:839.v35e2736cfd5c
ws-cleanup:0.45
xfpanel:2.0.1
xml-job-to-job-dsl:0.1.13
xray-connector:2.6.1
xunit:3.1.2
xvfb:1.2
zap:1.1.0

What Operating System are you using (both controller, and any agents involved in the problem)?

NAME="Debian GNU/Linux"
VERSION_ID="11"

Reproduction steps

The build-agent connection with Jenkins-Master 2.387.2 and Azure AD 313.v14b_f37ff114d works without any problems:
/opt/openjdk-17/bin/java
-jar ~/swarm-client.jar
-executors 1
-fsroot /home/jenkins
-master master.url
-name build-agent
-username user-name
-password user-token
-labels "java"
-mode exclusive

When the Azure AD plugin is updated to version 340.vdef002cf6415, the connection of the Build-Agents no longer works. Here is the error message on the Build-Agent:

May 02, 2023 3:11:59 PM hudson.plugins.swarm.Client run
INFO: Attempting to connect to https://ci-i.sbb.ch/
May 02, 2023 3:11:59 PM hudson.plugins.swarm.SwarmClient getCsrfCrumb
SEVERE: Could not obtain CSRF crumb. Response code: 401
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 401 Unauthorized</title>
</head>
<body><h2>HTTP ERROR 401 Unauthorized</h2>
<table>
<tr><th>URI:</th><td>/crumbIssuer/api/xml</td></tr>
<tr><th>STATUS:</th><td>401</td></tr>
<tr><th>MESSAGE:</th><td>Unauthorized</td></tr>
<tr><th>SERVLET:</th><td>Stapler</td></tr>
</table>
<hr/><a href="https://eclipse.org/jetty">Powered by Jetty:// 10.0.13</a><hr/>

</body>
</html>

May 02, 2023 3:11:59 PM hudson.plugins.swarm.Client run
SEVERE: An error occurred
hudson.plugins.swarm.RetryException: Failed to create a Swarm agent on Jenkins. Response code: 401
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 401 Unauthorized</title>
</head>
<body><h2>HTTP ERROR 401 Unauthorized</h2>
<table>
<tr><th>URI:</th><td>/plugin/swarm/createSlave</td></tr>
<tr><th>STATUS:</th><td>401</td></tr>
<tr><th>MESSAGE:</th><td>Unauthorized</td></tr>
<tr><th>SERVLET:</th><td>Stapler</td></tr>
</table>
<hr/><a href="https://eclipse.org/jetty">Powered by Jetty:// 10.0.13</a><hr/>

</body>
</html>

        at hudson.plugins.swarm.SwarmClient.createSwarmAgent(SwarmClient.java:367)
        at hudson.plugins.swarm.Client.run(Client.java:193)
        at hudson.plugins.swarm.Client.main(Client.java:68)

May 02, 2023 3:11:59 PM hudson.plugins.swarm.Client run
INFO: Retrying in 10 seconds

What is also strange, some tokens have disappeared from the user. But they still work to connect to the Jenkins-Master. But with a newly generated token, the Agent connection no longer works.

Expected Results

Build-Agents can connect with Azure AD 340.vdef002cf6415 just as they can with version 313.v14b_f37ff114d.

Actual Results

When the Azure AD plugin is updated to version 340.vdef002cf6415, the connection of the Build-Agents no longer works. Here is the error message on the Build-Agent:

May 02, 2023 3:11:59 PM hudson.plugins.swarm.Client run
INFO: Attempting to connect to https://ci-i.sbb.ch/
May 02, 2023 3:11:59 PM hudson.plugins.swarm.SwarmClient getCsrfCrumb
SEVERE: Could not obtain CSRF crumb. Response code: 401
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 401 Unauthorized</title>
</head>
<body><h2>HTTP ERROR 401 Unauthorized</h2>
<table>
<tr><th>URI:</th><td>/crumbIssuer/api/xml</td></tr>
<tr><th>STATUS:</th><td>401</td></tr>
<tr><th>MESSAGE:</th><td>Unauthorized</td></tr>
<tr><th>SERVLET:</th><td>Stapler</td></tr>
</table>
<hr/><a href="https://eclipse.org/jetty">Powered by Jetty:// 10.0.13</a><hr/>

</body>
</html>

May 02, 2023 3:11:59 PM hudson.plugins.swarm.Client run
SEVERE: An error occurred
hudson.plugins.swarm.RetryException: Failed to create a Swarm agent on Jenkins. Response code: 401
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 401 Unauthorized</title>
</head>
<body><h2>HTTP ERROR 401 Unauthorized</h2>
<table>
<tr><th>URI:</th><td>/plugin/swarm/createSlave</td></tr>
<tr><th>STATUS:</th><td>401</td></tr>
<tr><th>MESSAGE:</th><td>Unauthorized</td></tr>
<tr><th>SERVLET:</th><td>Stapler</td></tr>
</table>
<hr/><a href="https://eclipse.org/jetty">Powered by Jetty:// 10.0.13</a><hr/>

</body>
</html>

        at hudson.plugins.swarm.SwarmClient.createSwarmAgent(SwarmClient.java:367)
        at hudson.plugins.swarm.Client.run(Client.java:193)
        at hudson.plugins.swarm.Client.main(Client.java:68)

May 02, 2023 3:11:59 PM hudson.plugins.swarm.Client run
INFO: Retrying in 10 seconds

What is also strange, some tokens have disappeared from the user. But they still work to connect to the Jenkins-Master. But with a newly generated token, the Agent connection no longer works.

Anything else?

No response

[timja]

What username are you using? It needs to be the object Id now and not the UPN

[michelgasser]

Yes, the agents connect to the Azur username which is also the Jenkins username. In our case the username is the email address:
java -jar ~/swarm-client.jar -username [email protected]

[michelgasser]

It's very spooky that the connection still works with old tokens but not with new ones.

[michelgasser]

Why do you make the change to Object ID? Jenkins works inconsistently this way.

With the object ID, the user is found via the Jenkins search bar:

With the object ID, the user is not found in the Azure AD matrix-based security:

The Object ID is also not very reader-friendly in the case of support.

[timja]

see #276 (comment)