[v2-10-test] Sort "opts" element in click option dictionary before ha…

…shing (apache#45156) Some impleemntation details of click 8.1.8 caused instability in configuration directories that are produced by click config - the instability is in sequence of generated "opts" list - the sequence of options in it is random. While it can be likely fixed in later versions, better is to protect from it and sort the opts list before hashing the dict. (cherry picked from commit 1796c40) Co-authored-by: Jarek Potiuk <[email protected]>
potiuk · Dec 22, 2024 · 2a58ada · 2a58ada
1 parent 721ff80
commit 2a58ada
Show file tree

Hide file tree

Showing 29 changed files with 56 additions and 30 deletions.
diff --git a/dev/breeze/doc/images/output_build-docs.txt b/dev/breeze/doc/images/output_build-docs.txt
@@ -1 +1 @@
-91166ce4114ea9c162c139d2aff15886
+16278718c7278a46f343ae5a64a9ae21
diff --git a/dev/breeze/doc/images/output_ci_fix-ownership.txt b/dev/breeze/doc/images/output_ci_fix-ownership.txt
@@ -1 +1 @@
-3e5a73533cc96045e72cb258783cfc96
+9decf6bb3eecbe566006f39f22cb331a
diff --git a/dev/breeze/doc/images/output_ci_free-space.txt b/dev/breeze/doc/images/output_ci_free-space.txt
@@ -1 +1 @@
-49af17b032039c05c41a7a8283f365cc
+21f9955c2f47b4524f1df73af8594d54
diff --git a/dev/breeze/doc/images/output_ci_resource-check.txt b/dev/breeze/doc/images/output_ci_resource-check.txt
@@ -1 +1 @@
-bfcca92f18a403ca630955074eb5e9ad
+858a3c3a7f47ab6b47364a9fff1c19ac
diff --git a/dev/breeze/doc/images/output_cleanup.txt b/dev/breeze/doc/images/output_cleanup.txt
@@ -1 +1 @@
-231de69d5f47ba29c883164e4575e310
+45d1ac665d85bff99d1ea396be00f892
diff --git a/dev/breeze/doc/images/output_compile-www-assets.txt b/dev/breeze/doc/images/output_compile-www-assets.txt
@@ -1 +1 @@
-2c65b08cbff1b5be5b5c55fcd18580a9
+5b57c7464757837e78ed9ad7e2b39d6b
diff --git a/dev/breeze/doc/images/output_down.txt b/dev/breeze/doc/images/output_down.txt
@@ -1 +1 @@
-65834d56aa23a727a3b5a5c6f8197400
+17e350cf4ddd966bc9aa142d5452317c
diff --git a/dev/breeze/doc/images/output_exec.txt b/dev/breeze/doc/images/output_exec.txt
@@ -1 +1 @@
-42bbd3c1659128b0341ae118c3482da2
+300c15ecdfffbb85926ab31fc775521a
diff --git a/dev/breeze/doc/images/output_generate-migration-file.txt b/dev/breeze/doc/images/output_generate-migration-file.txt
@@ -1 +1 @@
-747d68bcaed5decfa6da146eb9c74e64
+fe2f7beb040bd87d2ba9ec143c067711
diff --git a/dev/breeze/doc/images/output_k8s_setup-env.txt b/dev/breeze/doc/images/output_k8s_setup-env.txt
@@ -1 +1 @@
-a34e94744ca4e0592371fe55478c3d54
+519e87a2048c3f841196e43a09f183f6
diff --git a/dev/breeze/doc/images/output_release-management_clean-old-provider-artifacts.txt b/dev/breeze/doc/images/output_release-management_clean-old-provider-artifacts.txt
@@ -1 +1 @@
-16862ea6a616ef66a192424838062813
+085dfd3570b78ad1be8dd95f480be582
diff --git a/dev/breeze/doc/images/output_release-management_create-minor-branch.txt b/dev/breeze/doc/images/output_release-management_create-minor-branch.txt
@@ -1 +1 @@
-a3834afc4aa5d1e98002c9e9e7a9931d
+5b63e90b9597e5a47cc4fbfa32ab9fbe
diff --git a/dev/breeze/doc/images/output_release-management_generate-issue-content-core.txt b/dev/breeze/doc/images/output_release-management_generate-issue-content-core.txt
@@ -1 +1 @@
-15723131610a3be9a3093f346e72ed06
+00382e6b2a84a1edbf77eb5ed0a9df72
diff --git a/dev/breeze/doc/images/output_release-management_generate-issue-content-helm-chart.txt b/dev/breeze/doc/images/output_release-management_generate-issue-content-helm-chart.txt
@@ -1 +1 @@
-3e0e080fa8422853ed7df3967725b35f
+798f6631dad1b650e8a48fa5d51d73e6
diff --git a/dev/breeze/doc/images/output_release-management_prepare-airflow-package.txt b/dev/breeze/doc/images/output_release-management_prepare-airflow-package.txt
@@ -1 +1 @@
-d1eb314d4c9e949306d06f079a1463e5
+fe93d72f5d749f318b0a13294ea95d08
diff --git a/dev/breeze/doc/images/output_release-management_prepare-helm-chart-package.txt b/dev/breeze/doc/images/output_release-management_prepare-helm-chart-package.txt
@@ -1 +1 @@
-8c3be9f78286641f5657f890745ef936
+27169c83fcbcca907a0b32a5c4faeb48
diff --git a/dev/breeze/doc/images/output_release-management_prepare-helm-chart-tarball.txt b/dev/breeze/doc/images/output_release-management_prepare-helm-chart-tarball.txt
@@ -1 +1 @@
-d57dc8d296addbea3fd3285e2063c102
+15c3eb3be97998320dc95431840d8b3a
diff --git a/dev/breeze/doc/images/output_release-management_prepare-python-client.txt b/dev/breeze/doc/images/output_release-management_prepare-python-client.txt
@@ -1 +1 @@
-aa5da16329704d537cf3c6d6f38ab8d9
+2a24df4fb60b12ac81b2fcbe151c1f8d
diff --git a/dev/breeze/doc/images/output_release-management_start-release.txt b/dev/breeze/doc/images/output_release-management_start-release.txt
@@ -1 +1 @@
-419f48f6a4ff4457cb9de7ff496aebbe
+495a5e3d024bc6039290d9f4528a3ca4
diff --git a/dev/breeze/doc/images/output_release-management_tag-providers.txt b/dev/breeze/doc/images/output_release-management_tag-providers.txt
@@ -1 +1 @@
-7c32e47c0359fa185ab30ed5fc4c91cf
+403dae670f839e37c83baa15ac65af9a
diff --git a/dev/breeze/doc/images/output_release-management_update-constraints.txt b/dev/breeze/doc/images/output_release-management_update-constraints.txt
@@ -1 +1 @@
-02ec4b119150e3fdbac52026e94820ef
+cd4df9af29227f6de406050fbb12dd3b
diff --git a/dev/breeze/doc/images/output_sbom_build-all-airflow-images.txt b/dev/breeze/doc/images/output_sbom_build-all-airflow-images.txt
@@ -1 +1 @@
-fbf15f8f21cdda9f815a801d9e266b61
+24b1be5f59a0d7db2bac65edbdb693a1
diff --git a/dev/breeze/doc/images/output_sbom_update-sbom-information.txt b/dev/breeze/doc/images/output_sbom_update-sbom-information.txt
@@ -1 +1 @@
-116a44d6ee103f8b57ce56a3e774c70b
+60ea5de9051747f494fe1d40310fba85
diff --git a/dev/breeze/doc/images/output_setup.txt b/dev/breeze/doc/images/output_setup.txt
@@ -1 +1 @@
-ea2ff499fd3c25ef89abaab313d5d04a
+ba00ab3fb2ed5a777684878c28b3ce65
diff --git a/dev/breeze/doc/images/output_setup_autocomplete.txt b/dev/breeze/doc/images/output_setup_autocomplete.txt
@@ -1 +1 @@
-ec3b4541a478afe5cb86a6f1c48f50f5
+3da8dc88411f34d3ed52c155df744c22
diff --git a/dev/breeze/doc/images/output_setup_self-upgrade.txt b/dev/breeze/doc/images/output_setup_self-upgrade.txt
@@ -1 +1 @@
-4af905a147fcd6670a0e33d3d369a94b
+9ad9de5cc99ce2133e14d7dc15c1042a
diff --git a/dev/breeze/doc/images/output_setup_synchronize-local-mounts.txt b/dev/breeze/doc/images/output_setup_synchronize-local-mounts.txt
@@ -1 +1 @@
-252c8ee48ca57e6539064544cbd63e58
+85797323723a480a895251c1b6db9356
diff --git a/dev/breeze/doc/images/output_setup_version.txt b/dev/breeze/doc/images/output_setup_version.txt
@@ -1 +1 @@
-be116d90a21c2afe01087f7609774e1e
+b41f292ef2af6d7988bedc6effd73263
diff --git a/dev/breeze/src/airflow_breeze/commands/setup_commands.py b/dev/breeze/src/airflow_breeze/commands/setup_commands.py
@@ -296,10 +296,36 @@ def dedent_help(dictionary: dict[str, Any]) -> None:
             dictionary[key] = textwrap.dedent(value)
 
 
-def dict_hash(dictionary: dict[str, Any], dedent_help_strings: bool = True) -> str:
-    """MD5 hash of a dictionary. Sorted and dumped via json to account for random sequence)"""
+def recursively_sort_opts(object: dict[str, Any] | list[Any]) -> None:
+    if isinstance(object, dict):
+        for key in object:
+            if key == "opts" and isinstance(object[key], list):
+                object[key] = sorted(object[key])
+            elif isinstance(object[key], dict):
+                recursively_sort_opts(object[key])
+            elif isinstance(object[key], list):
+                recursively_sort_opts(object[key])
+    elif isinstance(object, list):
+        for element in object:
+            recursively_sort_opts(element)
+
+
+def dict_hash(dictionary: dict[str, Any], dedent_help_strings: bool = True, sort_opts: bool = True) -> str:
+    """
+    MD5 hash of a dictionary of configuration for click.
+
+    Sorted and dumped via json to account for random sequence of keys in the dictionary. Also it
+    implements a few corrections to the dict because click does not always keep the same sorting order in
+    options or produced differently indented help strings.
+
+    :param dictionary: dictionary to hash
+    :param dedent_help_strings: whether to dedent help strings before hashing
+    :param sort_opts: whether to sort options before hashing
+    """
     if dedent_help_strings:
         dedent_help(dictionary)
+    if sort_opts:
+        recursively_sort_opts(dictionary)
     # noinspection InsecureHash
     dhash = hashlib.md5()
     try:
Original file line number	Diff line number	Diff line change
		@@ -1 +1 @@
		91166ce4114ea9c162c139d2aff15886
		16278718c7278a46f343ae5a64a9ae21
Original file line number	Diff line number	Diff line change
		@@ -1 +1 @@
		3e5a73533cc96045e72cb258783cfc96
		9decf6bb3eecbe566006f39f22cb331a
Original file line number	Diff line number	Diff line change
		@@ -1 +1 @@
		49af17b032039c05c41a7a8283f365cc
		21f9955c2f47b4524f1df73af8594d54
Original file line number	Diff line number	Diff line change
		@@ -1 +1 @@
		bfcca92f18a403ca630955074eb5e9ad
		858a3c3a7f47ab6b47364a9fff1c19ac
Original file line number	Diff line number	Diff line change
		@@ -1 +1 @@
		231de69d5f47ba29c883164e4575e310
		45d1ac665d85bff99d1ea396be00f892
Original file line number	Diff line number	Diff line change
		@@ -1 +1 @@
		2c65b08cbff1b5be5b5c55fcd18580a9
		5b57c7464757837e78ed9ad7e2b39d6b
Original file line number	Diff line number	Diff line change
		@@ -1 +1 @@
		65834d56aa23a727a3b5a5c6f8197400
		17e350cf4ddd966bc9aa142d5452317c
Original file line number	Diff line number	Diff line change
		@@ -1 +1 @@
		42bbd3c1659128b0341ae118c3482da2
		300c15ecdfffbb85926ab31fc775521a
Original file line number	Diff line number	Diff line change
		@@ -1 +1 @@
		747d68bcaed5decfa6da146eb9c74e64
		fe2f7beb040bd87d2ba9ec143c067711
Original file line number	Diff line number	Diff line change
		@@ -1 +1 @@
		a34e94744ca4e0592371fe55478c3d54
		519e87a2048c3f841196e43a09f183f6