chore(deps): bump selenium 4.14.0+

[gnought]

SUMMARY

This is a proper fix for the PR #24538.

However we have to think about how the WEBDRIVER_CONFIGURATION should be. The selenium 4.10+ only allows options and services to be passed to the WebDriver constructor.
https://github.com/SeleniumHQ/selenium/blob/c14d9678990942b93cb421c5567d0da7fb29c7bd/py/selenium/webdriver/firefox/webdriver.py#L41-L46

superset/superset/utils/webdriver.py

Lines 276 to 279 in a698587

	kwargs.update(current_app.config["WEBDRIVER_CONFIGURATION"])
	logger.debug("Init selenium driver")
	return driver_class(**kwargs)

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

• Has associated issue: Fixes update superset 3.1.0/3.1.1 dependency "selenium 3.141.0" #26992
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[john-bodley]

Thanks @gnought for the change. As your point out with Selenium 4.10+ the WebDriver signature changed which means that existing WEBDRIVER_CONFIGURATION definitions (including the default) could fail and thus this would be deemed a breaking change. It seems prudent to hold off this change until when the breaking window opens for Superset 4.0.

[john-bodley]

@gnought would you mind providing context for this change? Note irrespective of what the default values are this would still be deemed a breaking change given that the customizable Superset configurations are deemed a public interface.

CC @michael-s-molina for context regarding our recent discussion on breaking changes.

[gnought]

@john-bodley

In 4.9.0, service_log_path is one of parameters in WebDriver constructor.
https://github.com/SeleniumHQ/selenium/blob/bc7e0c7341fdde78cdd80ce572002a7866fd1769/py/selenium/webdriver/firefox/webdriver.py#L90-L110

However it is no longer in 4.10.0+.
https://github.com/SeleniumHQ/selenium/blob/c14d9678990942b93cb421c5567d0da7fb29c7bd/py/selenium/webdriver/firefox/webdriver.py#L50-L53

The following commit will support 4.10.0. It also opens for users to refine Webdrver Options and Service.
For sure, this will be a breaking change if users defining WEBDRIVER_CONFIGURATION themselves using 4.9.0 WebDriver defintion.

[john-bodley]

@gnought thanks for refactoring, however I sense this change isn't suffice. The TL;DR is either:

1. We hold of this change until 4.0 when the breaking window opens, or
2. We need to mutate existing user defined WEBDRIVER_CONFIGURATION to conform to the new options/service structure.

For example in the current form if an implementation has defined the following,

WEBDRIVER_CONFIGURATION = {"service_log_path": "/dev/null"}

then said configuration would be lost if this PR was merged because the service_log_path key isn't handled. It would be good to look at the Selenium changelog and check the codebase to see whether they have a mechanism for migrating from the old to new WebDriver class.

[gnought]

@john-bodley, modified code a bit for backward compatibility. Hope it makes us feel safe.

[nigzak]

FYI: if the change is made it should be updated to 4.14.x (or newer) to fix a CVE finding (refer linked #26992)

[rusackas]

Unfortunately, I think this missed the proposal window for breaking changes, and will have to go into 5.0. I'l add it to that project board now. I'm also not sure where we stand with the potential migration to Playwright (thus tagging @kgabryje here) and if that will become the standard by 5.0. If we want to proceed with this upgrade, which seems sensible, this PR will just need a rebase and bump to 4.14.x or greater by the time that breaking window re-opens.

[rusackas]

This has also been updated to auto-close #26992 when it does become mergeable.

[nigzak]

Hi @rusackas
the pull request here seems to update it to V4.10.x - only have in mind it should be updated to >= V4.14.x (newer as it is in this pull request)

[mistercrunch]

FYI I fixed the merge conflicts, can we remove the hold on this one?

In any case if the interface changed we should add a line in UPDATING.md as part of this PR

[michael-s-molina]

FYI I fixed the merge conflicts, can we remove the hold on this one?

@mistercrunch We can't remove the hold label if the PR contains a breaking change. That will only be possible during a breaking window.

[villebro]

@gnought we want to push this through for Superset 5.0 (we're approaching the breaking window during which this can be merged). Would you be able to rebase this PR and fix failing CI checks?

[codecov]

Codecov Report

Attention: Patch coverage is 82.60870% with 4 lines in your changes missing coverage. Please review.

Project coverage is 83.73%. Comparing base (76d897e) to head (7cc517d).
Report is 1170 commits behind head on master.

Files with missing lines	Patch %	Lines
superset/utils/webdriver.py	81.81%	4 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #25933       +/-   ##
===========================================
+ Coverage   60.48%   83.73%   +23.24%     
===========================================
  Files        1931      537     -1394     
  Lines       76236    39077    -37159     
  Branches     8568        0     -8568     
===========================================
- Hits        46114    32721    -13393     
+ Misses      28017     6356    -21661     
+ Partials     2105        0     -2105     

Flag	Coverage Δ
hive	48.73% <21.73%> (-0.44%)	⬇️
javascript	?
mysql	76.44% <82.60%> (?)
postgres	76.53% <82.60%> (?)
presto	53.25% <21.73%> (-0.56%)	⬇️
python	83.73% <82.60%> (+20.24%)	⬆️
sqlite	75.99% <82.60%> (?)
unit	60.85% <21.73%> (+3.23%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[gnought]

@gnought we want to push this through for Superset 5.0 (we're approaching the breaking window during which this can be merged). Would you be able to rebase this PR and fix failing CI checks?

@villebro done.