Merge pull request #399 from dunglas/feat/native-driver-binaries

feat: remove ChromeDriver and geckodriver from the repo, use native installations instead

Author: dunglas

.gitignore

@@ -5,3 +5,5 @@
 /composer.lock
 /phpunit.xml
 /vendor/
+/drivers/
+/screen.png

README.md

@@ -15,8 +15,8 @@ all the features you need to test your apps. It will sound familiar if you have
 as the API is exactly the same!
 Keep in mind that Panther can be used in every PHP project, as it is a standalone library.
 
-Panther automatically finds your local installation of Chrome or Firefox and launches them (thanks to [ChromeDriver](https://sites.google.com/a/chromium.org/chromedriver/) and [GeckoDriver](https://github.com/mozilla/geckodriver)),
-so you don't need to install anything on your computer, neither Selenium server nor any other obscure driver.
+Panther automatically finds your local installation of Chrome or Firefox and launches them,
+so you don't need to install anything else on your computer, a Selenium server is not needed!
 
 In test mode, Panther automatically starts your application using [the PHP built-in web-server](http://php.net/manual/en/features.commandline.webserver.php).
 You can focus on writing your tests or web-scraping scenario and Panther will take care of everything else.
@@ -27,28 +27,57 @@ Unlike testing and web scraping libraries you're used to, Panther:
 
 * executes the JavaScript code contained in webpages
 * supports everything that Chrome (or Firefox) implements
-* allows screenshots taking
+* allows taking screenshots
 * can wait for asynchronously loaded elements to show up
 * lets you run your own JS code or XPath queries in the context of the loaded page
 * supports custom [Selenium server](https://www.seleniumhq.org) installations
 * supports remote browser testing services including [SauceLabs](https://saucelabs.com/) and [BrowserStack](https://www.browserstack.com/)
 
 ## Documentation
 
-### Install
+### Installing Panther
 
 Use [Composer](https://getcomposer.org/) to install Panther in your project. You may want to use the `--dev` flag if you want to use Panther for testing only and not for web scraping in a production environment:
 
     composer req symfony/panther
 
     composer req --dev symfony/panther
 
-**Warning:** On \*nix systems, the `unzip` command must be installed or you will encounter an error similar to `RuntimeException: sh: 1: exec: /app/vendor/symfony/panther/src/ProcessManager/../../chromedriver-bin/chromedriver_linux64: Permission denied` (or `chromedriver_linux64: not found`).
+**Warning:** On \*nix systems, the `unzip` command must be installed, or you will encounter an error similar to `RuntimeException: sh: 1: exec: /app/vendor/symfony/panther/src/ProcessManager/../../chromedriver-bin/chromedriver_linux64: Permission denied` (or `chromedriver_linux64: not found`).
 The underlying reason is that PHP's `ZipArchive` doesn't preserve UNIX executable permissions.
 
+### Installing ChromeDriver and geckodriver
+
+Panther uses the WebDriver protocol to control the browser used to crawl websites.
+
+On all systems, you can use `dbrekelmans/browser-driver-installer` to install ChromeDriver and geckodriver locally:
+
+    composer require --dev dbrekelmans/bdi
+    vendor/bin/bdi detect drivers
+
+Panther will detect and use automatically drivers stored in the `drivers/` directory.
+
+Alternatively, you can use the package manager of your operating system to install them.
+
+On Ubuntu, run:
+
+    apt-get install chromium-chromedriver firefox-geckodriver
+
+On Mac, using [Homebrew](https://brew.sh):
+
+    brew install chromedriver geckodriver
+
+On Windows, using [chocolatey](https://chocolatey.org):
+
+    choco install chromedriver selenium-gecko-driver
+
+Finally, you can download manually [ChromeDriver](https://sites.google.com/a/chromium.org/chromedriver/) (for Chromium or Chrome)
+and [GeckoDriver](https://github.com/mozilla/geckodriver) (for Firefox) and put them anywhere in your `PATH`
+or in the `drivers/` directory of your project.
+
 #### Registering the PHPUnit Extension
 
-If you intend to use Panther to test your application, we strongly recommended to register the Panther PHPUnit extension.
+If you intend to use Panther to test your application, we strongly recommend registering the Panther PHPUnit extension.
 While not strictly mandatory, this extension dramatically improves the testing experience by boosting the performance and
 allowing to use the [interactive debugging mode](#interactive-mode).
 
@@ -81,11 +110,13 @@ This listener will start the web server on demand like previously, but it will s
 ```php
 <?php
 
+use Symfony\Component\Panther\Client;
+
 require __DIR__.'/vendor/autoload.php'; // Composer's autoloader
 
-$client = \Symfony\Component\Panther\Client::createChromeClient();
+$client = Client::createChromeClient();
 // Or, if you care about the open web and prefer to use Firefox
-$client = \Symfony\Component\Panther\Client::createFirefoxClient();
+$client = Client::createFirefoxClient();
 
 $client->request('GET', 'https://api-platform.com'); // Yes, this website is 100% written in JavaScript
 $client->clickLink('Get started');
@@ -103,7 +134,7 @@ $client->takeScreenshot('screen.png'); // Yeah, screenshot!
 
 The `PantherTestCase` class allows you to easily write E2E tests. It automatically starts your app using the built-in PHP
 web server and let you crawl it using Panther.
-To provide all of the testing tools you're used to, it extends [PHPUnit](https://phpunit.de/)'s `TestCase`.
+To provide all the testing tools you're used to, it extends [PHPUnit](https://phpunit.de/)'s `TestCase`.
 
 If you are testing a Symfony application, `PantherTestCase` automatically extends [the `WebTestCase` class](https://symfony.com/doc/current/testing.html#functional-tests).
 It means you can easily create functional tests, which can directly execute the kernel of your application and access all
@@ -243,11 +274,11 @@ class ChatTest extends PantherTestCase
 
 ### Checking the State of the WebDriver Connection
 
-Use the `Client::ping()` method to check if the WebDriver connection is still active (useful for long running tasks).
+Use the `Client::ping()` method to check if the WebDriver connection is still active (useful for long-running tasks).
 
 ## Additional Documentation
 
-Since Panther implements the API of popular libraries, it already has extensive documentation:
+Since Panther implements the API of popular libraries, it already has an extensive documentation:
 
 * For the `Client` class, read [the BrowserKit documentation](https://symfony.com/doc/current/components/browser_kit.html)
 * For the `Crawler` class, read [the DomCrawler documentation](https://symfony.com/doc/current/components/dom_crawler.html)
@@ -257,7 +288,7 @@ Since Panther implements the API of popular libraries, it already has extensive
 
 The following environment variables can be set to change some Panther's behaviour:
 
-* `PANTHER_NO_HEADLESS`: to disable browser's headless mode (will display the testing window, useful to debug)
+* `PANTHER_NO_HEADLESS`: to disable the browser's headless mode (will display the testing window, useful to debug)
 * `PANTHER_WEB_SERVER_DIR`: to change the project's document root (default to `./public/`, relative paths **must start** by `./`)
 * `PANTHER_WEB_SERVER_PORT`: to change the web server's port (default to `9080`)
 * `PANTHER_WEB_SERVER_ROUTER`:  to use a web server router script which is run at the start of each HTTP request
@@ -279,13 +310,11 @@ $client = self::createPantherClient([
 #### Chrome-specific Environment Variables
 
 * `PANTHER_NO_SANDBOX`: to disable [Chrome's sandboxing](https://chromium.googlesource.com/chromium/src/+/b4730a0c2773d8f6728946013eb812c6d3975bec/docs/design/sandbox.md) (unsafe, but allows to use Panther in containers)
-* `PANTHER_CHROME_DRIVER_BINARY`: to use another `chromedriver` binary, instead of relying on the ones already provided by Panther
 * `PANTHER_CHROME_ARGUMENTS`: to customize Chrome arguments. You need to set `PANTHER_NO_HEADLESS` to fully customize.
 * `PANTHER_CHROME_BINARY`: to use another `google-chrome` binary
 
 #### Firefox-specific Environment Variables
 
-* `PANTHER_GECKO_DRIVER_BINARY`: to use another `geckodriver` binary, instead of relying on the ones already provided by Panther
 * `PANTHER_FIREFOX_ARGUMENTS`: to customize Firefox arguments. You need to set `PANTHER_NO_HEADLESS` to fully customize.
 * `PANTHER_FIREFOX_BINARY`: to use another `firefox` binary
 
@@ -344,29 +373,29 @@ To force Chrome to accept invalid and self-signed certificates, set the followin
 
 ### Docker Integration
 
-Here is a minimal Docker image that can run Panther:
+Here is a minimal Docker image that can run Panther with both Chrome and Firefox:
 
-```
-FROM php:latest
+```Dockerfile
+FROM php:alpine
 
-RUN apt-get update && apt-get install -y libzip-dev zlib1g-dev chromium && docker-php-ext-install zip
+# Chromium and ChromeDriver
 ENV PANTHER_NO_SANDBOX 1
 # Not mandatory, but recommended
 ENV PANTHER_CHROME_ARGUMENTS='--disable-dev-shm-usage'
+RUN apk add --no-cache chromium chromium-chromedriver
+
+# Firefox and GeckoDriver (optional)
+ARG GECKODRIVER_VERSION=0.28.0
+RUN apk add --no-cache firefox libzip-dev; \
+    docker-php-ext-install zip
+RUN wget -q https://github.com/mozilla/geckodriver/releases/download/v$GECKODRIVER_VERSION/geckodriver-v$GECKODRIVER_VERSION-linux64.tar.gz; \
+    tar -zxf geckodriver-v$GECKODRIVER_VERSION-linux64.tar.gz -C /usr/bin; \
+    rm geckodriver-v$GECKODRIVER_VERSION-linux64.tar.gz
 ```
 
 Build it with `docker build . -t myproject`
 Run it with `docker run -it -v "$PWD":/srv/myproject -w /srv/myproject myproject bin/phpunit`
 
-If you are using **Alpine Linux**, you may need to use another `chromedriver` binary.
-
-```
-RUN apk add --no-cache \
-        chromium \
-        chromium-chromedriver
-ENV PANTHER_CHROME_DRIVER_BINARY /usr/lib/chromium/chromedriver
-```
-
 ### GitHub Actions Integration
 
 Panther works out of the box with [GitHub Actions](https://help.github.com/en/actions).
@@ -386,7 +415,7 @@ jobs:
       - uses: actions/checkout@v2
 
       - name: Install dependencies
-        run: composer install -q --no-ansi --no-interaction --no-scripts --no-suggest --no-progress --prefer-dist
+        run: composer install -q --no-ansi --no-interaction --no-scripts --no-progress --prefer-dist
 
       - name: Run test suite
         run: vendor/bin/phpunit
@@ -405,53 +434,34 @@ addons:
   firefox: latest
 
 php:
-  - 7.4
+  - 8.0
 
 script:
-  - phpunit
+  - vendor/bin/phpunit
 ```
 
 ### Gitlab CI Integration
 
 Here is a minimal `.gitlab-ci.yml` file to run Panther tests with [Gitlab CI](https://docs.gitlab.com/ee/ci/):
 
 ```yaml
-image: ubuntu:bionic
-
-services:
-  - postgres:11
-
-variables:
-  POSTGRES_PASSWORD: root
-  POSTGRES_USER: root
-  POSTGRES_DB: db
+image: ubuntu
 
 before_script:
-    - apt-get update
-    - apt-get install software-properties-common -y
-    - ln -sf /usr/share/zoneinfo/Asia/Tokyo /etc/localtime
-    - apt-get install curl wget php php-cli php7.2 php7.2-common php7.2-curl php7.2-pgsql php7.2-mysql php7.2-intl php7.2-gd php7.2-xml php7.2-opcache php7.2-mbstring php7.2-zip libfontconfig1 fontconfig libxrender-dev libfreetype6 libxrender1 zlib1g-dev xvfb chromium-browser chromium-chromedriver -y -qq
-    - export PANTHER_CHROME_DRIVER_BINARY="/usr/lib/chromium-browser/chromedriver"
-    - export PANTHER_NO_SANDBOX=1
-    - export PANTHER_WEB_SERVER_PORT=9080
-    - php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
-    - php composer-setup.php --install-dir=/usr/local/bin --filename=composer
-    - php -r "unlink('composer-setup.php');"
-    - composer
-    - chromedriver --version
-    - php -v
-    - php -m
-    - composer install --ignore-platform-reqs
-    - bin/console doctrine:schema:update --force
-    - bin/console doctrine:schema:validate
-    - bin/console doctrine:fixtures:load
-
-stages:
-- test
+  - apt-get update
+  - apt-get install software-properties-common -y
+  - ln -sf /usr/share/zoneinfo/Asia/Tokyo /etc/localtime
+  - apt-get install curl wget php php-cli php7.4 php7.4-common php7.4-curl php7.4-intl php7.4-xml php7.4-opcache php7.4-mbstring php7.4-zip libfontconfig1 fontconfig libxrender-dev libfreetype6 libxrender1 zlib1g-dev xvfb chromium-chromedriver firefox-geckodriver -y -qq
+  - export PANTHER_NO_SANDBOX=1
+  - export PANTHER_WEB_SERVER_PORT=9080
+  - php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
+  - php composer-setup.php --install-dir=/usr/local/bin --filename=composer
+  - php -r "unlink('composer-setup.php');"
+  - composer install
 
 test:
   script:
-    - vendor/bin/simple-phpunit tests/Controller/E2eTest.php
+    - vendor/bin/phpunit
 ```
 
 ### AppVeyor Integration
@@ -469,9 +479,9 @@ cache:
 
 install:
   - ps: Set-Service wuauserv -StartupType Manual
-  - cinst -y php composer googlechrome
+  - cinst -y php composer googlechrome chromedriver firfox selenium-gecko-driver
   - refreshenv
-  - cd c:\tools\php74
+  - cd c:\tools\php80
   - copy php.ini-production php.ini /Y
   - echo date.timezone="UTC" >> php.ini
   - echo extension_dir=ext >> php.ini
@@ -489,7 +499,10 @@ test_script:
 
 ### Usage with Other Testing Tools
 
-If you want to use Panther with other testing tools like [LiipFunctionalTestBundle](https://github.com/liip/LiipFunctionalTestBundle) or if you just need to use a different base class, Panther has got you covered. It provides you with the `Symfony\Component\Panther\PantherTestCaseTrait` and you can use it to enhance your existing test-infrastructure with some Panther awesomeness:
+If you want to use Panther with other testing tools like [LiipFunctionalTestBundle](https://github.com/liip/LiipFunctionalTestBundle)
+or if you just need to use a different base class, Panther has got you covered.
+It provides you with the `Symfony\Component\Panther\PantherTestCaseTrait` and you can use it to enhance your existing
+test-infrastructure with some Panther awesomeness:
 
 ```php
 <?php
@@ -534,4 +547,5 @@ If you like this software, help save the (real) panthers by [donating to the Pan
 
 Created by [Kévin Dunglas](https://dunglas.fr). Sponsored by [Les-Tilleuls.coop](https://les-tilleuls.coop).
 
-Panther is built on top of [PHP WebDriver](https://github.com/php-webdriver/php-webdriver) and [several other FOSS libraries](https://symfony.com/blog/introducing-symfony-panther-a-browser-testing-and-web-scrapping-library-for-php#thank-you-open-source). It has been inspired by [Nightwatch.js](http://nightwatchjs.org/), a WebDriver-based testing tool for JavaScript.
+Panther is built on top of [PHP WebDriver](https://github.com/php-webdriver/php-webdriver) and [several other FOSS libraries](https://symfony.com/blog/introducing-symfony-panther-a-browser-testing-and-web-scrapping-library-for-php#thank-you-open-source).
+It has been inspired by [Nightwatch.js](http://nightwatchjs.org/), a WebDriver-based testing tool for JavaScript.

examples/basic.php

@@ -11,16 +11,21 @@
 
 declare(strict_types=1);
 
+use Symfony\Component\Panther\Client;
+
 require __DIR__.'/../vendor/autoload.php'; // Composer's autoloader
 
-$client = \Symfony\Component\Panther\Client::createChromeClient();
-$crawler = $client->request('GET', 'http://api-platform.com'); // Yes, this website is 100% in JavaScript
+$client = Client::createChromeClient();
+// Or, if you care about the open web and prefer to use Firefox
+//$client = Client::createFirefoxClient();
 
-$link = $crawler->selectLink('Support')->link();
-$crawler = $client->click($link);
+$client->request('GET', 'https://api-platform.com'); // Yes, this website is 100% written in JavaScript
+$client->clickLink('Get started');
 
-// Wait for an element to be rendered
-$client->waitFor('.support');
+// Wait for an element to be present in the DOM (even if hidden)
+$crawler = $client->waitFor('#installing-the-framework');
+// Alternatively, wait for an element to be visible
+$crawler = $client->waitForVisibility('#installing-the-framework');
 
-echo $crawler->filter('.support')->text();
+echo $crawler->filter('#installing-the-framework')->text();
 $client->takeScreenshot('screen.png'); // Yeah, screenshot!

src/ProcessManager/ChromeManager.php

@@ -17,6 +17,7 @@
 use Facebook\WebDriver\Remote\DesiredCapabilities;
 use Facebook\WebDriver\Remote\RemoteWebDriver;
 use Facebook\WebDriver\WebDriver;
+use Symfony\Component\Process\ExecutableFinder;
 use Symfony\Component\Process\Process;
 
 /**
@@ -30,6 +31,9 @@ final class ChromeManager implements BrowserManagerInterface
     private $arguments;
     private $options;
 
+    /**
+     * @throws \RuntimeException
+     */
     public function __construct(?string $chromeDriverBinary = null, ?array $arguments = null, array $options = [])
     {
         $this->options = array_merge($this->getDefaultOptions(), $options);
@@ -72,20 +76,22 @@ public function quit(): void
         $this->process->stop();
     }
 
+    /**
+     * @throws \RuntimeException
+     */
     private function findChromeDriverBinary(): string
     {
         if ($binary = $_SERVER['PANTHER_CHROME_DRIVER_BINARY'] ?? null) {
+            @trigger_error('The "PANTHER_CHROME_DRIVER_BINARY" environment variable is deprecated since Panther 0.9, add ChromeDriver to your PATH instead.', E_USER_DEPRECATED);
+
             return $binary;
         }
 
-        switch (PHP_OS_FAMILY) {
-            case 'Windows':
-                return __DIR__.'/../../chromedriver-bin/chromedriver.exe';
-            case 'Darwin':
-                return __DIR__.'/../../chromedriver-bin/chromedriver_mac64';
-            default:
-                return __DIR__.'/../../chromedriver-bin/chromedriver_linux64';
+        if ($binary = (new ExecutableFinder())->find('chromedriver', null, ['./drivers'])) {
+            return $binary;
         }
+
+        throw new \RuntimeException('"chromedriver" binary not found. Install it using the package manager of your operating system or by running "composer require --dev dbrekelmans/bdi && vendor/bin/bdi detect drivers".');
     }
 
     private function getDefaultArguments(): array