Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Comparing core and lcache #3

Open
wants to merge 89 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
50bf868
Initial port.
davidstrauss Sep 6, 2016
1b81f31
Remove database constraint
stevector Sep 9, 2016
d742d0b
Starting on CircleCI integration
stevector Sep 9, 2016
1177aa6
Update circle.yml
stevector Sep 9, 2016
f6b3670
Update circle.yml
stevector Sep 9, 2016
a67e304
Update circle.yml
stevector Sep 9, 2016
831ef42
executable
stevector Sep 9, 2016
889b16d
Merge branch 'circle-ci' of github.com:lcache/drupal-8 into circle-ci
stevector Sep 9, 2016
e4bbab7
Update circle.yml
stevector Sep 9, 2016
e0fb67e
Adding behat dir
stevector Sep 9, 2016
151b4dd
Adding behat to composer
stevector Sep 9, 2016
ad4debe
adding gitignore
stevector Sep 9, 2016
7eb48f4
Core vs. LCache
stevector Sep 9, 2016
324b20d
changing script names
stevector Sep 9, 2016
827e948
no setting of php version
stevector Sep 9, 2016
45bf368
Going back to php version setting
stevector Sep 9, 2016
ca1a710
removing unused context
stevector Sep 9, 2016
b1fc9c4
Directory changing
stevector Sep 9, 2016
0adbaa5
Test saving a node visiting home page
stevector Sep 9, 2016
9c03a20
dir fix
stevector Sep 9, 2016
a583bea
[ci skip]
stevector Sep 9, 2016
d4f934e
bumping column size. Not sure if this is involved in the problem.
stevector Sep 9, 2016
601173d
Update lcache.install
stevector Sep 10, 2016
d0a9657
Use LCache v0.3 series with machine learning.
davidstrauss Sep 12, 2016
93fbf78
Cache tags should DELETE CASCADE on event deletion.
davidstrauss Sep 12, 2016
823dfa1
Adding invalidator
stevector Sep 12, 2016
4e36fef
Merge branch 'circle-ci' of github.com:lcache/drupal-8 into circle-ci
stevector Sep 12, 2016
a79cc9a
Cleaning up services
stevector Sep 13, 2016
d085f7b
Back to 255 column size
stevector Sep 13, 2016
cd9e622
Whitespace
stevector Sep 13, 2016
d7df578
Merge branch 'circle-ci' of github.com:lcache/drupal-8 into circle-ci
stevector Sep 13, 2016
037dfc3
Cleaning up behat file
stevector Sep 13, 2016
b10b2dc
Performance comparison
stevector Sep 13, 2016
39b7d24
35 runs with more debugging
stevector Sep 13, 2016
1245556
set username and pass
stevector Sep 13, 2016
de4797e
bumping lcache version
stevector Sep 20, 2016
bc50465
set terminus number
stevector Sep 21, 2016
ef4c8d4
table fixes
stevector Sep 21, 2016
c3911a6
updating debugging output
stevector Sep 21, 2016
0220364
100 test loops
stevector Sep 21, 2016
447ed27
2056 address length
stevector Sep 21, 2016
c48189a
fewer test runs
stevector Sep 21, 2016
9cc75c1
Update lcache.install
stevector Sep 21, 2016
2aef1e7
Increasing test runs
stevector Sep 21, 2016
1b41d81
more repetitions of test runs
stevector Sep 21, 2016
239a5d2
open all links
stevector Sep 21, 2016
a595cb8
LCache instead of ChainedFast
stevector Sep 21, 2016
ffd2453
LCache instead of DB for render and dynamic_page_cache
stevector Sep 21, 2016
b9d36a1
LCache for menu and toolbar
stevector Sep 21, 2016
70c8cbe
LCache for entity bin
stevector Sep 21, 2016
13f58e0
LCache for data bin
stevector Sep 21, 2016
12044d8
Taking data, menu and toolbar out of LCache
stevector Sep 22, 2016
f940a3b
50 runs each
stevector Sep 22, 2016
0446e27
render bin only
stevector Sep 22, 2016
f9ce731
dynamic_page_cache bin only
stevector Sep 22, 2016
aac41be
toolbar only
stevector Sep 22, 2016
c877507
menu bin only
stevector Sep 22, 2016
799c4dd
entity bin only
stevector Sep 22, 2016
d8b0b50
data bin only
stevector Sep 22, 2016
43024a3
Update BackendUnitTest.php
stevector Sep 22, 2016
df8b29f
Merging from d.o
stevector Sep 23, 2016
845c8f0
deleting lcache.cache.inc
stevector Sep 23, 2016
1622216
cleaning up merge
stevector Sep 23, 2016
1cdc2e6
cleaning up merge
stevector Sep 23, 2016
acb204f
cleaning up merge
stevector Sep 23, 2016
75e35cb
cleaning up merge
stevector Sep 23, 2016
051ed61
cleaning up merge
stevector Sep 23, 2016
d245b7e
Merging 8.x-1.x
stevector Oct 21, 2016
0324988
Update circle.yml
stevector Oct 21, 2016
44720dd
Autoloading
stevector Oct 21, 2016
2f81b36
Merge branch 'circle-ci--perf-comparison' of github.com:lcache/drupal-8
stevector Oct 21, 2016
907ff40
Bootstrap
stevector Oct 21, 2016
7b03fb3
4 node creations
stevector Oct 21, 2016
2821400
config bin
stevector Oct 21, 2016
45b36b9
discovery
stevector Oct 21, 2016
d2307bf
render bin
stevector Oct 21, 2016
7d3f695
toolbar bin
stevector Oct 21, 2016
66a565f
menu bin
stevector Oct 21, 2016
c262225
Entity bin
stevector Oct 21, 2016
6c81a0d
Whitespace
stevector Oct 21, 2016
74c8749
Adding 100 limit
stevector Oct 21, 2016
9aed420
Merge branch 'circle-ci--perf-comparison' of github.com:lcache/drupal…
stevector Oct 21, 2016
ddf7d02
Commenting out logging of urls
stevector Oct 21, 2016
98742a5
render bin
stevector Oct 21, 2016
39ceebc
menu bin
stevector Oct 21, 2016
b41533a
config
stevector Oct 21, 2016
5680d3c
render bin
stevector Oct 21, 2016
50928b8
dynamic_page_cache bin
stevector Oct 21, 2016
fe8c44d
Update core-vs-lcache.sh
stevector Oct 21, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
composer.lock
/drush/
/vendor/
.DS_Store
84 changes: 29 additions & 55 deletions circle.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,67 +4,41 @@ machine:
# https://circleci.com/docs/environment#php
version: 5.6.14
environment:
# DB config. Using default CircleCI's database.
DB_NAME: "circle_test"
DB_USERNAME: "ubuntu"
DB_PASSWORD: ""
DOCROOT: "$HOME/drupalcore"
SERVER: server.local
WEB_USER: $(whoami)
WEB_GROUP: www-data
hosts:
server.local: 127.0.0.1
TERMINUS_ENV: ci-$CIRCLE_BUILD_NUM
TERMINUS_SITE: d8lcache
DRUPAL_ADMIN_USERNAME: pantheon
# A random password is set in the dependencies:pre stage as a text file.
# This line reads the same file repeatedly. If the openssl call were used
# in this step, it would result in a different password being used in each
# line of other steps. Each CircleCI command runs in a separate shell.
DRUPAL_ADMIN_PASSWORD: $(cat ~/DRUPAL_ADMIN_PASSWORD)

dependencies:
cache_directories:
- ~/.composer/cache
pre:
- echo $(openssl rand -hex 8) > ~/DRUPAL_ADMIN_PASSWORD
# Set the PHP timezone so that Behat script does not fail.
# Using > instead of >> will overwrite the file and disable xdebug.
# xdebug makes composer slower.
#############- echo "date.timezone = 'US/Central'" > /opt/circleci/php/7.0.7/etc/conf.d/xdebug.ini
- git config --global user.email "[email protected]"
- git config --global user.name "Circle CI"
override:
- composer global require "hirak/prestissimo:^0.3"
- composer global require pantheon-systems/terminus "<0.13.0"
- composer global require drush/drush:8.*
# @todo, composer is probably a bad/slow way to install core here.
# Use something like drush's
- git clone --branch 8.1.x https://git.drupal.org/project/drupal.git $DOCROOT
- cd $DOCROOT && composer install
# Add apache config.
# Modify user to make sure that there will be no permission issues.
- sudo usermod -a -G $WEB_GROUP $WEB_USER
# Add apache config.
- |
echo "<VirtualHost *:80>
UseCanonicalName Off
DocumentRoot %DOCROOT%
ServerName %SERVER%
<Directory %DOCROOT%>
Options FollowSymLinks
AllowOverride All
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule %DOCROOT%/(.*)$ index.php/?q=$1 [L,QSA]
Order allow,deny
Allow from all
</Directory>
</VirtualHost>" > apache-vhost.conf
- cp apache-vhost.conf /etc/apache2/sites-available/default
- sudo sed -e "s?%DOCROOT%?$DOCROOT?g" --in-place /etc/apache2/sites-available/default
- sudo sed -e "s?%SERVER%?$SERVER?g" --in-place /etc/apache2/sites-available/default
- sudo a2enmod rewrite
- sudo service apache2 restart
- composer install
post:
- terminus auth login --machine-token=$TERMINUS_TOKEN
test:
pre:
# Copy the settings.local into place
# Disable sendmail binary to suppress any mailouts.
- echo 'sendmail_path = /bin/true' >> ~/.phpenv/versions/$(phpenv version-name)/etc/conf.d/circle.ini
- drush --yes --root=$DOCROOT site-install --db-url=mysql://$DB_USERNAME:[email protected]/$DB_NAME
- curl $SERVER
- cd $DOCROOT && drush en simpletest -y
- cd $DOCROOT && composer config repositories.d8lcache vcs [email protected]:lcache/drupal-8.git
- cd $DOCROOT && composer require "drupal/lcache:dev-master#$CIRCLE_SHA1"
- cd $DOCROOT && drush en lcache -y

override:
- ./vendor/bin/phpcs --report=full --extensions=php,module,inc,theme,info --standard=vendor/drupal/coder/coder_sniffer/Drupal/ --ignore=vendor .
- cd $DOCROOT && /home/ubuntu/.phpenv/shims/php core/scripts/run-tests.sh --url $SERVER --module lcache --php /home/ubuntu/.phpenv/shims/php --verbose --color


#- ./vendor/bin/phpcs --report=full --extensions=php,module,inc,theme,info,install --standard=vendor/drupal/coder/coder_sniffer/Drupal . --ignore=vendor,modules,core,drush,patches,tests
# Make a new multidev env from a vanilla D8 site.
- cd tests/circle-scripts && ./create-fresh-d8-site.sh
- cd tests/circle-scripts && ./clone-repo.sh
- cd tests/circle-scripts && ./core-vs-lcache.sh
#
# - cd tests/circle-scripts && ./enable-modules.sh
# override:
# - cd tests/circle-scripts && ./run-tests.sh
14 changes: 11 additions & 3 deletions composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,18 @@
}
],
"require": {
"lcache/lcache": "v0.3.*"
"lcache/lcache": "0.3.*"
},
"require-dev": {
"phpunit/phpunit": "4.*",
"phpunit/dbunit": "*",
"drupal/coder": "^8.2.0-beta1",
"squizlabs/php_codesniffer": "2.0.*@dev"
}
"squizlabs/php_codesniffer": "2.0.*@dev",
"drush-ops/behat-drush-endpoint": "*",
"drupal/drupal-extension": "dev-master",
"drupal/drupal-driver": "dev-master"
},
"autoload": {
"psr-4": { "Drupal\\LCache\\Behat\\": "tests/behat/bootstrap/" }
}
}
4 changes: 3 additions & 1 deletion src/Backend.php
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,6 @@ public function get($cid, $allow_invalid = FALSE) {
if (is_null($entry->expiration)) {
$entry->expiration = CacheBackendInterface::CACHE_PERMANENT;
}

$response->expire = $entry->expiration;
return $response;
}
Expand All @@ -81,16 +80,19 @@ public function get($cid, $allow_invalid = FALSE) {
* {@inheritdoc}
*/
public function getMultiple(&$cids, $allow_invalid = FALSE) {

if (empty($cids)) {
return;
}

$cache = array();
foreach ($cids as $cid) {
$c = $this->get($cid);
if (!empty($c)) {
$cache[$cid] = $c;
}
}

$cids = array_diff($cids, array_keys($cache));
return $cache;
}
Expand Down
2 changes: 1 addition & 1 deletion src/BackendFactory.php
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ public function __construct(Connection $connection) {
$l1 = new \LCache\APCuL1();
}
$l2 = new \LCache\DatabaseL2($this->getPdoHandle());
$this->integrated = new \LCache\Integrated($l1, $l2);
$this->integrated = new \LCache\Integrated($l1, $l2, 100);
$this->integrated->synchronize();
}

Expand Down
22 changes: 22 additions & 0 deletions tests/behat/behat-pantheon.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#
# behat.yml file for testing with a Drupal site on Pantheon.
#
default:
suites:
default:
contexts:
- Drupal\DrupalExtension\Context\DrupalContext
- Drupal\DrupalExtension\Context\DrushContext
- Drupal\DrupalExtension\Context\MinkContext
- Drupal\LCache\Behat\PerformanceHelpers
extensions:
Behat\MinkExtension:
goutte: ~
# Supplied by env variable.
# base_url: http://dev-d8lcache.pantheonsite.io/
Drupal\DrupalExtension:
blackbox: ~
api_driver: 'drush'
drush:
# Supplied by env variable.
# alias: '@pantheon.d8lcache.dev'
133 changes: 133 additions & 0 deletions tests/behat/bootstrap/PerformanceHelpers.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
<?php

namespace Drupal\LCache\Behat;

use Behat\Behat\Context\Context;
use Behat\Behat\Context\SnippetAcceptingContext;
use Behat\MinkExtension\Context\MinkContext;
use Behat\Behat\Hook\Scope\BeforeScenarioScope;

/**
* Define application features from the specific context.
*/
class PerformanceHelpers implements Context, SnippetAcceptingContext {

/** @var \Behat\MinkExtension\Context\MinkContext */
private $minkContext;

/** @BeforeScenario */
public function gatherContexts(BeforeScenarioScope $scope)
{
$environment = $scope->getEnvironment();
$this->minkContext = $environment->getContext('Drupal\DrupalExtension\Context\MinkContext');
}

/**
* Fills in form field with specified id|name|label|value
* Example: When I fill in "admin_password2" with a random string "12" characters long
*
* @When I fill in :arg1 with a random string :arg2 characters long
*/
public function fillFieldWithRandomString($field, $length)
{
$this->minkContext->fillField($field, $this->rand_string( $length ));
}

protected function rand_string( $length ) {
$chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
$str = '';
$size = strlen( $chars );
for( $i = 0; $i < $length; $i++ ) {
$str .= $chars[ rand( 0, $size - 1 ) ];
}

return $str;
}


/**
* @When I open the links to all homepage posts
*/
public function iOpenTheLinksToAllHomepagePosts()
{

$this->minkContext->visit("/");

$next_page_exists = TRUE;
$next_page_url ='';

while($next_page_exists) {

$page = $this->minkContext->getSession()->getPage();
$next_page_link = $page->find('css', '.pager__item--next a');
//echo $this->minkContext->getSession()->getPage()->find('css', '.pager__items')->getHtml();
if ($next_page_link) {
$next_page_url = $next_page_link->getAttribute('href');
}
else {
$next_page_exists = FALSE;
}

$this->openAllPostLinksOnASinglePage($page);

if (!empty($next_page_link)) {
$this->minkContext->visit($next_page_url);

print_r("\n\n");
print_r($next_page_url);
print_r("\n\n");

//print_r($this->minkContext->getSession()->getResponseHeaders());
}
}
echo "last page visited was $next_page_url";
}


protected function openAllPostLinksOnASinglePage($page) {

$post_urls = $this->getAllPostURLs($page);

foreach ($post_urls as $post_url) {
$this->minkContext->visit($post_url);
//echo "\n";
//echo $this->minkContext->getSession()->getPage()->find('css', 'h1.page-title')->getHtml();
//echo "\n";
//$this->minkContext->printCurrentUrl();
//echo "\n";
//print_r($this->minkContext->getSession()->getResponseHeaders());
}

}

protected function getAllPostURLs($page) {

$post_links = $page->findAll('css', 'article header h2 a');

$post_urls = [];
foreach ($post_links as $post_link) {
$post_urls[] =$post_link->getAttribute('href');
}

//print_r("\n\n");
//print_r($post_urls[0]);
//print_r("\n\n");
return $post_urls;
}

/**
* @Given I log in as an admin
*/
public function ILogInAsAnAdmin()
{
$this->minkContext->visit('user');
$this->minkContext->fillField('name', getenv('DRUPAL_ADMIN_USERNAME'));
$this->minkContext->fillField('pass', getenv('DRUPAL_ADMIN_PASSWORD'));
$this->minkContext->pressButton('Log in');
//$this->minkContext->printCurrentUrl();
$this->minkContext->visit('admin');
//$this->minkContext->printCurrentUrl();

}

}
32 changes: 32 additions & 0 deletions tests/behat/features/create-node-view-all-nodes.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
Feature: Create node, view all nodes



Background:
Given I log in as an admin

@api
Scenario: Create node, view all nodes


When I visit "node/add/article"
And I fill in "title[0][value]" with "Test article title"
And I fill in "title[0][value]" with a random string "12" characters long
And I press the "Save and publish" button

When I visit "node/add/article"
And I fill in "title[0][value]" with "Test article title"
And I fill in "title[0][value]" with a random string "12" characters long
And I press the "Save and publish" button

When I visit "node/add/article"
And I fill in "title[0][value]" with "Test article title"
And I fill in "title[0][value]" with a random string "12" characters long
And I press the "Save and publish" button

When I visit "node/add/article"
And I fill in "title[0][value]" with "Test article title"
And I fill in "title[0][value]" with a random string "12" characters long
And I press the "Save and publish" button

When I open the links to all homepage posts
11 changes: 11 additions & 0 deletions tests/behat/features/node-saving.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
Feature: Node saving

@api
Scenario: Node Saving
Given I am logged in as a user with the "administrator" role
And I visit "/"
When I visit "node/add/article"
And I fill in "title[0][value]" with "First Test Article"
And I press the "Save and publish" button
And I visit "/"
Then I see the text "First Test Article"
5 changes: 5 additions & 0 deletions tests/circle-scripts/clone-repo.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/bash

git clone $(terminus site connection-info --field=git_url) $TERMINUS_SITE
cd $TERMINUS_SITE
git checkout $TERMINUS_ENV
Loading