Skip to content

Commit 0172342

Browse files
committed
init
0 parents  commit 0172342

14 files changed

+543
-0
lines changed

.gitignore

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
node_modules
2+
package-lock.json

.travis.yml

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
language: node_js
2+
node_js:
3+
- '8'
4+
- '10'

license

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2019 Bart Veneman
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

package.json

+37
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
{
2+
"name": "distill-css",
3+
"description": "PACKAGE_DESCRIPTION",
4+
"version": "0.1.0",
5+
"homepage": "https://www.projectwallace.com/oss",
6+
"repository": "PACKAGE_REPOSITORY",
7+
"issues": "PACKAGE_ISSUES",
8+
"license": "MIT",
9+
"author": "Bart Veneman",
10+
"keywords": [
11+
"PACKAGE_KEYWORD"
12+
],
13+
"scripts": {
14+
"test": "xo && ava test"
15+
},
16+
"files": [
17+
"src"
18+
],
19+
"main": "src/index.js",
20+
"engines": {
21+
"node": ">=8.0"
22+
},
23+
"xo": {
24+
"prettier": true
25+
},
26+
"devDependencies": {
27+
"ava": "^1.3.1",
28+
"chromium": "^2.1.0",
29+
"create-test-server": "^2.4.0",
30+
"prettier": "^1.16.4",
31+
"puppeteer-core": "^1.13.0",
32+
"xo": "^0.24.0"
33+
},
34+
"dependencies": {
35+
"puppeteer": "^1.13.0"
36+
}
37+
}

readme.md

+94
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
<div align="center">
2+
<h1>extract-css-core</h1>
3+
<p>Extract all CSS from a given url, both server side and client side rendered.</p>
4+
</div>
5+
6+
[![NPM Version](https://img.shields.io/npm/v/extract-css-core.svg)](https://www.npmjs.com/package/extract-css-core)
7+
[![Build Status](https://travis-ci.org/bartveneman/extract-css-core.svg?branch=master)](https://travis-ci.org/bartveneman/extract-css-core)
8+
[![Known Vulnerabilities](https://snyk.io/test/github/bartveneman/extract-css-core/badge.svg)](https://snyk.io/test/github/bartveneman/extract-css-core)
9+
[![Weekly downloads](https://img.shields.io/npm/dw/extract-css-core.svg)](https://www.npmjs.com/package/extract-css-core)
10+
![Dependencies Status](https://img.shields.io/david/bartveneman/extract-css-core.svg)
11+
![Dependencies Status](https://img.shields.io/david/dev/bartveneman/extract-css-core.svg)
12+
[![XO code style](https://img.shields.io/badge/code_style-XO-5ed9c7.svg)](https://github.com/sindresorhus/xo)
13+
[![Project: Wallace](https://img.shields.io/badge/Project-Wallace-29c87d.svg)](https://www.projectwallace.com/oss)
14+
15+
## Problem, solution and shortcomings
16+
17+
### Problem
18+
19+
Existing packages like [get-css](https://github.com/cssstats/cssstats/tree/master/packages/get-css) look at a server-generated piece of HTML and get all the `<link>` and `<style>` tags from it. This works fine for 100% server rendered pages, but apps that employ style injection with JavaScript will not be covered.
20+
21+
### Solution
22+
23+
This module uses an instance of Chromium to render a page. This has the benefit that most of the styles can be rendered, even when generated by JavaScript. The [Puppeteer CSSCoverage API](https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#coveragestartcsscoverageoptions) is the power behind finding most of the CSS.
24+
25+
### Shortcomings
26+
27+
Currently, there is no solution to get the CSS from modules that use [Styled Components](https://www.styled-components.com) or something similar. Any help resolving this issue will be very much appreciated.
28+
29+
## Installation
30+
31+
```sh
32+
npm install extract-css-core
33+
34+
# Or with Yarn
35+
yarn add extract-css-core
36+
```
37+
38+
## Usage
39+
40+
```js
41+
const extractCss = require("extract-css-core");
42+
43+
extractCss("http://www.projectwallace.com").then(css => console.log(css));
44+
```
45+
46+
## API
47+
48+
### extractCss(url, [options])
49+
50+
Extract CSS from a page. Returns a Promise that contains the CSS as a single String.
51+
52+
### Options
53+
54+
Type: `Object`
55+
56+
#### debug
57+
58+
Type: `Boolean`
59+
Default: `false`
60+
61+
Set to `true` if you want a Chromium window to open as it works to get all the CSS from the page.
62+
63+
#### waitUntil
64+
65+
Type: `String`
66+
Default: `networkidle2`
67+
68+
Can be any value as provided by the [Puppeteer docs](https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pagegotourl-options).
69+
70+
#### customBrowser
71+
72+
Type: `Object`
73+
Default: `null`
74+
75+
This is useful if you want to run extract-css on AWS Lambda for example.
76+
77+
##### executablePath
78+
79+
Type: `String`
80+
Default: `null`
81+
82+
Pass in the executable path for a custom Chromium instance.
83+
84+
##### customPuppeteer
85+
86+
Type: `Object`
87+
Default: `null`
88+
89+
You probably want to provide [puppeteer-core](https://www.npmjs.com/package/puppeteer-core) for a custom browser instead of [puppeteer](https://www.npmjs.com/package/puppeteer) which brings it's own Chromium instance.
90+
91+
## Related
92+
93+
- [extract-css lambda](https://github.com/bartveneman/extract-css) - Extract CSS running as a serverless function
94+
- [get-css](https://github.com/cssstats/cssstats/tree/master/packages/get-css) - The original get-css

src/index.js

+85
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
const puppeteer = require("puppeteer");
2+
3+
function InvalidUrlError({ url, statusCode, statusText }) {
4+
this.name = "InvalidUrlError";
5+
this.message = `There was an error retrieving CSS from ${url}.\n\tHTTP status code: ${statusCode} (${statusText})`;
6+
}
7+
8+
InvalidUrlError.prototype = Error.prototype;
9+
10+
module.exports = async (
11+
url,
12+
{ debug = false, waitUntil = "networkidle2", customBrowser = {} } = {}
13+
) => {
14+
const browserOptions = {
15+
headless: debug !== true,
16+
puppeteer
17+
};
18+
19+
// Replace the puppeteer instance if a custom one is provided
20+
// This also means that the executablePath needs to be set to
21+
// a custom path where some chromium instance is running.
22+
if (
23+
customBrowser &&
24+
customBrowser.executablePath &&
25+
customBrowser.customPuppeteer
26+
) {
27+
browserOptions.executablePath = customBrowser.executablePath;
28+
browserOptions.puppeteer = customBrowser.customPuppeteer;
29+
}
30+
31+
// Setup a browser instance
32+
const browser = await browserOptions.puppeteer.launch(browserOptions);
33+
34+
// Create a new page and navigate to it
35+
const page = await browser.newPage();
36+
37+
// Start CSS coverage. This is the meat and bones of this module
38+
await page.coverage.startCSSCoverage();
39+
const response = await page.goto(url, { waitUntil });
40+
41+
// Make sure that we only try to extract CSS from valid pages.
42+
// Bail out if the response is an invalid request (400, 500)
43+
if (response.status() >= 400) {
44+
await browser.close(); // Don't leave any resources behind
45+
46+
return Promise.reject(
47+
new InvalidUrlError({
48+
url,
49+
statusCode: response.status(),
50+
statusText: response.statusText()
51+
})
52+
);
53+
}
54+
55+
// Coverage contains a lot of <style> and <link> CSS,
56+
// but not all...
57+
const coverage = await page.coverage.stopCSSCoverage();
58+
59+
// Fetch all <style> tags from the page, because the coverage
60+
// API may have missed some JS-generated <style> tags.
61+
// Some of them *were* already caught by the coverage API,
62+
// but they will be removed later on to prevent duplicates.
63+
const styleTagsCss = (await page.$$eval("style", styles => {
64+
// Get the text inside each <style> tag and trim() the
65+
// results to prevent all the inside-html indentation
66+
// clogging up the results and making it look
67+
// bigger than it actually is
68+
return styles.map(style => style.innerHTML.trim());
69+
})).join("");
70+
71+
await browser.close();
72+
73+
// Turn the coverage Array into a single string of CSS
74+
const coverageCss = coverage
75+
// Filter out the <style> tags that were found in the coverage
76+
// report since we've conducted our own search for them.
77+
// A coverage CSS item with the same url as the url of the page
78+
// we requested is an indication that this was a <style> tag
79+
.filter(styles => styles.url !== url)
80+
// The `text` property contains the actual CSS
81+
.map(({ text }) => text)
82+
.join("");
83+
84+
return Promise.resolve(coverageCss + styleTagsCss);
85+
};

test/css-in-js.html

+38
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
<!DOCTYPE html>
2+
<html lang="en">
3+
<head>
4+
<meta charset="UTF-8" />
5+
<title>CodePen - CSS-in-JS example for extract-css-core unit tests</title>
6+
7+
<style>
8+
html {
9+
color: #f00;
10+
}
11+
</style>
12+
</head>
13+
14+
<body translate="no">
15+
<div id="app"></div>
16+
17+
<script src="https://cdnjs.cloudflare.com/ajax/libs/react/16.8.4/umd/react.production.min.js"></script>
18+
<script src="https://cdnjs.cloudflare.com/ajax/libs/react-dom/16.8.4/umd/react-dom.production.min.js"></script>
19+
<script src="https://unpkg.com/styled-components/dist/styled-components.min.js"></script>
20+
21+
<script id="rendered-js">
22+
const Title = styled.h1`
23+
color: blue;
24+
font-family: sans-serif;
25+
font-size: 3em;
26+
`;
27+
28+
const App = () => {
29+
return React.createElement(Title, null, "Title");
30+
};
31+
32+
ReactDOM.render(
33+
React.createElement(App, null),
34+
document.querySelector("#app")
35+
);
36+
</script>
37+
</body>
38+
</html>

test/fixture.css

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
body {
2+
color: teal;
3+
}

0 commit comments

Comments
 (0)