Skip to content

Commit

Permalink
Fix merge
Browse files Browse the repository at this point in the history
  • Loading branch information
AliasIO committed Aug 18, 2023
1 parent b3075d6 commit eb52546
Show file tree
Hide file tree
Showing 3,522 changed files with 789 additions and 3,138 deletions.
The diff you're trying to view is too large. We only load the first 3000 changed files.
9 changes: 4 additions & 5 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,20 @@ about: Create a report to help us improve
title: ''
labels: Bug
assignees: ''

---

**Platform**
Which platform does the bug apply to?
* WebExtension (Google Chrome, Mozilla Firefox)
* NPM
* Bookmarklet
* Website

- WebExtension (Google Chrome, Mozilla Firefox)
- Website

**Describe the bug**
A clear and concise description of what the bug is.

**To Reproduce**
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '...'
3. Scroll down to '...'
Expand Down
20 changes: 0 additions & 20 deletions .github/workflows/submit.yml

This file was deleted.

9 changes: 3 additions & 6 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,10 +1,7 @@
/build/*
/src/drivers/**/categories.json
/src/drivers/**/technologies/*
/src/drivers/**/wappalyzer.js
/src/drivers/webextension/images/icons/converted/*
/src/drivers/webextension/manifest.json
/src/drivers/webextension/manifest.bak.json
/src/images/icons/converted/*
/src/manifest.json
/src/manifest.bak.json

node_modules
package-lock.json
Expand Down
16 changes: 8 additions & 8 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,29 +8,29 @@ To get started, see the [README](https://github.com/wappalyzer/wappalyzer/blob/m

## Submitting changes

- First, run `yarn run validate` to identify any issues.
- Use descriptive commit messages, e.g. 'Add WordPress detection'.
- Push your commits to a new branch on your own fork.
- Finally, submit a [pull request](https://help.github.com/articles/about-pull-requests/) and describe your changes.
- First, run `yarn run validate` to identify any issues.
- Use descriptive commit messages, e.g. 'Add WordPress detection'.
- Push your commits to a new branch on your own fork.
- Finally, submit a [pull request](https://help.github.com/articles/about-pull-requests/) and describe your changes.

## Adding a new technology

Wappalyzer uses [regular expressions](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions) to fingerprint technologies. Refer to the [specification](https://github.com/wappalyzer/wappalyzer/blob/master/README.md#specification) for detail.

- Add a new block to [`src/technologies/*.json`](https://github.com/wappalyzer/wappalyzer/blob/master/src/technologies). The filename should match the first letter of the technology name (a-z). Use `_.json` if the first character is a number or symbol.
- Add an icon to [`src/drivers/webextension/images/icons`](https://github.com/wappalyzer/wappalyzer/tree/master/src/drivers/webextension/images/icons). The image must be square, either SVG or PNG (32 x 32 pixels).
- Add a new block to [`src/technologies/*.json`](https://github.com/wappalyzer/wappalyzer/blob/master/src/technologies). The filename should match the first letter of the technology name (a-z). Use `_.json` if the first character is a number or symbol.
- Add an icon to [`src/images/icons`](https://github.com/wappalyzer/wappalyzer/tree/master/src/images/icons). The image must be square, either SVG or PNG (32 x 32 pixels).

Only widely used technologies are accepted. When creating a pull request, include ten or more links to websites that use the application, a GitHub page with at least 1,000 stars or anything that will help establish the size of the user base.

## Adding a new category

Please [open an issue on GitHub](https://github.com/wappalyzer/wappalyzer/issues) first to discuss the need for a new category.

To add a category, edit [`src/categories.json`](https://github.com/wappalyzer/wappalyzer/blob/master/src/categories.json) and update every [locale](https://github.com/wappalyzer/wappalyzer/tree/master/src/drivers/webextension/_locales). You may use the English category name in all of them.
To add a category, edit [`src/categories.json`](https://github.com/wappalyzer/wappalyzer/blob/master/src/categories.json) and update every [locale](https://github.com/wappalyzer/wappalyzer/tree/master/src/_locales). You may use the English category name in all of them.

## Adding a new translation

To add a new translation, copy the `en` folder in [`src/drivers/webextension/_locales`](https://github.com/wappalyzer/wappalyzer/tree/master/src/drivers/webextension/_locales), rename it to the relevant two-letter country code and update the containing `messages.json` file.
To add a new translation, copy the `en` folder in [`src/_locales`](https://github.com/wappalyzer/wappalyzer/tree/master/src/_locales), rename it to the relevant two-letter country code and update the containing `messages.json` file.

## Adding a new feature

Expand Down
47 changes: 19 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
[![Validate](https://github.com/wappalyzer/wappalyzer/actions/workflows/validate.yml/badge.svg)](https://github.com/wappalyzer/wappalyzer/actions/workflows/validate.yml)
[![wappalyzer NPM](https://img.shields.io/badge/npm-wappalyzer-blue)](https://www.npmjs.com/package/wappalyzer)
[![wappalyzer-core NPM](https://img.shields.io/badge/npm-wappalyzer--core-blue)](https://www.npmjs.com/package/wappalyzer-core)
[![Github Sponsor](https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&link=https://github.com/sponsors/AliasIO)](https://github.com/sponsors/AliasIO)

<a href="https://www.wappalyzer.com/?utm_source=readme&utm_medium=github&utm_campaign=wappalyzer"><img src="https://www.wappalyzer.com/images/logo/icon_192.png" height="72" alt="Wappalyzer" align="left" /></a>
Expand All @@ -11,43 +9,34 @@

**[Wappalyzer](https://www.wappalyzer.com) identifies technologies on websites, such as CMS, web frameworks, ecommerce platforms, JavaScript libraries, analytics tools and [more](https://www.wappalyzer.com/technologies).**

If you don't have time to configure, host, debug and maintain your own infrastructure to analyse websites at scale, we offer a SaaS solution that has all the same capabilities and a lot more. Our [apps](https://www.wappalyzer.com/apps/) and [APIs](https://www.wappalyzer.com/api/) not only reveal the technology stack a website uses but also company and contact details, social media profiles, keywords and metadata.

## Prerequisites

- [Git](https://git-scm.com)
- [Node.js](https://nodejs.org) version 14 or higher
- [Yarn](https://yarnpkg.com)
- [Git](https://git-scm.com)
- [Node.js](https://nodejs.org) version 14 or higher
- [Yarn](https://yarnpkg.com)

## Quick start

```sh
git clone https://github.com/wappalyzer/wappalyzer.git
cd wappalyzer
yarn install
yarn run link
```

## Usage

### Command line

```sh
node src/drivers/npm/cli.js https://example.com
```

### Chrome extension

* Go to `about:extensions`
* Enable 'Developer mode'
* Click 'Load unpacked'
* Select `src/drivers/webextension`
- Go to `about:extensions`
- Enable 'Developer mode'
- Click 'Load unpacked'
- Select `src`

### Firefox extension

* Go to `about:debugging#/runtime/this-firefox`
* Click 'Load Temporary Add-on'
* Select `src/drivers/webextension/manifest.json`
- Go to `about:debugging#/runtime/this-firefox`
- Click 'Load Temporary Add-on'
- Select `src/manifest.json`

## Specification

Expand Down Expand Up @@ -208,13 +197,15 @@ Find the JSON schema at [`schema.json`](https://github.com/wappalyzer/wappalyzer
Cost indicator (based on a typical plan or average monthly price) and available pricing models. For paid products only.

One of:

<ul>
<li><code>low</code>Less than US $100 / mo</li>
<li><code>mid</code>Between US $100 - $1,000 / mo</li>
<li><code>high</code>More than US $1,000 / mo</li>
</ul>

Plus any of:

<ul>
<li><code>freemium</code> Free plan available</li>
<li><code>onetime</code> One-time payments accepted</li>
Expand Down Expand Up @@ -319,7 +310,7 @@ Plus any of:
<td><code>dns</code></td>
<td>Object</td>
<td>
DNS records: supports MX, TXT, SOA and NS (NPM driver only).
DNS records: supports MX, TXT, SOA and NS.
</td>
<td>
<code>{ "MX": "example\\.com" }</code>
Expand Down Expand Up @@ -362,7 +353,7 @@ Plus any of:
<td><code>probe</code></td>
<td>Object</td>
<td>
Request a URL to test for its existence or match text content (NPM driver only).
Request a URL to test for its existence or match text content.
</td>
<td><code>{ "/path": "Example text" }</code></td>
</tr>
Expand Down Expand Up @@ -430,11 +421,11 @@ Patterns are essentially JavaScript regular expressions written as strings, but

### Quirks and pitfalls

- Because of the string format, the escape character itself must be escaped when using special characters such as the dot (`\\.`). Double quotes must be escaped only once (`\"`). Slashes do not need to be escaped (`/`).
- Flags are not supported. Regular expressions are treated as case-insensitive.
- Capture groups (`()`) are used for version detection. In other cases, use non-capturing groups (`(?:)`).
- Use start and end of string anchors (`^` and `$`) where possible for optimal performance.
- Short or generic patterns can cause applications to be identified incorrectly. Try to find unique strings to match.
- Because of the string format, the escape character itself must be escaped when using special characters such as the dot (`\\.`). Double quotes must be escaped only once (`\"`). Slashes do not need to be escaped (`/`).
- Flags are not supported. Regular expressions are treated as case-insensitive.
- Capture groups (`()`) are used for version detection. In other cases, use non-capturing groups (`(?:)`).
- Use start and end of string anchors (`^` and `$`) where possible for optimal performance.
- Short or generic patterns can cause applications to be identified incorrectly. Try to find unique strings to match.

### Tags

Expand Down
35 changes: 10 additions & 25 deletions bin/build.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
const fs = require('fs')
const Zip = require('adm-zip')

const currentVersion = JSON.parse(fs.readFileSync('./src/package.json')).version
const currentVersion = JSON.parse(
fs.readFileSync('./src/manifest-v3.json')
).version

const version = process.argv[2]

Expand All @@ -14,47 +16,30 @@ if (!version) {
process.exit(1)
}

;[
'./src/package.json',
'./src/drivers/npm/package.json',
'./src/drivers/webextension/manifest-v2.json',
'./src/drivers/webextension/manifest-v3.json',
].forEach((file) => {
;['./src/manifest-v2.json', './src/manifest-v3.json'].forEach((file) => {
const json = JSON.parse(fs.readFileSync(file))

json.version = version

fs.writeFileSync(file, JSON.stringify(json, null, 2))
})

fs.copyFileSync(
`./src/drivers/webextension/manifest.json`,
'./src/drivers/webextension/manifest.bak.json'
)
fs.copyFileSync(`./src/manifest.json`, './src/manifest.bak.json')

fs.copyFileSync(
`./src/drivers/webextension/manifest-v2.json`,
'./src/drivers/webextension/manifest.json'
)
fs.copyFileSync(`./src/manifest-v2.json`, './src/manifest.json')

let zip = new Zip()

zip.addLocalFolder('./src/drivers/webextension', '')
zip.addLocalFolder('./src', '')

zip.writeZip('./build/webextension-v2.zip')

fs.copyFileSync(
`./src/drivers/webextension/manifest-v3.json`,
'./src/drivers/webextension/manifest.json'
)
fs.copyFileSync(`./src/manifest-v3.json`, './src/manifest.json')

zip = new Zip()

zip.addLocalFolder('./src/drivers/webextension', '')
zip.addLocalFolder('./src', '')

zip.writeZip('./build/webextension-v3.zip')

fs.copyFileSync(
`./src/drivers/webextension/manifest.bak.json`,
'./src/drivers/webextension/manifest.json'
)
fs.copyFileSync(`./src/manifest.bak.json`, './src/manifest.json')
4 changes: 2 additions & 2 deletions bin/convert.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ const appPaths = () => {
return {
basePath: fileDir,
appPath: appDir,
iconPath: appDir + '/src/drivers/webextension/images/icons',
convertPath: appDir + '/src/drivers/webextension/images/icons/converted',
iconPath: appDir + '/src/images/icons',
convertPath: appDir + '/src/images/icons/converted',
}
}

Expand Down
28 changes: 0 additions & 28 deletions bin/link.js

This file was deleted.

5 changes: 1 addition & 4 deletions bin/manifest.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,4 @@ if (!version) {
process.exit(1)
}

fs.copyFileSync(
`./src/drivers/webextension/manifest-${version}.json`,
'./src/drivers/webextension/manifest.json'
)
fs.copyFileSync(`./src/manifest-${version}.json`, './src/manifest.json')
2 changes: 1 addition & 1 deletion bin/validate.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
const fs = require('fs')

const iconPath = './src/drivers/webextension/images/icons'
const iconPath = './src/images/icons'

const categories = JSON.parse(fs.readFileSync('./src/categories.json'))

Expand Down
5 changes: 2 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,13 @@
"terminal-overwrite": "^2.0.1"
},
"scripts": {
"link": "node ./bin/link.js; node ./bin/manifest.js v3",
"lint": "eslint src/**/*.{js,json}",
"lint:fix": "eslint --fix src/**/*.{js,json}",
"validate": "yarn run lint && jsonlint -qV ./schema.json ./src/technologies/ && node ./bin/validate.js",
"convert": "node --no-warnings ./bin/convert.js",
"prettify": "jsonlint -si --trim-trailing-commas --enforce-double-quotes ./src/categories.json ./src/technologies/*.json",
"build": "yarn run link && yarn run validate && yarn run prettify && yarn run convert && node ./bin/build.js",
"build:safari": "xcrun safari-web-extension-converter --swift --project-location build --force src/drivers/webextension",
"build": "yarn run validate && yarn run prettify && yarn run convert && node ./bin/build.js",
"build:safari": "xcrun safari-web-extension-converter --swift --project-location build --force src",
"manifest": "node ./bin/manifest.js"
}
}
Loading

0 comments on commit eb52546

Please sign in to comment.