Skip to content

Audits endpoint #47

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jun 19, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 95 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,9 +174,9 @@ Lists available versions.

#### Versions Parameters

- `version` (optional): Filter by version name(s) - comma-separated list
- `technology` (optional): Filter by technology name(s) - comma-separated list
- `category` (optional): Filter by category - comma-separated list
- `version` (optional): Filter by version name(s) - comma-separated list
- `onlyname` (optional): If present, returns only version names
- `fields` (optional): Comma-separated list of fields to include in the response (see [Field Selection API Documentation](#field-selection-api-documentation) for details)

Expand Down Expand Up @@ -209,10 +209,10 @@ Provides technology adoption data.
#### Adoption Parameters

- `technology` (required): Filter by technology name(s) - comma-separated list
- `geo` (required): Filter by geographic location
- `rank` (required): Filter by rank
- `start` (optional): Filter by date range start (YYYY-MM-DD or 'latest')
- `end` (optional): Filter by date range end (YYYY-MM-DD)
- `geo` (optional): Filter by geographic location
- `rank` (optional): Filter by rank

#### Adoption Response

Expand Down Expand Up @@ -334,8 +334,8 @@ Provides Page Weight metrics for technologies.
#### Page Weight Parameters

- `technology` (required): Filter by technology name(s) - comma-separated list
- `geo` (optional): Filter by geographic location
- `rank` (optional): Filter by rank
- `geo` (required): Filter by geographic location
- `rank` (required): Filter by rank
- `start` (optional): Filter by date range start (YYYY-MM-DD or 'latest')
- `end` (optional): Filter by date range end (YYYY-MM-DD)

Expand Down Expand Up @@ -387,6 +387,70 @@ Returns a JSON object with the following schema:
]
```


### `GET /audits`

Provides Lighthouse audits for technologies.

#### Audits Parameters

- `technology` (required): Filter by technology name(s) - comma-separated list
- `geo` (required): Filter by geographic location
- `rank` (required): Filter by rank
- `start` (optional): Filter by date range start (YYYY-MM-DD or 'latest')
- `end` (optional): Filter by date range end (YYYY-MM-DD)

#### Audits Response

```bash
curl --request GET \
--url 'https://{{HOST}}/v1/audits?start=latest&geo=ALL&technology=WordPress&rank=ALL'
```

Returns a JSON object with the following schema:

```json
[
{
"date": "2025-06-01",
"audits": [
{
"desktop": {
"pass_origins": 2428028
},
"mobile": {
"pass_origins": 2430912
},
"id": "first-contentful-paint",
"category": "performance"
},
{
"desktop": {
"pass_origins": 490451
},
"mobile": {
"pass_origins": 477218
},
"id": "largest-contentful-paint",
"category": "performance"
},
{
"desktop": {
"pass_origins": 1221876
},
"mobile": {
"pass_origins": 1296673
},
"id": "cumulative-layout-shift",
"category": "performance"
}
],
"technology": "WordPress"
},
...
]
```

### `GET /ranks`

Lists all available ranks.
Expand Down Expand Up @@ -456,6 +520,32 @@ Returns a JSON object with the following schema:
}
```

### `POST /v1/cache-reset`

Resets all caches in the API. This endpoint requires a POST request.

```bash
curl --request POST \
--url 'https://{{HOST}}/v1/cache-reset'
```

Returns a JSON object with the following schema:

```json
{
"success": true,
"message": "All caches have been reset",
"before": {
"queryCache": 150,
"dateCache": 12
},
"after": {
"queryCache": 0,
"dateCache": 0
}
}
```

## Testing

```bash
Expand Down
35 changes: 35 additions & 0 deletions src/__tests__/routes.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,10 @@ describe('API Routes', () => {
const res = await request(app).get('/v1/technologies?invalid=parameter');
expect(res.statusCode).toEqual(200);
expect(Array.isArray(res.body)).toBe(true);

//expect(res.body).toHaveProperty('errors');
//expect(res.body.errors[0]).toHaveProperty('error');
//expect(res.body.errors[0].error).toContain('Unsupported parameters: ');
});
});

Expand All @@ -358,4 +362,35 @@ describe('API Routes', () => {
expect(res.headers['timing-allow-origin']).toEqual('*');
});
});

describe('Cache Management', () => {
it('should provide cache stats', async () => {
const res = await request(app)
.get('/v1/cache-stats')
.expect(200);

expect(res.body).toHaveProperty('queryCache');
expect(res.body).toHaveProperty('dateCache');
expect(res.body).toHaveProperty('config');
});

it('should reset cache on POST request', async () => {
const res = await request(app)
.post('/v1/cache-reset')
.expect(200);

expect(res.body).toHaveProperty('success', true);
expect(res.body).toHaveProperty('message');
expect(res.body).toHaveProperty('before');
expect(res.body).toHaveProperty('after');
});

it('should handle cache reset OPTIONS request', async () => {
const res = await request(app)
.options('/v1/cache-reset')
.expect(204);

expect(res.headers['access-control-allow-methods']).toContain('POST');
});
});
});
13 changes: 13 additions & 0 deletions src/controllers/categoriesController.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,19 @@ import { executeQuery, validateArrayParameter } from '../utils/controllerHelpers
*/
const listCategories = async (req, res) => {
const queryBuilder = async (params) => {
/*
// Validate parameters
const supportedParams = ['category', 'onlyname', 'fields'];
const providedParams = Object.keys(params);
const unsupportedParams = providedParams.filter(param => !supportedParams.includes(param));

if (unsupportedParams.length > 0) {
const error = new Error(`Unsupported parameters: ${unsupportedParams.join(', ')}.`);
error.statusCode = 400;
throw error;
}
*/

const isOnlyNames = params.onlyname || typeof params.onlyname === 'string';
const hasCustomFields = params.fields && !isOnlyNames;

Expand Down
24 changes: 22 additions & 2 deletions src/controllers/reportController.js
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,10 @@ const REPORT_CONFIGS = {
cwv: {
table: 'core_web_vitals',
dataField: 'vitals'
},
audits: {
table: 'audits',
dataField: 'audits'
}
};

Expand All @@ -49,6 +53,19 @@ const createReportController = (reportType) => {
try {
const params = req.query;

/*
// Validate supported parameters
const supportedParams = ['technology', 'geo', 'rank', 'start', 'end'];
const providedParams = Object.keys(params);
const unsupportedParams = providedParams.filter(param => !supportedParams.includes(param));

if (unsupportedParams.length > 0) {
const error = new Error(`Unsupported parameters: ${unsupportedParams.join(', ')}.`);
error.statusCode = 400;
throw error;
}
*/

// Validate required parameters using shared utility
const errors = validateRequiredParams(params, [
REQUIRED_PARAMS.GEO,
Expand Down Expand Up @@ -133,7 +150,10 @@ const createReportController = (reportType) => {
};

// Export individual controller functions
export const listAuditsData = createReportController('audits');
export const listAdoptionData = createReportController('adoption');
export const listPageWeightData = createReportController('pageWeight');
export const listLighthouseData = createReportController('lighthouse');
export const listCWVTechData = createReportController('cwv');
export const listLighthouseData = createReportController('lighthouse');
export const listPageWeightData = createReportController('pageWeight');


13 changes: 13 additions & 0 deletions src/controllers/technologiesController.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,19 @@ import { executeQuery, validateTechnologyArray, validateArrayParameter, FIRESTOR
*/
const listTechnologies = async (req, res) => {
const queryBuilder = async (params) => {
/*
// Validate parameters
const supportedParams = ['technology', 'category', 'onlyname', 'fields'];
const providedParams = Object.keys(params);
const unsupportedParams = providedParams.filter(param => !supportedParams.includes(param));

if (unsupportedParams.length > 0) {
const error = new Error(`Unsupported parameters: ${unsupportedParams.join(', ')}.`);
error.statusCode = 400;
throw error;
}
*/

const isOnlyNames = params.onlyname || typeof params.onlyname === 'string';
const hasCustomFields = params.fields && !isOnlyNames;

Expand Down
13 changes: 13 additions & 0 deletions src/controllers/versionsController.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,19 @@ import { executeQuery, validateTechnologyArray, FIRESTORE_IN_LIMIT } from '../ut
*/
const listVersions = async (req, res) => {
const queryBuilder = async (params) => {
/*
// Validate parameters
const supportedParams = ['version', 'technology', 'category', 'onlyname', 'fields'];
const providedParams = Object.keys(params);
const unsupportedParams = providedParams.filter(param => !supportedParams.includes(param));

if (unsupportedParams.length > 0) {
const error = new Error(`Unsupported parameters: ${unsupportedParams.join(', ')}.`);
error.statusCode = 400;
throw error;
}
*/

let query = firestore.collection('versions');

// Apply technology filter with validation
Expand Down
20 changes: 19 additions & 1 deletion src/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ const controllers = {
cwvtech: null,
lighthouse: null,
pageWeight: null,
audits: null,
ranks: null,
geos: null,
versions: null
Expand All @@ -30,6 +31,7 @@ const getController = async (name) => {
case 'cwvtech':
case 'lighthouse':
case 'pageWeight':
case 'audits':
controllers[name] = await import('./controllers/reportController.js');
break;
case 'ranks':
Expand All @@ -49,7 +51,7 @@ const getController = async (name) => {
// Helper function to set CORS headers
const setCORSHeaders = (res) => {
res.setHeader('Access-Control-Allow-Origin', '*');
res.setHeader('Access-Control-Allow-Methods', 'GET, OPTIONS');
res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS');
res.setHeader('Access-Control-Allow-Headers', 'Content-Type, Timing-Allow-Origin');
res.setHeader('Access-Control-Max-Age', '86400');
};
Expand Down Expand Up @@ -105,6 +107,14 @@ const handleRequest = async (req, res) => {
return;
}

// Validate URL to skip XSS attacks
const unsafe = /onerror|onload|javascript:/i;
if (unsafe.test(req.url)) {
res.statusCode = 400
res.end(JSON.stringify({ error: 'Invalid input' }));
return;
}

// Parse URL
const parsedUrl = url.parse(req.url, true);
const pathname = parsedUrl.pathname;
Expand Down Expand Up @@ -136,6 +146,9 @@ const handleRequest = async (req, res) => {
} else if (pathname === '/v1/page-weight' && req.method === 'GET') {
const { listPageWeightData } = await getController('pageWeight');
await listPageWeightData(req, res);
} else if (pathname === '/v1/audits' && req.method === 'GET') {
const { listAuditsData } = await getController('audits');
await listAuditsData(req, res);
} else if (pathname === '/v1/ranks' && req.method === 'GET') {
const { listRanks } = await getController('ranks');
await listRanks(req, res);
Expand All @@ -150,6 +163,11 @@ const handleRequest = async (req, res) => {
const { getCacheStats } = await import('./utils/controllerHelpers.js');
const stats = getCacheStats();
sendJSONResponse(res, stats);
} else if (pathname === '/v1/cache-reset' && req.method === 'POST') {
// Cache reset endpoint
const { resetCache } = await import('./utils/controllerHelpers.js');
const result = resetCache();
sendJSONResponse(res, result);
} else {
// 404 Not Found
res.statusCode = 404;
Expand Down
Loading
Loading