Crow Nest 🐦

A visual regression testing dashboard for monitoring website changes across multiple platforms.

Overview

Crow Nest is a comprehensive visual regression testing system that automatically captures screenshots of websites and compares them against baseline images to detect visual changes. It provides a beautiful dashboard to monitor test results and manage multiple entities.

Features

• Multi-Entity Support: Test multiple websites/platforms simultaneously
• Automated Screenshots: Capture full-page screenshots with custom viewport settings
• Visual Comparison: Detect changes between baseline and current screenshots
• Beautiful Dashboard: Modern React-based UI with real-time results
• Flexible Configuration: Customize viewport sizes, wait times, and other settings per entity
• GitHub Integration: Automated deployment to GitHub Pages
• Detailed Reporting: Comprehensive test results with error handling

How It Works

Architecture

The system consists of three main components:

1. Backend Script (scripts/visual-regression.js): Node.js script using Playwright to capture screenshots
2. Frontend Dashboard (frontend/): React application to display results
3. Configuration (config.json): JSON configuration for entities and their settings

Workflow

1. Configuration: Define entities and their URLs in config.json
2. Screenshot Capture: Run the visual regression script to capture screenshots
3. Comparison: Compare current screenshots against baseline images
4. Results Storage: Save results in JSON format with metadata
5. Dashboard Display: View results in the web dashboard

Quick Start

Prerequisites

• Node.js 18+
• Yarn package manager

Installation

1. Clone the repository:

   git clone https://github.com/codeinwp/crow-nest.git
   cd crow-nest

2. Install dependencies:

   yarn install
   yarn run install-frontend

3. Configure entities: Edit config.json to add your websites (see Configuration section below)

4. Run tests:

   yarn test

5. Start the dashboard:

   yarn dev

Configuration

Adding New Entities

To add a new website/platform for testing, edit the config.json file:

{
  "my-website": {
    "title": "My Website",
    "links": [
      "https://example.com",
      "https://example.com/about",
      "https://example.com/contact"
    ],
    "description": "My company website",
    "viewport": {
      "width": 1920,
      "height": 1080
    },
    "waitTime": 3000
  }
}

Configuration Options

Option	Type	Default	Description
title	string	-	Human-readable name for the entity
links	array	-	Array of URLs to test
description	string	-	Description of the entity
viewport	object	{width: 1920, height: 1080}	Browser viewport dimensions
waitTime	number	2000	Wait time in milliseconds after page load

Example Configuration

{
  "themeisle": {
    "title": "Themeisle",
    "links": [
      "https://themeisle.com"
    ],
    "description": "WordPress themes and plugins marketplace",
    "viewport": {
      "width": 1920,
      "height": 1080
    },
    "waitTime": 2000
  },
  "optimole": {
    "title": "Optimole",
    "links": [
      "https://optimole.com"
    ],
    "description": "Image optimization and CDN service",
    "viewport": {
      "width": 1440,
      "height": 900
    },
    "waitTime": 3000
  }
}

Running Tests

Manual Testing

Run visual regression tests for all configured entities:

yarn test

Automated Testing

The system can be integrated with CI/CD pipelines. The GitHub Actions workflow automatically:

1. Runs tests on push to master
2. Builds the frontend dashboard
3. Deploys to GitHub Pages

Test Results

Test results are stored in the entities/ directory:

entities/
├── themeisle/
│   ├── baseline/
│   │   └── screenshot-1.png
│   ├── 2025.01.14/
│   │   ├── results.json
│   │   └── screenshot-1.png
│   └── latest.json
└── optimole/
    ├── baseline/
    └── latest.json

Dashboard

Features

• Entity Overview: View all configured entities with their status
• Detailed Results: Click on any entity to see individual test results
• Screenshot Comparison: View baseline and current screenshots side-by-side
• Status Indicators: Visual indicators for passing/failing tests
• Error Handling: Display and manage test errors

Accessing the Dashboard

• Development: yarn dev (runs on http://localhost:3000)
• Production: Automatically deployed to GitHub Pages

Customization

Adding Custom Viewport Sizes

Different websites may require different viewport sizes for optimal testing:

{
  "mobile-site": {
    "title": "Mobile Website",
    "links": ["https://mobile.example.com"],
    "viewport": {
      "width": 375,
      "height": 667
    }
  }
}

Custom Wait Times

Some websites may need longer load times:

{
  "slow-site": {
    "title": "Slow Loading Site",
    "links": ["https://slow.example.com"],
    "waitTime": 5000
  }
}

Extending the System

The modular architecture makes it easy to extend:

• New Comparison Methods: Modify compareImages() in the script
• Additional Configuration: Add new fields to entity configs
• Custom UI Components: Extend the React dashboard
• API Integration: Add endpoints for remote test triggering

Troubleshooting

Common Issues

1. Screenshots not captured:

   • Check if URLs are accessible
   • Verify viewport settings
   • Increase wait time for slow-loading sites

2. Dashboard not loading:

   • Ensure config.json exists and is valid JSON
   • Check that entities directory exists
   • Verify frontend dependencies are installed

3. GitHub Pages deployment issues:

   • Check GitHub Actions workflow
   • Verify repository permissions
   • Ensure build process completes successfully

Debug Mode

Enable debug logging by modifying the script:

// In scripts/visual-regression.js
const DEBUG = true;

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

License

MIT License - see LICENSE file for details.

Support

For issues and questions:

• Create an issue on GitHub
• Check the troubleshooting section
• Review the configuration examples

---

Built with ❤️ by the Themeisle team