A browser-based JavaScript application that uses MLP (Multi-Layer Perceptron) autoencoders to detect anomalous market conditions in cryptocurrency trading data from Binance API.
This application implements an unsupervised learning approach to identify market manipulation patterns such as:
- Liquidation cascades
- Unusual funding spikes
- Abnormal volume patterns
- Price manipulation attempts
The system uses a neural autoencoder trained on "normal" market periods and flags periods with high reconstruction error as potential anomalies.
- Real-time Data Fetching: Direct integration with Binance API for live market data
- Batch Data Processing: Efficient handling of large date ranges with automatic batching
- MLP Autoencoder: Deep neural network architecture for pattern recognition
- Ensemble Autoencoder: Multiple model voting system for improved accuracy
- Advanced Feature Engineering: 12 comprehensive market features including RSI, Bollinger Bands, and market regime detection
- Flexible Date Ranges: Custom date range selection or default lookback periods
- Multiple Time Intervals: Support for 1h, 4h, 1d, and 1w intervals
- Data Quality Analysis: Automatic detection of missing data, zero volume, and corrupted records
- Raw Data Export: Download processed data as JSON files
- MinMax Normalization: Consistent data scaling for optimal training
- Multi-level Severity Classification: Normal, Warning, and Critical anomaly levels
- Sensitive Threshold Detection: Optimized thresholds for catching major market events
- Feature Contribution Analysis: Detailed breakdown of which market features contribute to anomalies
- Major Event Detection: Specialized algorithms for identifying crash patterns and market manipulation
- Historical Event Analysis: Automatic detection of significant market events
- Interactive Charts: Training progress, reconstruction errors, and severity distribution
- Price & Volume Timeline: Dual-axis charts showing price and volume trends
- Anomaly Timeline: Color-coded scatter plots showing anomaly distribution over time
- Severity Distribution: Doughnut charts showing anomaly breakdown
- Feature Contribution Charts: Visual analysis of feature importance
- Modern Dark Theme: Professional gradient-based UI with glassmorphism effects
- Responsive Design: Mobile-friendly interface with adaptive layouts
- Real-time Progress Tracking: Live progress bars and status updates
- Interactive Controls: Customizable training parameters and data range selection
- Data Sample Tables: Preview of raw and processed data
- Statistics Dashboard: Comprehensive data quality and performance metrics
- Model Persistence: Save and load trained models using IndexedDB
- Configuration Storage: Persistent storage of model parameters and thresholds
- Training History: Track and visualize training progress over epochs
- Early Stopping: Automatic training termination to prevent overfitting
- Memory Management: Efficient tensor disposal and memory cleanup
The autoencoder encoder currently uses 12 carefully selected market features optimized for anomaly detection:
-
priceChange- Close-to-close percentage change- Formula:
((close - previous_close) / previous_close) * 100 - Purpose: Captures basic price movements and trends
- Formula:
-
priceAcceleration- Second derivative of price changes- Formula:
priceChange - previous_priceChange - Purpose: Detects sudden price accelerations (crash patterns)
- Formula:
-
priceGap- Gap between opening price and previous close- Formula:
|open - previous_close| / previous_close * 100 - Purpose: Identifies overnight gaps and flash crashes
- Formula:
-
priceMomentum- 3-period price momentum- Formula:
(close - close_3_periods_ago) / close_3_periods_ago * 100 - Purpose: Captures medium-term price trends
- Formula:
-
volume- Raw trading volume (MinMax normalized)- Purpose: Market activity and liquidity indicator
-
volumeSpike- Volume spikes above 30% threshold- Formula:
max(0, volumeChange - 30) - Purpose: Detects unusual volume surges
- Formula:
-
volumeMomentum- 3-period volume momentum- Formula:
(volume - volume_3_periods_ago) / volume_3_periods_ago * 100 - Purpose: Captures volume trend changes
- Formula:
-
rsi- 14-period Relative Strength Index- Range: 0-100 (normalized to 0-1)
- Purpose: Momentum oscillator for overbought/oversold conditions
-
bollingerPosition- Position within Bollinger Bands- Range: -1 (lower band) to 1 (upper band)
- Purpose: Volatility and mean reversion indicator
-
marketRegime- Market trend detection- Values: -1 (downtrend), 0 (ranging), 1 (uptrend)
- Purpose: Contextual market state classification
-
fundingRateProxy- Simulated funding rate for spot markets- Formula:
(momentumFactor * 0.5 + volumeFactor * 0.3 + volatilityFactor * 0.2) * 0.01 - Purpose: Approximates futures funding rate behavior
- Formula:
-
openInterestProxy- Simulated open interest- Formula:
(volumeFactor * 0.4 + tradeIntensity * 0.3 + priceMomentum * 0.3) - Purpose: Estimates market positioning
- Formula:
For optimal performance with reduced complexity:
const coreFeatures = [
'priceChange', // Essential price movement
'volume', // Market activity
'priceAcceleration', // Sudden price changes
'volumeSpike', // Unusual volume
'priceGap', // Price gaps
'rsi', // Momentum oscillator
'bollingerPosition', // Volatility position
'marketRegime' // Trend context
];For comprehensive analysis with moderate complexity:
const enhancedFeatures = [
'priceChange', // Price movement
'volume', // Volume
'priceAcceleration', // Price acceleration
'volumeSpike', // Volume spikes
'priceGap', // Price gaps
'priceMomentum', // Price momentum
'volumeMomentum', // Volume momentum
'rsi', // RSI
'bollingerPosition', // Bollinger position
'marketRegime' // Market regime
];priceChange- Fundamental price movement indicatorpriceAcceleration- Critical for crash detectionvolumeSpike- Direct anomaly indicatorpriceGap- Flash crash detection
volume- Market activity baselinersi- Momentum contextbollingerPosition- Volatility contextmarketRegime- Trend context
priceMomentum- Redundant with priceAccelerationvolumeMomentum- Redundant with volumeSpikefundingRateProxy- Limited value for spot marketsopenInterestProxy- Simulated feature
When reducing features, adjust the autoencoder architecture:
// For 8 features
this.inputSize = 8;
this.encoderUnits = [12, 6, 3];
this.latentSize = 2;
this.decoderUnits = [3, 6, 12];
// For 10 features
this.inputSize = 10;
this.encoderUnits = [14, 7, 3];
this.latentSize = 2;
this.decoderUnits = [3, 7, 14];volatility- Intraday price range:((high - low) / close) * 100volatilityRatio- Current vs average volatility
volumeRatio- Current vs average volumebuySellRatio- Taker buy vs sell volume ratio
vwapDeviation- Deviation from Volume Weighted Average PricepriceRangePosition- Position within daily range:(close - low) / (high - low)
- Train models with different feature combinations
- Compare reconstruction errors and anomaly detection accuracy
- Analyze feature contribution scores for each anomaly
- Test on different time periods (bull/bear markets)
- Validate on different trading pairs
- Measure performance across various market conditions
- Reconstruction Error: Lower is better
- Anomaly Detection Rate: Balance between sensitivity and specificity
- False Positive Rate: Minimize false alarms
- Training Time: Faster training with fewer features
A comprehensive feature selection configuration file (feature-selection-config.js) is provided with:
- Predefined Feature Sets: Core, Enhanced, Minimal, and Alternative feature combinations
- Architecture Configurations: Optimized network architectures for different feature counts
- Feature Descriptions: Detailed formulas and importance rankings
- Testing Utilities: Ablation studies and cross-validation configurations
Usage Example:
import { CORE_FEATURES, ARCHITECTURE_CONFIGS } from './feature-selection-config.js';
// Use core features (8 features)
const config = ARCHITECTURE_CONFIGS[CORE_FEATURES.length];
this.inputSize = config.inputSize;
this.encoderUnits = config.encoderUnits;The application performs comprehensive EDA on cryptocurrency market data to understand patterns, detect anomalies, and ensure data quality before model training.
- Default Period: 60 days of historical data
- Time Intervals: 1h, 4h, 1d, 1w (configurable)
- Data Points: ~1,440 hourly records (60 days × 24 hours)
- Trading Pairs: BTC/USDT, ETH/USDT, BNB/USDT, ADA/USDT, SOL/USDT
- API Source: Binance REST API with batch processing support
The system automatically evaluates data quality with the following metrics:
- Data Completeness: Percentage of non-missing records
- Zero Volume Detection: Count of periods with zero trading volume
- Negative Price Detection: Count of invalid price records
- Missing Data Points: Count of incomplete OHLCV records
- Outlier Detection: Z-score threshold of 3.0 for extreme values
- Mean Price: Average closing price over the period
- Standard Deviation: Price volatility measure
- Min/Max Prices: Price range boundaries
- Price Change Distribution: Statistical analysis of price movements
- Mean Volume: Average trading volume
- Volume Volatility: Standard deviation of volume changes
- Volume Range: Min/max volume values
- Volume Change Distribution: Analysis of volume fluctuations
- Price Change Mean: Average percentage price change
- Price Change Std: Volatility of price changes
- Volume Change Mean: Average percentage volume change
- Volume Change Std: Volatility of volume changes
- RSI Analysis: 14-period Relative Strength Index distribution
- Bollinger Bands: Position analysis within volatility bands
- Market Regime: Trend detection (uptrend/ranging/downtrend)
- Price Acceleration: Second derivative analysis for crash detection
- Volume Spikes: Detection of unusual volume increases (>30%)
- Price Gaps: Analysis of overnight price gaps
- Momentum Indicators: 3-period price and volume momentum
The system automatically identifies significant market events:
- Major Price Drops: >5% price decreases (Warning: -5% to -10%, Critical: <-10%)
- Major Volume Spikes: >100% volume increases (Warning: 100-200%, Critical: >200%)
- Crash Patterns: Price drop + volume spike combinations
- Extreme Events: Sensitive detection for major market disruptions
- Critical Events: High-impact market disruptions
- Warning Events: Moderate market anomalies
- Normal Events: Regular market behavior
- Dual-axis Charts: Price (left) and normalized volume (right)
- Interactive Tooltips: Detailed point information
- Trend Analysis: Visual identification of market trends
- Histogram Analysis: Distribution of price and volume changes
- Box Plot Visualization: Quartile analysis and outlier detection
- Correlation Analysis: Relationships between different features
- MinMax Scaling: All features normalized to [0,1] range
- Feature Scaling: Consistent scaling across all 12 features
- Outlier Handling: Robust scaling to handle extreme values
- Training Data: 80% of chronological data
- Validation Data: 20% of chronological data
- Temporal Split: Maintains chronological order for time series
- Error Distribution: Statistical analysis of reconstruction errors
- Threshold Calculation: Mean + 1.5 × Standard Deviation
- Severity Classification: Normal/Warning/Critical levels
- Individual Contributions: Which features drive anomalies
- Pattern Recognition: Common anomaly patterns
- Root Cause Analysis: Understanding anomaly sources
- Missing Data Rate: Percentage of incomplete records
- Data Integrity: Validation of OHLCV consistency
- Temporal Continuity: Gap detection in time series
- Green (Good): >95% completeness, no data issues
- Yellow (Warning): 90-95% completeness, minor issues
- Red (Critical): <90% completeness, significant issues
- Uptrend: Strong positive price momentum with low volatility
- Downtrend: Strong negative price momentum with low volatility
- Ranging: High volatility with minimal directional movement
- Low Volatility: Stable price movements
- High Volatility: Erratic price movements
- Volatility Clustering: Periods of sustained high/low volatility
- Fetch Time: 2-5 seconds (single request), 10-30 seconds (batch)
- Processing Time: <1 second for feature engineering
- Memory Usage: ~50-100MB during processing
- Event Detection Rate: Sensitivity to major market events
- False Positive Rate: Minimization of false alarms
- Pattern Recognition: Accuracy in identifying market patterns
Price Statistics:
- Mean: $45,230.50 USDT
- Std Dev: $2,150.30 USDT
- Min: $38,900.00 USDT
- Max: $52,100.00 USDT
Volume Statistics:
- Mean: 1,250,000 units
- Std Dev: 450,000 units
- Min: 200,000 units
- Max: 3,500,000 units
Data Quality Assessment:
- Completeness: 98.5%
- Missing Data: 22 records
- Zero Volume: 0 records
- Negative Prices: 0 records
- Status: ✅ Excellent Quality
This comprehensive EDA framework ensures data quality, identifies patterns, and provides insights essential for effective anomaly detection in cryptocurrency markets.
- Batch Data Fetching: Handles large date ranges with automatic API batching (up to 1000 records per request)
- Multiple Time Intervals: Support for 1s, 1m, 3m, 5m, 15m, 30m, 1h, 2h, 4h, 6h, 8h, 12h, 1d, 3d, 1w, 1M intervals
- Advanced Feature Engineering: Calculates 12 comprehensive features:
- Price change percentage and acceleration
- Volume changes and spikes
- Funding rate proxy (simulated for spot markets)
- Open interest proxy (simulated for spot markets)
- Price gaps and momentum indicators
- Volume momentum analysis
- RSI (Relative Strength Index) calculation
- Bollinger Bands position
- Market regime detection (trending vs ranging)
- Data Quality Control: Automatic detection and reporting of data quality issues
- MinMax Normalization: Consistent scaling across all features for optimal training
- Historical Event Analysis: Built-in detection of major market events and crash patterns
- MLP Autoencoder: Multi-layer perceptron with encoder-decoder architecture
- Encoder: 12 → 16 → 8 → 4 → 2 neurons (compression to latent space)
- Decoder: 2 → 4 → 8 → 16 → 12 neurons (reconstruction)
- Dropout: 0.2 dropout rate for regularization
- Activation: ReLU for hidden layers, linear for output
- Ensemble Autoencoder: Multiple model voting system for improved accuracy
- 3 different autoencoder configurations
- Majority voting for anomaly classification
- Confidence scoring for ensemble predictions
- Training Optimization:
- Adam optimizer with learning rate 0.001
- Mean Squared Error loss function
- Early stopping with patience of 5 epochs
- Batch size optimization (default 64)
- Anomaly Detection:
- Statistical threshold calculation (mean + 1.5 × std)
- Multi-level severity classification (Normal/Warning/Critical)
- Feature contribution analysis for root cause identification
- Major event pattern recognition
- UI Management: Comprehensive interface for all user interactions
- Workflow Coordination: Orchestrates data fetching, training, and detection processes
- Real-time Visualization:
- Training loss charts with validation curves
- Reconstruction error timeline with color-coded severity
- Anomaly severity distribution (doughnut charts)
- Price and volume timeline with dual-axis display
- Data Visualization:
- Raw data sample tables
- Comprehensive statistics dashboard
- Data quality metrics and warnings
- Historical events analysis with October 10th special detection
- Model Persistence: Save/load functionality using IndexedDB and localStorage
- Memory Management: Proper tensor disposal and resource cleanup
- Open
index.htmlin a modern web browser - Ensure internet connection for Binance API access
- No additional installation required (uses CDN libraries)
- Select a trading pair from the dropdown (BTC/USDT, ETH/USDT, etc.)
- Choose data interval (1h, 4h, 1d, 1w)
- Configure date range:
- Default: Use lookback days (default: 60 days)
- Custom: Enable custom range and select start/end dates
- Click "Fetch Market Data" to download historical data
- Monitor batch processing progress for large date ranges
- Review comprehensive data summary and quality metrics
- Examine data sample table and statistics dashboard
- Adjust training parameters if needed:
- Epochs: Number of training iterations (default: 30)
- Batch Size: Training batch size (default: 64)
- Click "Train Autoencoder" to begin training
- Monitor real-time training progress and loss curves
- Early stopping will automatically terminate training if no improvement
- Training typically takes 1-3 minutes for standard model
- Review training history and model performance metrics
- Click "Detect Anomalies" after training completes
- Review the reconstruction error timeline chart with color-coded severity
- Examine the anomaly severity distribution (doughnut chart)
- Check the top anomalies table for specific timestamps and importance scores
- Analyze feature contributions for the most significant anomalies
- Review historical events analysis for major market events
- Examine price and volume timeline for context
- Save Model: Store trained weights and configuration locally using IndexedDB
- Load Model: Restore previously saved model and configuration
- Download Data: Export raw market data as JSON file
- Reset: Clear all data and start fresh with clean state
- Klines:
https://api.binance.com/api/v3/klines- Parameters:
symbol,interval,startTime,endTime,limit - Returns: OHLCV data with comprehensive market information
- Rate Limiting: 100ms delay between batch requests
- Maximum Records: 1000 per request (Binance API limit)
- Parameters:
The application processes raw Binance data into twelve normalized features:
- Price Change: Close-to-close percentage change
- Volume: Normalized trading volume
- Funding Rate Proxy: Simulated funding rate using price momentum and volume
- Open Interest Proxy: Simulated open interest using volume and trade activity
- Price Acceleration: Second derivative of price changes
- Volume Spike: Detection of unusual volume increases (>30%)
- Price Gap: Gap between opening price and previous close
- Price Momentum: 3-period price momentum indicator
- Volume Momentum: 3-period volume momentum indicator
- RSI: 14-period Relative Strength Index
- Bollinger Position: Position within Bollinger Bands (-1 to 1)
- Market Regime: Market trend detection (uptrend/ranging/downtrend)
- Training data reconstruction errors are analyzed using statistical methods
- Primary threshold: Mean + 1.5 × Standard Deviation
- Percentile-based threshold: 95th percentile of reconstruction errors
- Final threshold: Minimum of statistical and percentile methods for sensitivity
- This captures ~95% of normal market behavior while detecting major events
- Green (Normal): Error < threshold
- Yellow (Warning): threshold ≤ error < 1.2 × threshold
- Red (Critical): error ≥ 1.2 × threshold
For each anomaly, the system calculates which features contributed most to the reconstruction error, helping identify the root cause of unusual market behavior. The analysis includes:
- Individual Feature Contributions: Absolute difference between original and reconstructed values
- Feature Importance Ranking: Sorted by contribution magnitude
- Pattern Recognition: Detection of specific anomaly patterns (crashes, spikes, etc.)
- Major Event Classification: Special flags for significant market events
The system includes specialized algorithms for detecting major market events:
- Crash Patterns: Price drop + volume spike combinations
- Flash Crashes: Extreme negative price changes with massive volume
- Market Manipulation: Unusual volume spikes with minimal price change
- Liquidation Cascades: Multiple consecutive anomalies with increasing severity
- X-axis: Time points (chronological order)
- Y-axis: Reconstruction error magnitude
- Colors:
- Green dots: Normal market conditions
- Yellow dots: Elevated risk periods
- Red dots: Critical anomalies
- Gray line: Anomaly threshold
- Shows percentage breakdown of normal vs. anomalous periods
- Helps assess overall market stability
- Higher anomaly rates may indicate volatile market conditions
- Lists timestamps of most significant anomalies
- Shows exact reconstruction error values
- Provides severity classification
- Enables correlation with external market events
- Identifies which market features drove the anomaly
- Price Change: Unusual price movements
- Volume: Abnormal trading volume
- Funding Rate Proxy: Momentum-based anomalies
- Open Interest Proxy: Activity-based anomalies
- High volume with negative price change
- Elevated funding rate proxy
- Multiple consecutive anomalies
- Unusual volume spikes with minimal price change
- Abnormal funding rate patterns
- Sudden reversals in price direction
- Extreme negative price changes
- Massive volume spikes
- Critical severity classification
- TensorFlow.js: 4.22.0 (neural network framework)
- Chart.js: 4.4.1 (data visualization)
- Native Fetch API: Binance API communication
- IndexedDB: Model persistence storage
- LocalStorage: Configuration and metadata storage
- Modern browser with ES6+ support
- WebGL support for TensorFlow.js GPU acceleration
- IndexedDB support for model persistence
- Minimum 4GB RAM recommended
- Chrome 80+, Firefox 75+, Safari 13+, Edge 80+
- Data Fetching:
- Single request: ~2-5 seconds (up to 1000 records)
- Batch processing: ~10-30 seconds (large date ranges)
- Rate limiting: 100ms delay between batches
- Training Performance:
- Standard model: 1-3 minutes (30 epochs)
- Ensemble model: 3-9 minutes (3 models × 30 epochs)
- Early stopping: Automatic termination after 5 epochs without improvement
- Anomaly Detection: <1 second for validation data
- Memory Usage:
- During training: ~100-200MB
- During inference: ~50-100MB
- Tensor cleanup: Automatic disposal after operations
- Maximum Data Points: 1000 per API request (Binance limit)
- Batch Processing: Automatic splitting for large date ranges
- Feature Engineering: 12 comprehensive market indicators
- Normalization: MinMax scaling across all features
- Data Quality: Automatic validation and error reporting
- Input Features: 12-dimensional feature vectors
- Latent Space: 2-dimensional compressed representation
- Network Depth: 5 layers (encoder) + 5 layers (decoder)
- Total Parameters: ~1,000 trainable parameters
- Activation Functions: ReLU (hidden), Linear (output)
- Regularization: Dropout (0.2) and early stopping
- Glassmorphism Effects: Frosted glass appearance with backdrop blur
- Gradient Backgrounds: Professional blue-purple gradient theme
- Responsive Grid Layout: Adaptive design for all screen sizes
- Interactive Animations: Smooth hover effects and transitions
- Color-coded Elements: Consistent color palette for different data types
- Custom Date Range Selection: DateTime picker with validation
- Flexible Time Intervals: Support for multiple trading timeframes
- Training Parameter Adjustment: Real-time modification of epochs and batch size
- Data Range Options: Choice between custom dates or default lookback periods
- Model Management: Save/load functionality with persistent storage
- Real-time Progress Tracking: Live progress bars with percentage indicators
- Interactive Charts:
- Training loss curves with validation metrics
- Reconstruction error scatter plots with severity coloring
- Anomaly distribution doughnut charts
- Price/volume dual-axis timeline charts
- Data Quality Metrics: Comprehensive statistics with color-coded indicators
- Sample Data Tables: Preview of raw and processed market data
- Historical Events Analysis: Special detection for significant market events
- Status Messages: Real-time feedback on all operations
- Error Handling: User-friendly error messages with troubleshooting hints
- Loading States: Visual indicators during data processing and training
- Button State Management: Context-aware enabling/disabling of controls
- Memory Management: Automatic cleanup and resource optimization
- Check internet connection
- Verify Binance API is accessible
- Try different trading pair
- Check browser console for detailed errors
- Ensure sufficient system memory
- Try reducing batch size or epochs
- Close other browser tabs
- Restart browser if memory issues persist
- Normal market conditions (good sign!)
- Try different time periods
- Adjust threshold sensitivity in code
- Check if training completed successfully
- Ensure Chart.js loaded correctly
- Check browser console for JavaScript errors
- Try refreshing the page
- Verify WebGL support
- Use smaller batch sizes on slower devices
- Reduce training epochs for faster results
- Close unnecessary browser tabs
- Use Chrome/Firefox for best performance
- Real-time Monitoring Mode: Live data streaming with WebSocket integration
- Multiple Timeframe Analysis: Simultaneous analysis across different intervals
- Custom Threshold Adjustment: User-configurable anomaly detection sensitivity
- Advanced Export Functionality: CSV, JSON, and PDF report generation
- Additional Trading Pairs: Support for more cryptocurrency pairs
- Advanced Feature Engineering: Technical indicators like MACD, Stochastic, etc.
- Model Comparison Tools: Side-by-side comparison of different model architectures
- WebSocket Integration: Real-time data streaming from Binance
- Model Ensemble Methods: Advanced voting and stacking techniques
- Automated Hyperparameter Tuning: Grid search and Bayesian optimization
- Cloud Deployment Options: Docker containers and cloud hosting
- Mobile-responsive Design: Enhanced mobile experience
- Progressive Web App: Offline functionality and app-like experience
- API Integration: RESTful API for external system integration
- Pattern Recognition: Machine learning-based pattern identification
- Market Sentiment Analysis: Integration with social media and news data
- Risk Assessment: Portfolio-level risk analysis
- Backtesting Framework: Historical performance validation
- Alert System: Email and push notifications for anomalies
- Custom Dashboards: User-configurable visualization layouts
This is an educational project demonstrating autoencoder-based anomaly detection in financial markets. Contributions and improvements are welcome.
- Clone the repository
- Open
index.htmlin a local web server - Modify code as needed
- Test thoroughly before deployment
This project is for educational purposes. Please ensure compliance with Binance API terms of service when using their data endpoints.
- Binance API Documentation
- TensorFlow.js Documentation
- Chart.js Documentation
- Autoencoder Theory: Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks.