A curated blogroll based on revskill's Ask HN thread "Could you share your personal blog here?". This application aggregates personal blogs from HN users and provides a modern interface to discover great content.
- 🔍 Smart Discovery: Browse blogs by author or discover recent posts across all blogs
- 📰 RSS Integration: Automatically fetches and stores posts from blog RSS feeds
- 🔄 Background Processing: Robust Sidekiq-based job system for data updates
- 🎯 Advanced Search: Search through blogs, posts, and content with Ransack
- 📊 Admin Interface: Comprehensive admin panel for blog management
- 🚀 API Support: RESTful API for blogs and posts
- 📈 HN Integration: Tracks HN karma and validates blog status
- 📤 Multiple Export Formats: CSV and OPML export capabilities
- Backend: Rails 7.0.5, Ruby 3.2.1
- Database: PostgreSQL (production), SQLite3 (development)
- Background Jobs: Sidekiq with Redis
- Frontend: Tabler UI with Bootstrap
- Search: Ransack with database indexes
- Caching: Redis-based caching system
- RSS Parsing: Feedjira with duplicate detection
- Ruby 3.2.1
- Rails 7.0.5
- Redis server
- PostgreSQL (for production)
-
Clone the repository
git clone https://github.com/your-repo/hn-blogroll.git cd hn-blogroll -
Install dependencies
bundle install
-
Setup database
rails db:create rails db:migrate
-
Start Redis server
redis-server
-
Start Sidekiq worker (in separate terminal)
bundle exec sidekiq -
Start the Rails server
rails server
-
Visit the application
- Main site: http://localhost:3000
- Sidekiq monitor: http://localhost:3000/sidekiq
- Admin interface: http://localhost:3000/admin
Use the provided Rake tasks to manually trigger updates:
# Fetch new blogs from HN thread
rake blogroll:fetch_hn_data
# Update all blog posts from RSS feeds
rake blogroll:update_posts
# Update specific blog posts
rake blogroll:update_blog_posts[blog_id]
# Update HN karma for all users
rake blogroll:update_karma
# Check online status of all blogs
rake blogroll:check_status
# Run all updates
rake blogroll:full_updateThe application automatically runs scheduled jobs:
- Every 6 hours: Fetch new blogs from HN thread
- Daily at 2 AM: Update all blog posts from RSS feeds
- Weekly on Sunday at 4 AM: Update HN karma for all users
- Weekly on Monday at 6 AM: Check blog online status
GET /api/v1/blogs- List all approved blogsGET /api/v1/blogs/:id- Get specific blog with postsGET /api/v1/blogs/:id/posts- Get posts for specific blog
GET /api/v1/posts- List all posts from approved blogsGET /api/v1/posts/:id- Get specific post
page- Page number for paginationper_page- Items per page (default: 20)q[field_name_cont]- Search within field (Ransack syntax)
Example API calls:
# Get recent posts
curl http://localhost:3000/api/v1/posts
# Search for posts containing "rails"
curl "http://localhost:3000/api/v1/posts?q[title_or_content_cont]=rails"
# Get blogs with pagination
curl "http://localhost:3000/api/v1/blogs?page=2&per_page=10"Access the admin interface at /admin to:
- Approve/reject submitted blogs
- Manually refresh posts for specific blogs
- Check blog status and update karma
- View comprehensive statistics
- Manage posts (view and delete if needed)
Security: The admin interface is protected by Devise authentication. Create your first admin user using the rake task:
# Create admin user (uses ADMIN_EMAIL and ADMIN_PASSWORD env vars)
rake admin:create
# Or create with custom credentials
[email protected] ADMIN_PASSWORD=yourpassword rake admin:create
# List all admin users
rake admin:listFor production deployment, set these environment variables:
# Database (PostgreSQL for production)
DATABASE_HOST=localhost
DATABASE_NAME=hn_blogroll_production
DATABASE_USERNAME=your_db_user
DATABASE_PASSWORD=your_db_password
# Redis
REDIS_URL=redis://localhost:6379/0
# Rails
RAILS_ENV=production
SECRET_KEY_BASE=your_secret_key_base
# Admin User Creation (for first setup)
[email protected]
ADMIN_PASSWORD=your_secure_passwordThe application uses Sidekiq for background processing. Jobs are scheduled using sidekiq-cron and configured in config/sidekiq_schedule.yml.
Monitor job performance and status at /sidekiq.
The application follows Rails conventions with these key components:
- Models:
BlogandPostwith proper associations and validations - Jobs: Sidekiq jobs for data fetching and processing
- Services: Business logic encapsulated in service classes
- Controllers: Separate namespaces for main site, admin, and API
- Background Processing: Scheduled jobs with error handling and logging
HackerNewsParser: Parses HN thread for new blog submissionsRssFeedParser: Fetches and processes RSS feeds with duplicate detectionKarmaUpdater: Updates HN karma using HN APIBlogStatusChecker: Validates blog accessibility
The application includes comprehensive error handling:
- Graceful failures with logging
- Retry mechanisms for transient errors
- Status tracking for blogs and posts
- Detailed error reporting in jobs
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
This project is open source and available under the MIT License.
For questions, suggestions, or issues:
- Create an issue on GitHub
- Contact: [email protected]
The stack: Rails 7 + Tabler UI + Sidekiq + Redis + PostgreSQL