Advanced Categorization, Rate Limiting, and Feature Flags

[kevingatera]

Key Changes

• Added feature flag system for dynamic configuration
• Implemented robust rate limiter with token bucket algorithm
• Added new category suggestions with similarity matching
• Integrated web search for merchant context
• Improved transaction processing with batch support

Testing

• Enable features via FEATURES env var
• Test with dry run mode first
• Verify rate limiting with low API quotas
• Test web search with unfamiliar merchants

Configuration

All features configurable via environment variables as documented in README.md. Maintains backward compatibility while adding new capabilities.

[sakowicz]

This looks amazing. I will take a closer look and test it in following days.
I really appreciate your contribution 🍻

[kevingatera]

Thank you! Let me know what you'd need. Also you can get a valueserp API key with about 1000 requests for free so you can test

[kevingatera]

What do you think @sakowicz ? Needs more work? I put most work behind feature flags and most setups should not be affected

[sakowicz]

I'm sorry this is taking so long. The PR is really big and I've never had a chance to to test it yet.

However:

1. I don't see any Open Source/free alternative to web search. Can we implement some alternative so everyone can selfhosted it?
2. I think we could migrate CLASSIFY_ON_STARTUP, SYNC_ACCOUNTS_BEFORE_CLASSIFY and DRY_RUN to Feature Flags. What do you think?
3. "If the transaction is a Credit Card Payment, categorize it as "Transfer" unless it is a fee." This seems strange to me. Why is that? Seems like something that not everyone would've like as default.
4. RateLimiter class seems pretty complicated. Could you add some Unit tests to cover it?

Thank you for the contribution. I'm going to test it soon. I'm doing this CR from my phone 🙈

[kevingatera]

Those are valid points and I will be fixing them soon. I will look into implementing bing/duckduckgo

For the credit card, that's a mistake from my own rules bleeding into my pr. It's quite large and I kept needing this and that until everything depended on everything else so I pushed it here as a self contained featureset

[kevingatera]

I've made the changes and update, for search we will be using Duck Duck Go's public search, it's surprisingly good. It will be enabled through the freeWebSearch, updated the features except for dry_run. Finally, removed the credit card rule I had accidentally added.

[sakowicz]

Thank you for the contribution. And sorry that it took so long.

[sakowicz]

FYI: I've split the transaction-service into the multiple files bdc9ad1

If you've time it would be nice to unit test src/similarity-calculator.ts and src/category-suggestion-optimizer.ts

Transaction-service becomes huge and we'd have to refactor it if we want to maintain this project.
Sadly I have no time for that at the moment,