-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] Nuclei Proxy Handling Issues: Ignores Concurrency Settings & Runs Extremely Slow #6029
Comments
I was using This means that nuclei doesn't respect proxy or is Non-Proxy-Aware. So what options do we have currently? I think this needs to be fixed from nuclei's side. If the proxy server supports thousands of requests per second then why to limit from our side? |
Thanks, @dwisiswant0, for adding those tags. Currently, I have around 25k URLs to scan with my custom templates through Nuclei. I don't want to do it with my own IP, so I’ve bought several hundred IPs from different providers—all of them are super fast and have no concurrency limits. But what else can I do? I know I could try creating a script out of those templates instead, but that would take a lot of time. I would really appreciate it if you could prioritize this issue. Alternatively, if you have any quick workarounds or possible solutions in the meantime, I’m open to trying them. |
@Nishantbhagat57 Given what you've shared, I couldn't find any details about the proxy service, logs, or metrics suggesting that nuclei requests are either slow or not being sent. Do you know if we can get this information? You also mentioned having different providers; is this issue occurring with all of them? |
@dogancanbakir I have already provided all the Steps to Reproduce this issue and that too in most easy way. Yes the issue is occurring with all of them. I have 3 different providers and as I said before, they don't impose any limits on concurrent requests or sessions and I have verified that too by opening hundreds of browsers at same time and with curl too. Okay, I will send you a screen recording in the next comment. But it would be great if you still try to reproduce this on your end. I can provide you the proxy, please DM me on discord, my username: |
@dogancanbakir @dwisiswant0 Here's the proof/details/metrics with recordings: Curl With Proxy: PROXY_WITH_CURL.MP410 seconds for 1000 requests - around 100 req / second Nuclei Without Proxy: 39 seconds for 424 requests - around 11 - 20 req / second which is still slow (I have set every frequency/concurrent value to high) Nuclei With Proxy: 6.5 minutes for 427 requests - around 1-2 req / second which is super super slow (I have set every frequency/concurrent value to high like above) I have changed the password of proxy, if you need it for testing on your side then please DM me on my discord |
Related to projectdiscovery#6029 Address proxy handling issues in Nuclei to respect concurrency settings and improve request speed. * **internal/runner/proxy.go** - Add logic to handle concurrency settings for proxy servers. - Ensure proxy settings respect concurrency values. * **internal/runner/runner.go** - Initialize the HTTP client with proxy settings and ensure concurrency is respected. - Add logic to handle concurrency settings for proxy servers. * **pkg/protocols/http/httpclientpool/clientpool.go** - Manage HTTP client pooling and enforce concurrency settings with proxies. - Ensure proxy settings respect concurrency values.
Is there an existing issue for this?
Current Behavior
Well, this is a complex one to explain, but I don't think it's hard to fix.
In simple words:
Expected Behavior
Nuclei should respect the configured concurrency values when using a proxy and maintain the same request speed as when running without a proxy. Proxy usage should not significantly degrade performance.
Steps To Reproduce
Note: Recording this issue would have taken more time, so I wrote it all down. However, I think I have provided everything in an easy-to-understand way so you can reproduce this issue easily from your side.
You will see it easily sends 1000 requests within 5-15 seconds. This confirms that the proxy is super fast and doesn't impose any limits (concurrent/rate limits) that could slow down a Nuclei scan.
It works great, right? You can verify the request speed on https://ssrf.cvssadvisor.com/instance/your-instance by observing the Requests Count value.
First, clear the logs on https://ssrf.cvssadvisor.com/instance/your-instance, then use the same command but with our fast proxy:
At the same time, observe the Requests Count value on https://ssrf.cvssadvisor.com/instance/your-instance webpage. You will notice that requests are being sent Super Slow (only 2-5 requests per second) despite having already set high concurrency limits in nuclei.
Want more assurance that this has nothing to do with the proxy provider? Let's use Nuclei with
proxychains-ng
instead.Testing With Proxychains-NG
A) Install
proxychains-ng
:B) Set up the
proxychains4.conf
file with this content:C) Clear the logs on https://ssrf.cvssadvisor.com/instance/your-instance
D) Run the same Nuclei command, but this time using
proxychains-ng
:E) At same time, observe the Requests Count value on the https://ssrf.cvssadvisor.com/instance/your-instance webpage. Bingo! Requests are now getting sent super fast, just like they were without a proxy—but this time with our proxy, thanks to
proxychains-ng
!Conclusion
This confirms that Nuclei has a major issue with proxy handling.
It doesn't respect concurrency values when using a proxy, making scans significantly slower than expected.
Would appreciate it if you could check this issue out. Let me know if you need further details!
@ehsandeep @dogancanbakir @DhiyaneshGeek @dwisiswant0
Relevant log output
Environment
Anything else?
No response
The text was updated successfully, but these errors were encountered: