Undetect chrome driver is being detected

[n1ckisthereu]

I have an application to search for some specific information from the cnpj here in Brazil, the application worked normally until the day before yesterday, but now the page is able to detect the driver.

The application does not present any captcha tests, but if you are detected, you cannot proceed with the request, and you will be faced with the following message:

I've tried everything, and I don't know what else to do...

Below is an excerpt from the part where you search for the CNPJ

class cnpj_utils:
    def __init__(self, cnpj, arguments: list = []):
        self.arguments = arguments
        self.driver: uc.Chrome = self.create_driver()
        self.cnpj = cnpj

    def create_driver(self) -> uc.Chrome:
        options = Options()
        options.add_argument("--headless")
        options.add_argument("--user-agent=Chrome/77")

        if len(self.arguments) > 0:
            for i in self.arguments:
                argument = i.split("=")
                options.add_argument(f"--{argument[0]}={argument[1]}")

        driver = uc.Chrome(
            browser_executable_path=keys["CHROMIUM_PATH"],
            options=options,
            version_main=123,
        )

        driver.get(config["url"])
        return driver

    def fill_cnpj(self):
        self.driver.get(config["url"])

        WebDriverWait(self.driver, 10).until(
            EC.presence_of_element_located(config["cnpj_el"])
        )

        el = self.driver.find_element(*config["cnpj_el"])

        el.click()
        el.send_keys(self.cnpj)

    def submit_form(self):
        WebDriverWait(self.driver, 10).until(
            EC.presence_of_element_located(config["submit_form"])
        )

        self.driver.find_element(*config["submit_form"]).click()

    def extract_data(self) -> object:
        WebDriverWait(self.driver, 10).until(
            EC.presence_of_element_located(config["cnpj_val"])
        )

        cnpj = self.driver.find_element(*config["cnpj_val"])
        company = self.driver.find_element(*config["company_name"])
        simples_nacional = self.driver.find_element(*config["simples_nacional"])
        simei = self.driver.find_element(*config["simei"])

        simples_nacional = (
            False
            if simples_nacional.get_property("innerText")
            == "NÃO optante pelo Simples Nacional"
            else True
        )
        simei = (
            False
            if simei.get_property("innerText") == "NÃO enquadrado no SIMEI"
            else True
        )

        return dumps(
            {
                "cnpj": cnpj.get_property("value"),
                "company_name": company.get_property("innerText"),
                "simples_nacional": simples_nacional,
                "simei": simei,
            }
        )

url: https://consopt.www8.receita.fazenda.gov.br/consultaoptantes

[unlucky-kit]

undetected-chromedriver has nothing to do with you being detected.

First, your user-agent is just Chrome/77. A typical user agent should look like this:
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36

Second, you are running your automation headlessly. It is quite easy for websites to detect headless automations.

How to fix:

1. Use the user agent I provided. That user-agent is from Windows, so if you are using Mac or Linux it is advisable to use user agents that reflect your OS. (Just open google chrome normally and type "what is my user agent?").
2. Remove --headless argument. If the computer on which you are running your automation does not have a physical monitor attached to it, you can look into XVFB in python.

If that doesn't work, you may have ruined your IP's reputation, so try accessing the website using a proxy.

[n1ckisthereu]

@unlucky-kit I tried to follow the steps above, changed the user agent, used a proxy, VPN, even changed networks and machines, but I was not successful, the user agent was like this because I had a bug previously and this helped me resolve it, but even changing everything, and taking it out of headless mode, I am detected :(

[n1ckisthereu]

@unlucky-kit, you can help me ?