fingerprint for telegram

[bosoft-ESP]

Hello.
These last years, every 11 months, the fingerprint has been suddenly changed leaving our ESPs disconnected.
I am trying to implement a fingerprint server, but for that I need the ESP that acts as a server to read correctly the https://www.grc.com/fingerprints.htm?domain=api.telegram.org page for my ESPs to download it from the server, implemented in an ESP01s.
My idea is to read every 24H the Telegram finger on grc.com for when the ESPs need to update the finger. But there is no way to read correctly the page from grc. https.GET() results in HTTP_CODE_OK, but reading the web gives an empty String.

Any idea?
`/**
BasicHTTPSClient.ino

Created on: 20.08.2018

*/
// search telegram finger

//https://www.grc.com/fingerprints.htm?domain=api.telegram.org
// data: 27/03/23 to .... 27/02/24????
//const char* finger="8A:10:B5:B9:B1:57:AB:DA:19:74:5B:AB:62:1F:38:03:72:FE:8E:47"; //telegram

//#include <Arduino.h>

#include <ESP8266WiFi.h>
#include <ESP8266WiFiMulti.h>
#include <ESP8266HTTPClient.h>
#include <WiFiClientSecureBearSSL.h>

//data https://www.grc.com
const char* GRC_host = "https://www.grc.com/fingerprints.htm?domain=api.telegram.org";
const char fingerprint_GRC_com [] PROGMEM = "A6:8F:8C:47:6B:D0:DE:9E:1D:18:4A:0A:51:4D:90:11:31:93:40:6D";

#ifndef STASSID
#define STASSID "your ssid"
#define STAPSK "your pass"
#endif

ESP8266WiFiMulti WiFiMulti;

void setup() {

Serial.begin(115200);
// Serial.setDebugOutput(true);

Serial.println();
Serial.println();
Serial.println();

WiFi.mode(WIFI_STA);
WiFiMulti.addAP(STASSID, STAPSK);
Serial.println("setup() done connecting to ssid '" STASSID "'");
}

void loop() {
// wait for WiFi connection
if ((WiFiMulti.run() == WL_CONNECTED)) {

std::unique_ptr<BearSSL::WiFiClientSecure> client(new BearSSL::WiFiClientSecure);

//client->setFingerprint(fingerprint_GRC_com);
// Or, if you happy to ignore the SSL certificate, then use the following line instead:
client->setInsecure();

HTTPClient https;

Serial.print("[HTTPS] begin...\n");
if (https.begin(*client, GRC_host)) {  // HTTPS

  Serial.print("[HTTPS] GET...\n");
  // start connection and send HTTP header
  int httpCode = https.GET();

  // httpCode will be negative on error
  if (httpCode > 0) {
    // HTTP header has been send and Server response header has been handled
    Serial.printf("[HTTPS] GET... code: %d\n", httpCode);

    // file found at server
    if (httpCode == HTTP_CODE_OK) {
      String payload = https.getString();
      Serial.println("Init Ok");
      Serial.println(payload);  //<----- void string
      Serial.println("End");
    }
  } else {
    Serial.printf("[HTTPS] GET... failed, error %d: %s\n",httpCode, https.errorToString(httpCode).c_str());
  }

  https.end();
} else {
  Serial.printf("[HTTPS] Unable to connect\n");
}

}

Serial.println("Wait 20s before next round...");
delay(20000);
}
`
Thank you

[shurillu]

Hello bosoft-ESP,
mmm have you already tested the code with the fingerprint of https://www.grc.com/?
I mean without the setInsecure() member function.. maybe the grc server deny to respond without it..

A quick and dirty solution could be a config file stored in a internet connected server (no https or another protocol like FTP) where you keep updated the fingerprint.
Your ESP8266 check this file and update the telegram fingerprint accordingly (even when it can't connect to the telegram server).

Cheers,

Stefano

[bosoft-ESP]

Yes, I have tried the 2 ways with the same result.

Now I have a server mounted on a ESP01s, that when I call it, it answers the telegram fingerprint. But that makes me keep an eye on the change and recompile the server when it changes.
What I don't know is why grc.com answers with a blank page. or is there some other place where the telegram fingerprint is?

Thanks shurillu and greetings

[shurillu]

Mmmm do you have an ESP32? With little changes, I believe you can run the same code and it doesn't have the fingerprint issue.
Actually I can't have the opportunity to do the tests.

Stefano

[bosoft-ESP]

I hadn't thought about it. I think I have one, but it will take me a few days to test it. I'll tell you something

Regards

[bosoft-ESP]

Hello, at last, although it was not expected.
I have tested with an ESP32 and the result was:
with client->setInsecure() works correctly.
with client -> setCACert() always gives result -1
with client->setFingerprint() gives error: setFingerprint does not belong to client() class.

But there is something else: it fails (empty String) sometimes if date is not updated.

`const char* GRC_host = "https://www.grc.com/fingerprints.htm?domain=api.telegram.org";
char finger[70]="8A:10:B5:B9:B1:57:AB:DA:19:74:5B:AB:62:1F:38:03:72:FE:8E:47"; // telegem fingerprint

#include <WiFi.h>
#include <WiFiMulti.h>
#include <HTTPClient.h>
#include <WiFiClientSecure.h>

const char* ssid = "yor ssid";
const char* password ="your pass";

// only works well with client->setInsecure();
//const char fingerprint_GRC_com [] PROGMEM = "A6:8F:8C:47:6B:D0:DE:9E:1D:18:4A:0A:51:4D:90:11:31:93:40:6D";

// Not sure if WiFiClientSecure checks the validity date of the certificate.
// Setting clock just to be sure...
struct tm timeinfo;

void setClock() {
configTime(0, 0, "pool.ntp.org");

Serial.print(F("Waiting for NTP time sync: "));
time_t nowSecs = time(nullptr);
while (nowSecs < 8 * 3600 * 2) {
delay(500);
Serial.print(F("."));
yield();
nowSecs = time(nullptr);
}

Serial.println();
gmtime_r(&nowSecs, &timeinfo);
Serial.print(F("Current time: "));
Serial.print(asctime(&timeinfo));
}

WiFiMulti WiFiMulti;

void setup() {
Serial.begin(115200);
Serial.println();

WiFi.mode(WIFI_STA);
WiFiMulti.addAP(ssid,password);

// wait for WiFi connection
Serial.print("Waiting for WiFi to connect...");
while ((WiFiMulti.run() != WL_CONNECTED)) {
Serial.print(".");
}
Serial.println(" connected");

setClock();
}

void loop() {
WiFiClientSecure *client = new WiFiClientSecure;
if(client) {
//client -> setCACert(rootCACertificate); //result: httpCode=-1
//client->setFingerprint(fingerprint_GRC_com); //does not exist client->setFingerprint
client->setInsecure(); // works fine. ???
{
HTTPClient https;

  Serial.print("[HTTPS] begin...\n");
  if (https.begin(*client, GRC_host)) {  // HTTPS
    Serial.print("[HTTPS] GET...\n");
    // start connection and send HTTP header
    int httpCode = https.GET();

    // httpCode will be negative on error
    if (httpCode > 0) {
      // HTTP header has been send and Server response header has been handled
      Serial.printf("[HTTPS] GET... code: %d\n", httpCode);

      // file found at server
      if (httpCode == HTTP_CODE_OK || httpCode == HTTP_CODE_MOVED_PERMANENTLY) {
        String Finger,payload = https.getString();
		      String old=finger;
        //Serial.println(payload);
        int Pos=payload.indexOf("&#8212;</td><td>");
        if(Pos!=-1){
            Pos+=16; //add "&#8212;</td><td>"
            Finger=payload.substring(Pos,Pos+59); //Finger len=59
            Serial.print("Finger= ");
			        if(old!=Finger){
			              Serial.print(" CHANGED:= ");
			              strcpy(finger,Finger.c_str());
			              }
            Serial.println(Finger);
            }
      }
    } 
    else {
      Serial.printf("[HTTPS] GET... failed, error %d: %s\n",httpCode, https.errorToString(httpCode).c_str());
      }

    https.end();
  } 
  else {
    Serial.printf("[HTTPS] Unable to connect\n");
    }

  // End extra scoping block
}

delete client;

}
else {
Serial.println("Unable to create client");
}
Serial.printf("[HTTPS] program end");
while(1); // stop program
}`
It could be implemented in Ctbot, since it does not require the grc.com finger, but of course only for ESP32.

Greetings and thanks

[shurillu]

Hi bosoft-ESP,
the ESP32 SOC use another way to verify the telegram server certificate: check the file CTBotSecureConnection.h (spoiler: use the method setCACert).
For the reasons above, the ESP32 doesn't need the fingerprint.
Unfortunately, the ESP8266 can't perform the same task (low resources) so we must use the fingerprint.

Stefano

[bosoft-ESP]

But fingerprint has not worked with grc.com on an ESP01s either.
As for the certificate, it is always -1. The program was as is inside the ESP32 (no includes) and does not work with certificate.
In a few days I will test on a dated ESP01s. We will see what happens.
My interest is that each ESP01s will auto-update if when doing several testconnection it does not connect. Somehow, it would be "standalone". ESP32 is too "big" for some small things.

Regards

[bosoft-ESP]

but... why does ctbot work with fingerprint for telegram on a 8266?, and, why does fingerprint NOT work with grc.com? That's the dilemma.
I analyze tests done with ESP32 and want to move them to 8266. The one that surprised me the most was setInsecure() with the date.
I will try in a few days and let you know.

Regards

[shurillu]

Hi bosoft-ESP,
good news: digging around on internet I found a way to use the Telegram Server Certificate even with the ESP8266 SOC.
This mean no more fingerprint updates! With a simple sketch I managed to send a message without using the fingerprint.
I have to modify the library a bit, maybe the next week if I found time.

Stefano

[bosoft-ESP]

I have no words to express your great work.
I think I speak on behalf of many people: thanking you.

Greetings, and thanks for your great work

[shurillu]

Hello bosoft-ESP,
I've just done some changes on the library so it will be use the fingerprint anymore: now the library use the same certificate validation as ESP32.
Would you do some test with it? You can download the new version form the master branch.
I'll draft a new release (2.1.12) after some other tests.

Cheers

Stefano

[bosoft-ESP]

Yes, of course.
I'll test with an ESP01s and let you know.

Regards

[bosoft-ESP]

Hello shurillu.
I have already done some tests of sending and receiving text on an ESP01s, and they are going fine.
I have seen that you have put a call to NPT, could you make it accessible to be able to call it from our loop()?
As for the rest, I see that it is going perfect. In a few days I will install the new version in a bot that is working.

I am using:
IDE: 1.8.19
CTBot 1.1.12
ArduinoJson: 6.18.4
ESP8266 3.1.2

If you need more tests, let me know. I'll let you know.
Regards

[shurillu]

Hello bosoft-ESP,
nice to hear that all the tests are went fine. I have an ESP826 board running for hours without memory leaks, working simply fine.
Tomorrow I will draft a new release as it will be available in the Arduino Library manager.
The time sync with NTP call is necessary for the certificate validation: it simply set the NTP server to use.
You can retrieve the date/time with the time function: here a snippet:

	time_t now = time(nullptr);
	struct tm timeinfo;
	gmtime_r(&now, &timeinfo);
	Serial.printf("Current time: %s\n", asctime(&timeinfo));

Thank you very much for your test!
Regards,

Stefano

[bosoft-ESP]

Thanks for the note on date/time.
I will be watching for you to release the version and will reinstall it in case there are any last minute modifications.
Great work and more effort.

Greetings and many thanks

[bosoft-ESP]

Hello shurillu.
There is a little problem with the time zone.
In CTBotWifiSetup.cpp:
configTime(3 * 3600, 0, "pool.ntp.org", "time.nist.gov");
the 3 must change for each time zone and the 0 is for if daylight saving time is used. Central Europe would be 13600,0
Could you set a variable to adjust these 2 parameters (a=-12..+12, b=0..1) which would be (a3600,b*3600). Although there are more types (+10.5, +11.45, etc).

Check Wikipedia's 'List of UTC offsets'.

Regards

Translated with DeepL.com (free version)

[shurillu]

Hello bosoft-ESP,
for the certificate validation task, it does not need the time zone/daylight saving time.
According to what is written here: GlobalTrust, the current time is needed to verify the period of validity of the certificate. So little difference in terms of hours are not important.
If you want to set the ESP8266 time to your time zone/daylight saving, you can call the function that you pointed out:

configTime(3 * 3600, 0, "pool.ntp.org", "time.nist.gov");

with your local time zone/daylight saving: doing that, the time is synchronized to the time where you live and the library still works.

Cheers

Stefano

[bosoft-ESP]

ok, I will follow your directions, although I have already installed my modification on an ESP01s, and it seems to work both things correctly.

We will see in the next few days

Regards

[bosoft-ESP]

Hello, Stefano
I have updated the library to version 2.1.12 from IDE 1.8.19. I have compiled a sketch for ESP01s (esp8266) and it is OK.
I have tried to compile another sketch for an ESP32-WROOM-32D, putting the compiler for ESP32 WROOM Dev module and ESP32 dev module, and in both cases I get the same error. I have installed the version that I downloaded days ago, and the same error appears. I only tested with the ESP01s

`D:\arduino\libraries\CTBot\src\CTBotSecureConnection.cpp: In constructor 'CTBotSecureConnection::CTBotSecureConnection()':

D:\arduino\libraries\CTBot\src\CTBotSecureConnection.cpp:11:2: error: 'm_cert' was not declared in this scope

m_cert.append(m_CAcert);
^~~~~~
`

Greetings

PS: Compiling echoBot for ESP32, results in the same error

[bosoft-ESP]

I have solved the problem.
In line 11 of CTBotSecureConnection.cpp
`#if defined(ARDUINO_ARCH_ESP8266)

m_cert.append(m_CAcert);

#endif`

With that there is no problem with ES32

Regards

[shurillu]

Hello bosoft-ESP,

thank you so much! Bug corrected and I drafted a new release with your changes.
Thanks again.

Stefano

[Nevnevnev1]

Hello, is it running ok? All my ESP8266 don´t run, don´t connect.

Thank you.

[DumahBrazorf]

The fingerprint has changed few days ago.
If you update your project with the new version of CTBot you solve the issue once for good otherwise once a year or so you have to update the fingerprint in the beginning of the code. Search a row like this:

uint8_t m_fingerprint[20]{ 0x1F, 0x77, 0x5F, 0x20, 0xC5, 0xD3, 0xBD, 0x67, 0xDE, 0xE8, 0x07, 0x9B, 0x59, 0x1D, 0x22, 0xE9, 0xC0, 0xE4, 0x52, 0x4B };

You can find the fingerprint at this page: https://www.grc.com/fingerprints.htm?domain=api.telegram.org

[shurillu]

Hello Nevnevnev1,

as already said by DumahBrazorf, the new library version (2.1.13) fix the fingerprint problem for the ESP8266 SOC.
So, my advice is: update your project with the new version and the fingerprint issue should be done.

Cheers,

Stefano