RTCM messages not recognized by PyGPSClient

[pat-2010]

I just switched to CRTN (California Real Time Network) from rtgpsout.earthscope.org to be able to access a base station that is closer to my house. When I start the NTRIP client with PyGPSClient, no RTCM messages show up on the console or in the log. Hopefully someone else has had the same issue and can point me in the right direction.

I'm using:
RPi 5, 4gb, NVME drive
PyGPSClient 1.4.24
simpleRTK2B Budget

I used my iPad using SWmaps, connected to the CRTN server and did receive a RTK FIX status, showing that SWMaps was interpreting and forwarding the RTCM messages to my receiver. I also used u-center on my desktop and saw NTRIP message reception.

I used wireshark on my RPi and saw messages once per second of the appropriate size after RTCM was enabled using PyGPSClient. I scanned the messages from the server and they had 1004 and 1012 imbedded in about the right locations to indicate I was getting the 1004 and 1012 messages as expected.

I noticed that the CRTN server also has access to the station I was connecting to previously (P276) using the rtgpsout.earthscope.org server. I tried connecting to P276 through CRTN and saw the same problem. When I went back to P276 using the rtgpsout.earthscope.org server the RTCM messages were showing up on the PyGPSClient console.

I ran experiment where I connected to a station using my original server then switched to the new CRTN server. I have a PyGPSClient log file showing RTCM output when I was connected using the original server but no output when I was connected to the CRTN server. I had Wireshark set up and saw message streams from both servers at the appropriate time. I have the Wireshark log if it would be of use but it is 14.5MB so I didn't upload it. And my RPi just crashed and I think the file is corrupted. Before that happened I noticed that the message sequence from the 2 servers was different. The old server sends 2 messages per second. The first one is short with a 5 byte data segment (31 35 41 0d 0a). The second one is much longer and I believe has both the 1004 and 1012 messages. The new server does not send the first short message. If the Wireshark .pcapng file would be of use I could redo the test. This time I'll learn how to do a capture filter in Wireshark and won't have a 14.5MB file.

Unfortunately, CRTN is not free. Well it is free to use, but it is a $100 donation to sign up. Let me know if I can run any tests that would be useful.

This may only affect people in California that want to spend the $100 to use the CRTN service. I expect it would be low on your priority list. No worries.

I also read other discussion threads here which gave me a few ideas. Tomorrow I'll try to set up gnssntripclient CLI with 2 instances, one for each server, both using the same base station. Output routed to a log file. I'll also use Wireshark to capture the full message traffic.

Update.
I set up gnssntripclient with a logfile set to a verbosity of debug. I connected one gnssntripclient to station P276 through the new server (CRTN) and a second gnssntripclient to station P276 from the old server (rtgpsout.earthscope.org).

New server:
gnssntripclient -S 132.239.152.175 -P 2104 -M 'P276_RTCM3' --ntripuser 'xxxxx' --ntrippassword 'xxxx' --logtofile '/home/pi/crtntestnew.log' --verbosity 3 --clioutput 1 --output '/home/pi/crtnoutfilenew.log'

Old server:
gnssntripclient -S 3.23.196.92 -P 2101 -M 'P276_RTCM3' --ntripuser 'xxxxxx' --ntrippassword 'xxxxx' --logtofile '/home/pi/crtntestold.log' --verbosity 3 --clioutput 1 --output '/home/pi/crtnoutfileold.log'

I have attached the log file from both runs. The log file from the old server shows 1004 and 1012 messages. The log file from the new server does not.
crtntestnewserver.log
crtntestoldserver.log

The binary file out from the new server was empty.

I also attached the wireshark capture for 5 or so seconds. Well I tried to but the app doesn't support that file type. I have it if we can find a way to get it to you. Here's a capture of a few seconds worth of data from both servers.

Note that the main message length from the old server is 18 bytes shorter and that the old server sends that extra message. Here's what I could easily clip out of Wireshark for one of those extra messages:

Frame 15: 73 bytes on wire (584 bits), 73 bytes captured (584 bits) on interface any, id 0
Section number: 1
Interface id: 0 (any)
Encapsulation type: Linux cooked-mode capture v1 (25)
Arrival Time: Nov 3, 2024 22:17:04.371586174 Pacific Standard Time
UTC Arrival Time: Nov 4, 2024 06:17:04.371586174 UTC
Epoch Arrival Time: 1730701024.371586174
[Time shift for this packet: 0.000000000 seconds]
[Time delta from previous captured frame: 0.303148752 seconds]
[Time delta from previous displayed frame: 0.303148752 seconds]
[Time since reference or first frame: 1.928478051 seconds]
Frame Number: 15
Frame Length: 73 bytes (584 bits)
Capture Length: 73 bytes (584 bits)
Linux cooked capture v1
Internet Protocol Version 4, Src: 3.23.196.92, Dst: 192.168.1.80
Transmission Control Protocol, Src Port: 2101, Dst Port: 55334, Seq: 675, Ack: 1, Len: 5
Source Port: 2101
Destination Port: 55334
[Stream index: 0]
[Stream Packet Number: 9]
[Conversation completeness: Incomplete (12)]
[TCP Segment Len: 5]
Sequence Number: 675 (relative sequence number)
Sequence Number (raw): 2341200883
[Next Sequence Number: 680 (relative sequence number)]
Acknowledgment Number: 1 (relative ack number)
Acknowledgment number (raw): 3985272067
1000 .... = Header Length: 32 bytes (8)
Flags: 0x018 (PSH, ACK)
Window: 488
[Calculated window size: 488]
[Window size scaling factor: -1 (unknown)]
Checksum: 0x6f3c [unverified]
[Checksum Status: Unverified]
Urgent Pointer: 0
Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), Timestamps
[Timestamps]
[SEQ/ACK analysis]
TCP payload (5 bytes)
Data (5 bytes)
Data: 3134410d0a
[Length: 5]

[semuadmin]

So very quick test using nominal credentials and no mountpoint (which I would expect to return a sourcetable response)...

gnssntripclient -S 132.239.152.175 -P 2104  --ntripuser 'xxxxx' --ntrippassword 'xxxx' --verbosity 3

2024-11-04 08:09:52.551 - DEBUG - pygnssutils.gnssntripclient - Response: {'protocol': 'SOURCETABLE', 'code': 200, 'description': 'OK'}
{}
... loops endlessly

The fact that the protocol is SOURCETABLE rather than HTTP/1.1 tells me straight away that the new server is running NTRIP 1.0, not NTRIP 2.0. Trying again with the --ntripversion 1.0 flag:

gnssntripclient -S 132.239.152.175 -P 2104  --ntripuser 'xxxxx' --ntrippassword 'xxxx' --verbosity 3 --ntripversion 1.0

2024-11-04 08:36:40.505 - DEBUG - pygnssutils.gnssntripclient - Response: {'protocol': 'SOURCETABLE', 'code': 200, 'description': 'OK'}
{}
2024-11-04 08:36:40.647 - INFO - pygnssutils.gnssntripclient - Sourcetable:
Server: Geodetics CommLinkProxy Version 3.5 EV
Date: 04/Nov/2024:08:36:40 UTC
Content-Type: text/plain
Content-Length: 23289

CAS;132.239.152.175;2104;Geodetics CommLinkProxy;SOPAC;0;USA;32.84;-117.25;http://sopac.ucsd.edu/projects/realtime
NET;CRTN;SOPAC;B;N;http://sopac.ucsd.edu/input/realtime;http://sopac.ucsd.edu/input/realtime/CRTN_Access.xls;[email protected];none
STR;DIXN_RTCM3;DIXN:DixonCity_CA2012;RTCM 3.0;1004,1006,1008,1012,1033;2;GPS+GLONASS;CRTN;USA;38.47;-121.83;0;0;CRTN_server/CommLinkProxy;none;B;N;4000;none
STR;HCRO_RTCM3;HCRO:HCRO_BARD_CN2005;RTCM 3.0;1004,1006,1008,1033;2;GPS;CRTN;USA;40.82;-121.47;0;0;CRTN_server/CommLinkProxy;none;B;N;4000;none
...etc.

Suggest trying your tests again with --ntripversion 1.0

[pat-2010]

Thanks again for your quick responses.

I ran gnssntripclient with --ntripversion 1.0 using the new server (CRTN) and it worked. At least I saw reception of 1004 and 1012 messages once per second. I looked at PyGPSclient and the cli option to set the ntripversion is not supported. I checked the pygpsclient.json config file and couldn't see anything there to set the ntrip version. Can PyGPSclient be set up to run using NTRIP 1.0? I'm also going to talk to CRTN to see if I can get ntrip version 2.0 turned on. I saw one presentation where the device they were using had a check box for NTRIP and a second check box for NTRIP 1.0. They had the NTRIP box checked. That implies they are sending NTRIP 2.0, which doesn't match what we are seeing. I did find a user's forum hidden in their documentation so I'll sign up for it and see if I can find anything there.

[semuadmin]

The NTRIP version can be set in the pygpsclient.json configuration file using the ntripclientversion_s setting, and this does appear in the sample config file provided. It defaults to "2.0". There is no PyGPSClient command line option for this setting, but it can be saved along with all other configuration settings via the 'Save Configuration' menu option.

(I realised there is currently no NTRIP version option in the gnssstreamer CLI utility, which is an omission I'll rectify in the next version - it has no bearing on the issue at hand)

It would be possible for an NTRIP client to 'auto-detect' the NTRIP version once a preliminary response is received, or 'fall back' to an NTRIP 1.0 service if NTRIP 2.0 is not available (u-center may well do this), but I'm loath to do this here as there may be good reasons why a client would explicitly request one or the other. Plus, as I say, NTRIP 1.0 is long deprecated.

if you're happy with this response, please tick the 'Mark as answer' box - thanks.

[pat-2010]

I don't know how I missed this the first time. I set the parameter to 1.0 and now am seeing RTCM messages in PyGPSClient. I believe I was looking for ntripversion and with insufficient caffeine I missed ntripclientversion_s.

[semuadmin]

I don't know how I missed this the first time. I set the parameter to 1.0 and now am seeing RTCM messages in PyGPSClient. I believe I was looking for ntripversion and with insufficient caffeine I missed ntripclientversion_s.

Lack of caffeine has caused many a hiccup on my part, so don't beat yourself up :-)

[semuadmin]

FYI Apologies if you're already well aware of this, but just to highlight the clear differences between NTRIP 1.0 and NTRIP 2.0 which you can see if you use gnssntripclient with --verbosity 3 (which logs the complete HTTP request and response - no need for wireshark!)

While both protocols are ostensibly based on HTTP/1.n, NTRIP 1.0 uses obsolete protocol classifiers in its HTTP response header to signify 'sourcetable' or 'data', whereas NTRIP 2.0 uses an explicit content-type: label.

Note also that NTRIP 2.0 can take advantage of chunked and/or zipped encoding, whereas NTRIP 1.0 cannot.

NTRIP 1.0 sourcetable response:

2024-11-04 08:51:35.876 - DEBUG - pygnssutils.gnssntripclient - Response: {'protocol': 'SOURCETABLE', 'code': 200, 'description': 'OK'}

NTRIP 1.0 data response:

2024-11-04 08:52:34.988 - DEBUG - pygnssutils.gnssntripclient - Response: {'protocol': 'ICY', 'code': 200, 'description': 'OK'}

('ICY' is supposedly an acronym for 'I Can Yell' (aka 'Shoutcast'), which was used in the early days of the HTTP protocol for streaming data services - it's now deprecated under HTML 5)

NTRIP 2.0 sourcetable response:

2024-11-04 08:54:03.835 - DEBUG - pygnssutils.gnssntripclient - Response: {'protocol': 'HTTP/1.1', 'code': 200, 'description': 'OK'}
{'ntrip-version': 'Ntrip/2.0', 'ntrip-flags': 'st_filter,st_auth,st_match,st_strict,rtsp,plain_rtp', 'server': 'NTRIP BKG Caster/2.0.45', 'date': 'Mon, 04 Nov 2024 08:58:48 GMT', 'connection': 'close', 'content-type': 'gnss/sourcetable', 'content-length': '53892'}

NTRIP 2.0 data response:

2024-11-04 08:55:01.474 - DEBUG - pygnssutils.gnssntripclient - Response: {'protocol': 'HTTP/1.1', 'code': 200, 'description': 'OK'}
{'ntrip-version': 'Ntrip/2.0', 'server': 'NTRIP BKG Caster/2.0.45', 'date': 'Mon, 04 Nov 2024 08:55:01 GMT', 'cache-control': 'no-store,no-cache,max-age=0', 'pragma': 'no-cache', 'connection': 'close', 'content-type': 'gnss/data', 'transfer-encoding': 'chunked'}

NTRIP 1.0 was originally published in 2004 and has been deprecated for nearly a decade now - from the RTCM Best Practice guidelines:

Modern NTRIP casters really should have upgraded by now, especially if they're asking for payment.

[pat-2010]

Thanks for the detailed information. I'm just starting using RTK GPS so there is a lot that I am not aware of.

I checked on u-center (ublox utility) and they use NTRIP version 1. In wireshark the initial connection info on the new server is GET / HTTP/1.0. Now I know why the new server works there. And I confirmed that u-center requests HTTP 1.0 on the old server as well while PyGPSclient requests HTTP 1.1. I'm assuming that HTTP 1.0 vs 1.1 is tied to NTRIP 1.0 vs 2.0.

[semuadmin]

Thanks for the detailed information. I'm just starting using RTK GPS so there is a lot that I am not aware of.

You're welcome. It is a complex area with lots of potential pitfalls.

I checked on u-center (ublox utility) and they use NTRIP version 1. In wireshark the initial connection info on the new server is GET / HTTP/1.0. Now I know why the new server works there. And I confirmed that u-center requests HTTP 1.0 on the old server as well while PyGPSclient requests HTTP 1.1. I'm assuming that HTTP 1.0 vs 1.1 is tied to NTRIP 1.0 vs 2.0.

Not exactly. An NTRIP 2.0 service request is signified by an explicit Ntrip-Version: Ntrip/2.0 tag in the HTTP request header, which is missing from an NTRIP 1.0 request (you can see this using --verbosity 3 logging - you don't necessarily need Wireshark). It is not necessarily a consequence of the HTTP/1.n minor version, though some clients may choose to differentiate (I can't speak for u-center specifically).

NTRIP 2.0 request header:

GET /DARE00GBR0 HTTP/1.1
Host: euref-ip.net:443
User-Agent: NTRIP pygnssutils/1.1.6
Authorization: Basic ******************************=
Ntrip-Version: Ntrip/2.0
Accept: */*
Connection: close

NTRIP 1.0 request header:

GET /DARE00GBR0 HTTP/1.0
Host: euref-ip.net:443
User-Agent: NTRIP pygnssutils/1.1.6
Authorization: Basic ******************************=
Accept: */*
Connection: close

If you're intererested in the implementation details, the definitive NTRIP protocol documentation costs a princely $345, but the NTRIP Client Devices/Best Practices guidelines are available as a free download. I've endeavoured to follow them to the letter in PyGPSClient and pygnssutils.

[semuadmin]

Just to clarify - there's no significant functional difference between NTRIP 1.0 and NTRIP 2.0 from an RTK perspective - the RTK results will be identical.

As I say, NTRIP 2.0 has two practical advantages:

1. ICY is already deprecated and will eventually be dropped, so NTRIP service providers will have to upgrade at some point
2. NTRIP 2.0 supports chunked and/or zipped encoding, which potentially improves connection stability and uses less bandwidth (not a significant issue for most internet users, but potentially useful if you're out in the field using e.g. a low-bandwidth / high-latency satellite link)

[semuadmin]

@pat-2010 FYI I've introduced a update to gnssntripclient in pygnssutils v1.1.7. It will now tolerate an NTRIP 1.0 response to an NTRIP 2.0 request, or vice versa.

[pat-2010]

thanks