Flemish data is not being processed

[joostschouppe]

Probably because the URL changed again

[jbelien]

I have indeed checked the URL but the download system has changed.
Instead of having a stable download URL, you receive now a "custom" URL: https://download.vlaanderen.be/product/10412-wegenregister-15122023

Do you have a contact there that could help us have the stable URL back ?

[joostschouppe]

Do you mean that the link this generates e.g. https://download.vlaanderen.be/bff/v1/Orders/165305/download/e944ca59-f726-4649-87e9-6e51fb2ad49f is changing all the time?
If not: with Wegenregister it has always been the case that the URL changed with every release (because of the release date in the file name)
Would it be an option to use the WFS instead? That's not exactly the best way to download bulk data, for traffic sign project that does work (though we did build in a loop to try until you get a full download)

[jbelien]

Yes, that's indeed what I mean.
I know before the link changed regularly but it was manageable because it was every 3,4 months when they published an update but with the new link I have no idea how long it is valid ; I wouldn't be surprise that that link is only valid for a few hours/days which makes the whole process impossible to manage.
It was exactly like that for Wallonia until we asked them to give us the "stable" link. Unfortunately, we need now to do the same for Vlaanderen.
I guess the WFS could be an option but it will require quite some refactoring to make it work.

[joostschouppe]

Since Wegenregister is now integrated in GRB, we can also use any service built on that product. The OGC api was recommended to me: https://www.vlaanderen.be/datavindplaats/catalogus/ogc-api-features-grb
The WFS would indeed be annoying; you would have to paginate through it for ever 10k cases or so.
What the folks at Wegenregister themselves recommended was to replicate the database, and use changefiles to keep it updated on our end.
Sounds like there are no easy solutions right now...

[jbelien]

I'll try to have a look at the OGC API! 👍

[jbelien]

I've run an update with the temporary URL ; update have been published.
Let's see how long the temporary URL is valid.

[jbelien]

(Only) 2 runs worked successfully:

• 2024-09-08T17:00:30+02:00
  https://github.com/osmbe/road-completion/actions/runs/10760897271
• 2024-09-15T14:04:03+02:00
  https://github.com/osmbe/road-completion/actions/runs/10870815716

The temporary URL seems only by valid for 7 days.