Migration to eat api #1422
Migration to eat api #1422
Conversation
|
This is still a very rough draft with lots missing, however I would still appreciate some feedback on the direction I have taken with changing the |
|
This looks great already! You're definitely going in the right direction. The only concern I have is the request size: This is a little more work but I think it would be best to load only the weeks that are needed right now (perhaps with a week or two padding). |
|
Good point, will look into that. |
|
So it seems like the For example
|
|
@joschahenningsen Okay, so Sadly that is still a lot larger than the old API, but since it is already only using 2 days padding for the future and one day backwards, I dont really know how to reduce it even further. Potentially only using 1 week could shave off another couple hundred KB, but would also be a lot more work to implement. |
|
Ahh thanks for clearing that up @Liqs-v2 The IOS app seems to load the canteen-specific week when needed: If that's too much work, all_ref should do as well though. |
|
Alright I ll take a look at it. This will probably be more difficult to do though, because the old API used to just load everything, meaning this will be more of a rewrite. |
|
But yea I agree loading for a canteen when needed is a better way of handling this. |
Lmk if there's any way I can assist here :) |
|
Seems like using Side note: I opened TUM-Dev/eat-api#109 |
|
Actually there is a thing that I need help with. The Eat-API uses a cafeteria slug as id (e.g: mensa-garching). That means I would need to migrate the |
|
Sure thing! The old api deployed is not open source, that's why we currently rebuild the backend over at https://github.com/TUM-Dev/Campus-Backend/ I can take care of a new endpoint that returns the right cafeteria ids, or we schedule a meeting and I show you around the backend if you want to implement this yourself :) Once I got #1423 to compile it should be fairly easy to implement a new endpoint. |
|
Lets have a meeting sometime during the weekend, preferably Saturday. Always good to know how to do that for the future :) |
Alright, I guess that means the only real way of doing it is rewriting to do lazy fetching. |
|
Saturday works for me, but I think https://tum-dev.github.io/eat-api/canteens.json could be used instead of adapting the API. |
|
That seems promising indeed, I ll see if I can change the fetching to use this enpoint instead. |
|
The switch from an integer cafeteriaId to a string cafeteriaId is causing me some problems, as it comes with a huge cascade of changes throught the entire project and database. Might it be better to introduce some sort of a mapping table for the old cafeteriaId to the new cafeteriaId and leave the components mostly alone? I am not sure what the best practice is in this case and would appreciate some input. @joschahenningsen |
Liqs-v2
force-pushed
the
issue/1370_switch_to_eat-api
branch
from
877a9e0
to
b58dd4a
Compare
|
@Liqs-v2 What exactly is still needed on this issue? |
|
Theres a bug causing the cafeteria card to not be shown which I am still investigating. Apart from that there are some other known issues that should be addressed once this PR is merged. Please be patient for a bit longer. |
|
I am sorry If you persieved my message as threatening. |
|
I didnt, no worries. All i meant to say is that I think it makes more sense to work on new PRs once this one is merged in, since it is rather old by now. For the other issues seperate PRs would probably be better to keep things small-ish. |
Liqs-v2
force-pushed
the
issue/1370_switch_to_eat-api
branch
from
0adfb27
to
edf9aca
Compare
|
Alright, so I finally found some time to take another look at this over the past few weeks. I found out why the CafeteriaCard is no longer showing in the Home (Landing) screen and also attempted to implement a fix, but now the Cafeteria component is no longer able to download the eat-api data either. The error seems like some certificate issue, so I was wondering if there were any changes to the eat-api in that regard. Regardless of this, because I atleast know which section is causing the issue and in the interest of time, I have decided to just put this PR up for review and add this issue to the list of known issues, that I would have put up as a disclaimer anyways. This means that now would be a good time to jump in and help, if ur still down for that @CommanderStorm :) |
|
The file permmissions for |
|
One of the reasons, why this issue exists you noticed, is that I dont think we currently give feedback to the user, that a menu is not availiable (e.g. mediziner mensa). |
Ah yea, I'll take another look at that. My project was being all weird since I am trying out Windows again since April and I had Linux before, so it didnt really like that. |
I'll try to see if this error can be reduced to occuring for certain cafeterias and menu availability. But it has been occuring for mensa-garching, which typically does pretty decently with providing menus. |
Yep, this is why I said one of the issues ;) |
|
Any updates on this topic? |
| @@ -2,18 +2,12 @@ | |||
| "formatVersion": 1, | |||
| "database": { | |||
| "version": 6, | |||
| "identityHash": "f743f786b0866afd26a4cc1c1fafc8d1", | |||
| "identityHash": "07b6e7b518ae4df546549d9a64c87963", | |||
There was a problem hiding this comment.
As far as I know, existing migrations should not be changed. Could you have another look into why this was changed?
|
No, not really. I tried what @CommanderStorm mentioned, but it didn't work and I also didn't have any other ideas. I can take another look in the time after my exam and before the semester starts, but quite frankly, my priorities have shifted and I have other commitments at this time and will probably not be contributing much in the coming semester. |
|
closing in favor of #1583 |
Issue
resolves: #1370
Why this is useful for all students
Migrating to the latest, still maintained API for getting cafeteria data ensures this component will keep functioning for the forseeable future.
Known issues
2022-06-15 16:06:12.712 14801-14876/de.tum.in.tumcampus E/AuthenticationManager$$ExternalSyntheticLambda1: javax.net.ssl.SSLHandshakeException: Chain validation failed javax.net.ssl.SSLHandshakeException: Chain validation failed at com.android.org.conscrypt.SSLUtils.toSSLHandshakeException(SSLUtils.java:362) at com.android.org.conscrypt.ConscryptEngine.convertException(ConscryptEngine.java:1134) at com.android.org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1089) at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:876) at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:747) at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:712) at com.android.org.conscrypt.ConscryptEngineSocket$SSLInputStream.processDataFromSocket(ConscryptEngineSocket.java:849) at com.android.org.conscrypt.ConscryptEngineSocket$SSLInputStream.access$100(ConscryptEngineSocket.java:722) at com.android.org.conscrypt.ConscryptEngineSocket.doHandshake(ConscryptEngineSocket.java:238) at com.android.org.conscrypt.ConscryptEngineSocket.startHandshake(ConscryptEngineSocket.java:217) at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.kt:379) at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.kt:337) at okhttp3.internal.connection.RealConnection.connect(RealConnection.kt:209) at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.kt:226) at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.kt:106) at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.kt:74) at okhttp3.internal.connection.RealCall.initExchange$okhttp(RealCall.kt:255) at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:32) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at de.tum.in.tumcampusapp.api.app.ChaosMonkeyInterceptor.intercept(ChaosMonkeyInterceptor.kt:27) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at de.tum.in.tumcampusapp.api.app.ApiHelper.lambda$getDeviceInterceptor$2(ApiHelper.java:97) at de.tum.in.tumcampusapp.api.app.ApiHelper.$r8$lambda$Ed6Z45oC4dmTmrD4C58bWLoLAoM(Unknown Source:0) at de.tum.in.tumcampusapp.api.app.ApiHelper$$ExternalSyntheticLambda1.intercept(Unknown Source:4) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at de.tum.in.tumcampusapp.api.app.ApiHelper.lambda$disableGzip$1(ApiHelper.java:72) at de.tum.in.tumcampusapp.api.app.ApiHelper.$r8$lambda$OUC4myTOllc53YNAK_DOFdZp_xU(Unknown Source:0) at de.tum.in.tumcampusapp.api.app.ApiHelper$$ExternalSyntheticLambda2.intercept(Unknown Source:0) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201) at okhttp3.internal.connection.RealCall.execute(RealCall.kt:154) at retrofit2.OkHttpCall.execute(OkHttpCall.java:204) at retrofit2.adapter.rxjava2.CallExecuteObservable.subscribeActual(CallExecuteObservable.java:46) at io.reactivex.Observable.subscribe(Observable.java:12267) at retrofit2.adapter.rxjava2.BodyObservable.subscribeActual(BodyObservable.java:35) at io.reactivex.Observable.subscribe(Observable.java:12267) at io.reactivex.internal.operators.observable.ObservableFlatMap$MergeObserver.subscribeInner(ObservableFlatMap.java:165) 2022-06-15 16:06:12.712 14801-14876/de.tum.in.tumcampus E/AuthenticationManager$$ExternalSyntheticLambda1: at io.reactivex.internal.operators.observable.ObservableFlatMap$MergeObserver.onNext(ObservableFlatMap.java:139) at io.reactivex.internal.operators.observable.ObservableDoOnEach$DoOnEachObserver.onNext(ObservableDoOnEach.java:101) at io.reactivex.internal.operators.observable.ObservableSubscribeOn$SubscribeOnObserver.onNext(ObservableSubscribeOn.java:58) at io.reactivex.internal.operators.observable.ObservableFilter$FilterObserver.onNext(ObservableFilter.java:52) at io.reactivex.internal.operators.observable.ObservableScalarXMap$ScalarDisposable.run(ObservableScalarXMap.java:248) at io.reactivex.internal.operators.observable.ObservableJust.subscribeActual(ObservableJust.java:35) at io.reactivex.Observable.subscribe(Observable.java:12267) at io.reactivex.internal.operators.observable.ObservableFilter.subscribeActual(ObservableFilter.java:30) at io.reactivex.Observable.subscribe(Observable.java:12267) at io.reactivex.internal.operators.observable.ObservableSubscribeOn$SubscribeTask.run(ObservableSubscribeOn.java:96) at io.reactivex.Scheduler$DisposeTask.run(Scheduler.java:578) at io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66) at io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.lang.Thread.run(Thread.java:923) Caused by: java.security.cert.CertificateException: Chain validation failed at com.android.org.conscrypt.TrustManagerImpl.verifyChain(TrustManagerImpl.java:724) at com.android.org.conscrypt.TrustManagerImpl.checkTrustedRecursive(TrustManagerImpl.java:554) at com.android.org.conscrypt.TrustManagerImpl.checkTrustedRecursive(TrustManagerImpl.java:575) at com.android.org.conscrypt.TrustManagerImpl.checkTrustedRecursive(TrustManagerImpl.java:620) at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:510) at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:428) at com.android.org.conscrypt.TrustManagerImpl.getTrustedChainForServer(TrustManagerImpl.java:356) at android.security.net.config.NetworkSecurityTrustManager.checkServerTrusted(NetworkSecurityTrustManager.java:94) at android.security.net.config.RootTrustManager.checkServerTrusted(RootTrustManager.java:90) at com.android.org.conscrypt.ConscryptEngineSocket$2.checkServerTrusted(ConscryptEngineSocket.java:161) at com.android.org.conscrypt.Platform.checkServerTrusted(Platform.java:250) at com.android.org.conscrypt.ConscryptEngine.verifyCertificateChain(ConscryptEngine.java:1644) at com.android.org.conscrypt.NativeCrypto.ENGINE_SSL_read_direct(Native Method) at com.android.org.conscrypt.NativeSsl.readDirectByteBuffer(NativeSsl.java:568) at com.android.org.conscrypt.ConscryptEngine.readPlaintextDataDirect(ConscryptEngine.java:1095) at com.android.org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1079) ... 58 more Caused by: java.security.cert.CertPathValidatorException: Response is unreliable: its validity interval is out-of-date at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:135) at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:222) at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:140) at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:79) at java.security.cert.CertPathValidator.validate(CertPathValidator.java:301) at com.android.org.conscrypt.TrustManagerImpl.verifyChain(TrustManagerImpl.java:720) ... 73 more 2022-06-15 16:06:12.713 14801-14876/de.tum.in.tumcampus E/AuthenticationManager$$ExternalSyntheticLambda1: Caused by: java.security.cert.CertPathValidatorException: Response is unreliable: its validity interval is out-of-date at sun.security.provider.certpath.OCSPResponse.verify(OCSPResponse.java:619) at sun.security.provider.certpath.RevocationChecker.checkOCSP(RevocationChecker.java:709) at sun.security.provider.certpath.RevocationChecker.check(RevocationChecker.java:363) at sun.security.provider.certpath.RevocationChecker.check(RevocationChecker.java:337) at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:125) ... 78 more Suppressed: java.security.cert.CertPathValidatorException: Unable to determine revocation status due to network error at sun.security.provider.certpath.RevocationChecker.checkCRLs(RevocationChecker.java:562) at sun.security.provider.certpath.RevocationChecker.checkCRLs(RevocationChecker.java:465) at sun.security.provider.certpath.RevocationChecker.check(RevocationChecker.java:394) ... 80 more Caused by: sun.security.provider.certpath.PKIX$CertStoreTypeException: java.io.IOException: Cleartext HTTP traffic to crl4.digicert.com not permitted at sun.security.provider.certpath.URICertStore.engineGetCRLs(URICertStore.java:430) at java.security.cert.CertStore.getCRLs(CertStore.java:190) at sun.security.provider.certpath.DistributionPointFetcher.getCRL(DistributionPointFetcher.java:245) at sun.security.provider.certpath.DistributionPointFetcher.getCRLs(DistributionPointFetcher.java:189) at sun.security.provider.certpath.DistributionPointFetcher.getCRLs(DistributionPointFetcher.java:121) at sun.security.provider.certpath.RevocationChecker.checkCRLs(RevocationChecker.java:552) ... 82 more Caused by: java.io.IOException: Cleartext HTTP traffic to crl4.digicert.com not permitted at com.android.okhttp.HttpHandler$CleartextURLFilter.checkURLPermitted(HttpHandler.java:127) at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:462) at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:411) at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:248) at sun.security.provider.certpath.URICertStore.engineGetCRLs(URICertStore.java:396) ... 87 moreI would suggest to tackle issue 1+2 in a single PR or potentially add the fix to this one and 3-5 as separate PRs.
Caveat
There probably are some performance issues with the downloading of the menus, that could potentially be moved into a seperate thread and done asynchronously. I have not implemented it in this way so far, due to my inexperience in Android and Kotlin development.
Screenrecordings
Migrated version (before issue 2 started occuring)