Skip to content

Add Protobuf/gRPC-based EtsIR loader #318

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 35 commits into
base: neo
Choose a base branch
from
Open

Add Protobuf/gRPC-based EtsIR loader #318

wants to merge 35 commits into from

Conversation

Lipen
Copy link
Member

@Lipen Lipen commented May 5, 2025

This PR introduces a Protobuf-based EtsIR loader that communicates with the local ArkAnalyzer gRPC server, replacing the previous JSON-based approach.

Currently, this PR contains two different implementations of Protobuf models of EtsIR: auto-generated (via Gradle plugin that wraps protoc) and manual-written (with kotlinx-serialization-protobuf).

  • We have to decide which one is more convenient for us.
  • We also need to take a look at Wire --- it can auto-generate Kotlin classes (which are much better than Java classes from protoc) from .proto models.
  • After selecting the desired final solution, we are going to clean up this PR.

In addition, this PR contains the implementation of an HTTP client (using Ktor) for sending requests to a HTTP ArkAnalyzer server, internally using either JSON or ProtoBuf format. This shows the benefits of having the proto-model written with kotlinx-serialization, and not standalone auto-generated files. However, currently there is no simple way of easily switching between HTTP and gRPC protocols, so by using Ktor as a client, we are limited to HTTP-based solutions --- most importantly, from the ArkAnalyzer side as well, where we already began integrating gRPC server (after discussing that gRPC is indeed more suitable and future-proof for our means). Thus, we probably end up deleting all Ktor-related code from this PR, and the kx-serialization model as well, since it can't be (easily) used in auto-generated gRPC client/server stubs instead of the auto-generated messages.

Overall, we have two possible final solutions:

  • Official protobuf and grpc tools. Auto-generated (protoc) Java classes for model, Kotlin type-safe wrappers, auto-generated gRPC client/server stubs. Works with Java 8.
  • Wire. Similar to the first solution, but different. Also provides auto-generated model (pretty convenient Kotlin classes via kotlin-poet), client/server stubs. Requires Java 11.

Note: the description above mentions both client and server, though in this particular PR we only have to introduce a client for the ArkAnalyzer gRPC server. However, the same arguments will be applicable when we are going to develop the USVM server for the type inference. This PR is just a playground for all these possibilities.

@Lipen Lipen requested a review from CaelmBleidd May 5, 2025 15:53
Copy link
Contributor

github-actions bot commented May 5, 2025

Lifecycle test results

0 tests   - 78   0 ✅  - 78   0s ⏱️ - 1m 28s
0 suites  -  7   0 💤 ± 0 
0 files    -  7   0 ❌ ± 0 

Results for commit b84bc72. ± Comparison against base commit bccc018.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented May 5, 2025

Test results on JDK 8

0 files   -  68  0 suites   - 68   0s ⏱️ - 4m 32s
0 tests  - 569  0 ✅  - 557  0 💤  - 12  0 ❌ ±0 
0 runs   - 597  0 ✅  - 585  0 💤  - 12  0 ❌ ±0 

Results for commit b84bc72. ± Comparison against base commit bccc018.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented May 5, 2025

Test results on JDK 19

 68 files  ±0   68 suites  ±0   4m 35s ⏱️ -3s
569 tests ±0  560 ✅ ±0  9 💤 ±0  0 ❌ ±0 
597 runs  ±0  588 ✅ ±0  9 💤 ±0  0 ❌ ±0 

Results for commit b84bc72. ± Comparison against base commit bccc018.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented May 5, 2025

Test results on JDK 11

 68 files  ±0   68 suites  ±0   5m 4s ⏱️ -6s
569 tests ±0  562 ✅ ±0  7 💤 ±0  0 ❌ ±0 
597 runs  ±0  590 ✅ ±0  7 💤 ±0  0 ❌ ±0 

Results for commit b84bc72. ± Comparison against base commit bccc018.

♻️ This comment has been updated with latest results.

Copy link

codecov bot commented May 7, 2025

Codecov Report

Attention: Patch coverage is 49.15663% with 422 lines in your changes missing coverage. Please review.

Project coverage is 69.55%. Comparing base (77b83e4) to head (476ed40).

Files with missing lines Patch % Lines
...c/main/kotlin/org/jacodb/ets/proto/ConvertToEts.kt 76.05% 72 Missing and 30 partials ⚠️
...main/kotlin/org/jacodb/ets/proto/EtsTypeToProto.kt 0.00% 59 Missing ⚠️
...main/kotlin/org/jacodb/ets/proto/ConvertToProto.kt 0.00% 53 Missing ⚠️
...src/main/kotlin/org/jacodb/ets/dto/EtsTypeToDto.kt 0.00% 49 Missing ⚠️
...ain/kotlin/org/jacodb/ets/proto/EtsValueToProto.kt 0.00% 39 Missing ⚠️
...src/main/kotlin/org/jacodb/ets/dto/ConvertToDto.kt 0.00% 30 Missing ⚠️
...rc/main/kotlin/org/jacodb/ets/dto/EtsValueToDto.kt 0.00% 30 Missing ⚠️
...-ets/src/main/kotlin/org/jacodb/ets/grpc/Server.kt 67.14% 14 Missing and 9 partials ⚠️
...ts/src/main/kotlin/org/jacodb/ets/utils/Loaders.kt 30.76% 17 Missing and 1 partial ⚠️
...db-ets/src/main/kotlin/org/jacodb/ets/dto/Model.kt 27.27% 8 Missing ⚠️
... and 7 more
Additional details and impacted files
@@             Coverage Diff              @@
##                neo     #318      +/-   ##
============================================
- Coverage     70.56%   69.55%   -1.02%     
- Complexity     2457     2459       +2     
============================================
  Files           275      285      +10     
  Lines         15288    16094     +806     
  Branches       2269     2378     +109     
============================================
+ Hits          10788    11194     +406     
- Misses         3581     3937     +356     
- Partials        919      963      +44     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Lipen Lipen changed the title Add gRPC-based IR loader Add Protobuf/gRPC-based EtsIR loader May 14, 2025
@Lipen Lipen force-pushed the lipen/grpc branch 2 times, most recently from 8a0061b to 00a6a24 Compare June 3, 2025 10:04
Copy link
Contributor

github-actions bot commented Jun 3, 2025

Test Results

  216 files  +1    216 suites  +1   16m 17s ⏱️ -3s
  714 tests +3    707 ✅ +1   7 💤 +2  0 ❌ ±0 
1 980 runs  +3  1 949 ✅ +1  31 💤 +2  0 ❌ ±0 

Results for commit 476ed40. ± Comparison against base commit 77b83e4.

♻️ This comment has been updated with latest results.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant