Replace Jython with JPype

.github/workflows/ci.yml

@@ -3,10 +3,44 @@ name: CI
 on:
   push:
     branches:
-    - "**"
+      - "master"
   pull_request:
   workflow_dispatch:
 
 jobs:
   ci:
     uses: angr/ci-settings/.github/workflows/angr-ci.yml@master
+
+  test:
+    strategy:
+      matrix:
+        python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
+        os: [ubuntu-22.04, macos-12, windows-2022]
+      fail-fast: false
+    name: Test on ${{ matrix.os }} with Python ${{ matrix.python-version }}
+    runs-on: ${{ matrix.os }}
+    steps:
+    - name: Checkout Pysoot
+      uses: actions/checkout@v4
+      with:
+        path: pysoot
+    - name: Checkout binaries
+      uses: actions/checkout@v4
+      with:
+        repository: angr/binaries
+        path: binaries
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v2
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Setup Java
+      uses: actions/setup-java@v4
+      with:
+        distribution: 'zulu'
+        java-version: '8'
+
+    - name: Install pysoot
+      run: pip install ./pysoot[testing]
+
+    - name: Run tests
+      run: pytest pysoot

.gitignore

@@ -9,3 +9,5 @@ tests/test_samples/android1_code/build
 
 build
 dist
+
+.venv

README.md

@@ -6,7 +6,7 @@ PySoot
 The `master` branch supports Python 3, the `py2k` branch supports Python2.
 
 # Installation
-`pip install -e .`
+`pip install .`
 
 # How to use
 ```Python 3
@@ -19,35 +19,18 @@ print(classes[list(classes.keys())[0]]) # print the IR of one of the translated
 
 Many other examples are in `tests/test_pysoot.py`
 
-`lifter.soot_wrapper` gives direct access to some Soot functionality.
-As of now, I added functions from `Hierarchy.java`, but it is easy (and "almost" automatic) to add others.
-
 # Requirements
 * Java. Currently tested using OpenJDK 8 (`sudo apt-get install openjdk-8-jdk`).
 
 Other components used by `pysoot` are:
-* `Jython`. Already included in this repo, it is not neccesary to install it. The embedded version "simulates" a virtualenv with `pysoot` installed.
+* `jpype`, used for accessing Java from Python.
 * `soot-trunk.jar`. This is a slightly modified version of the pre-compiled Soot JAR. At some point, I will upload its source code and the compilation script somewhere.
 `pysoot` should also work with a normal version of `soot-trunk.jar`.
 
 # Internals
 #### Components
-`pysoot` works by running Soot (compiled in the embedded `soot-trunk.jar`) using Jython (embedded) and the code in `soot_manager.py`
-
-`jython_wrapper.py` and `jython_runner.py` establish an IPC bi-directional channel which allows a Python process to call methods of an instance of a class in Jython (data is serialized/deserialized using `pickle`).
-`jython_wrapper.py` runs in Python, while `jython_runner.py` runs in Jython.
-In the future we could release this IPC-layer as a separate component.
-
-`lifter.py` uses this IPC channel to ask Jython to create and serialize the IR.
-
-Classes in `pysoot.sootir` are used both by the Jython code and the Python one.
-
-#### Data-Flow Overview
-Python --> `lifter.py` --> `jython_wrapper.py` --> Jython --> `jython_runner.py` --> `soot_manager.py` --> Soot --> Soot IR
-
-Jython --> Soot IR --> `classes in pysoot.sootir` --> `jython_runner.py, pickle` --> Python --> `jython_wrapper.py, unpickle` --> `classes in pysoot.sootir` --> `lifter.py`
-
-<br/>
+`pysoot` works by running Soot (compiled in the embedded `soot-trunk.jar`) using jpype, in `soot_manager.py`.
 
-![Pysoot Architecture](pysoot_arch.png "Pysoot Architecture")
+`lifter.py` uses `soot_manager.py` to translate the JAR/APK file to a Soot-like Python IR.
 
+Classes in `pysoot.sootir` provide the exposed IR.

pyproject.toml

@@ -0,0 +1,17 @@
+[build-system]
+requires = ["setuptools"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "pysoot"
+version = "8.0.0.dev0"
+description = "Python bindings for Shimple/Jimple IR from Soot."
+urls = {Homepage = "https://github.com/angr/pysoot"}
+requires-python = ">=3.8"
+dependencies = ["jpype1", "psutil", "frozendict"]
+
+[tool.setuptools.package-data]
+pysoot = ["soot-trunk.jar"]
+
+[project.optional-dependencies]
+testing = ["pytest"]

pysoot/__init__.py

@@ -1,43 +1 @@
-
-import sys
-
-
-def is_jython():
-    ee = sys.executable
-    # it is None when you start Jython from Java
-    return ee is None or ee.endswith("jython")
-
-
-from .errors import *
-import struct
-import pickle
-
-PICKLE_PROTOCOL = 2
-
-def send_obj(fp, obj, pickled_object=None, otype = b"n"):
-    # TODO is there a faster/less memory-consuming method than using a pipe
-    # what about pickle + shared memory?
-    if pickled_object is None:
-        pickled_object = pickle.dumps(obj, PICKLE_PROTOCOL)
-    full_data = otype + struct.pack("<Q", len(pickled_object)) + pickled_object
-    fp.write(full_data)
-    fp.flush()
-
-
-def recv_obj(fp):
-    state = 1
-    to_recv = 1+8
-    buf = ""
-    while True:
-        tstr = fp.read(to_recv)
-        if tstr == "":
-            raise RecvException()
-        to_recv -= len(tstr)
-        buf += tstr
-        if to_recv == 0:
-            if state == 1:
-                to_recv = struct.unpack("<Q", buf[1:])[0]
-                buf = ""
-                state = 2
-            elif state == 2:
-                return pickle.loads(buf)
+from .lifter import Lifter

pysoot/errors.py

@@ -14,3 +14,10 @@ class JythonClientException(PySootError):
 class RecvException(PySootError):
     pass
 
+
+class JavaNotFoundError(PySootError):
+    pass
+
+
+class MissingJavaRuntimeJarsError(PySootError):
+    pass