Some IDA Python scripts for auto-analysis and a Hive-plot visualizer.
The scripts in this repo are an extraction from a larger set of scripts used for semi-automated analysis in security assurance of Irdeto's anti-reverse-engineering technology. In that environment we developed other, more detailed, analyses on-top-of what you find in this repo. What we have published here are some simple, but also hopefully useful scripts. The scripts were developed over the period spanning 2016-2018; tempered by valuable feedback from Irdeto Ottawa Security Assurance teammates @JonathanBeverley and @kennethblite.
The IDA Python parts of the scripts use the idascript framework from @devttys0 to enable both command-line and IDA Python shell command invocation. There are a couple features in this repo (as mentioned above, there are more advanced analysis features internally which we won't be releasing):
- batch+parallel analysis of a directory tree of binaries
- batch+parallel analysis of the binaries in an APK
- automating rizzo (also from @devttys0) analysis
- using IDA for
strings:)
Alongside those scripts above, we have also published an interactive hive-plot tool for visualizing the cross-references in a given IDA Pro .idb.
Hive plots are an idea first described by Krzywinski, Birol, Jones and Marra, please see http://www.hiveplot.com/. Mike Bostock (the author of d3.js) made a hive-plot plugin and a demo based on rendering of references in source code, please see https://bost.ocks.org/mike/hive/.
We have built on-top of the d3-hive.js plugin and added some UI elements to make contextual analysis of binaries easier as well as adding dynamic adjustments to weights and other visual elements as one zooms through the plots.
Here are some sample hive plots rendered from samples in https://github.com/maldroid/samples and https://github.com/JonathanSalwan/binary-samples. These were produced by running the test/run_tests.sh script:
You will need:
- bash, with at least the utilities
readlinkandfind- on OSX you'll need to install gnu coreutils to get
greadlinkinstalled
- on OSX you'll need to install gnu coreutils to get
- GNU
parallel - IDA (6.95 tested) with the following installed:
- IDAPython (2.7 tested)
- idascript https://github.com/devttys0/idascript (275be937d98da280fecdaf623fcc3c60ff1237e4 tested)
- Sark https://github.com/tmr232/Sark (1.0 tested)
- apktool https://github.com/iBotPeaches/Apktool (2.2.1 tested)
For the hive-plots (viz/ stuff), you will also need:
- node.js (v6.9.1 tested)
curl- a browser to view the
.htmlfiles (chrome tested)
With all of those installed, clone the repo and run a script. You will then need to run npm install in the viz/ subdirectory.
There are both .py and shell-scripts in this repo. The .py files are all idascript-based IDA Python scripts; the shell scripts mostly call-out to those to enable batch processing.
| Shell Script Name | Description |
|---|---|
apk-recipe |
An all-in-one script to unpack an apk, find all .dex and binaries and analyze them all in parallel. |
get-analysis-targets |
get-analysis-targets <dirs...> to recursively descend and extract all ELF and Mach-o files to named dirs in the current directory. 'Thinning' the macho files into individual architectures |
analyze-all |
analyze-all <target...> to recursively descend and produce .idb for every ELF in parallel |
unpack-all |
unpack-all <packages...> to unpack all .ipa and .apk to named dirs in the current directory. |
riz-all |
riz-all <target...> to recursively descend and produce .idb for every .idb in parallel |
viz/do |
creates a d3.js based vizualization given a json export of cross-references |
viz/*{.js,.css,.html} |
other parts of the vizualization scripts, mostly dependencies for the d3.js client-side scripts. |
IDA Python idascript Name |
Description |
|---|---|
rename_default_functions_with_a_prefix.py |
Useful when preparing rizzo signatures, gives all subXXXX functions a prefix so that they are not ignored by rizzo |
idb2riz.py |
simple idascript to run RizzoBuild() from the rizzo.py plugin on a X.idb to produce X.riz |
just_analysis.py |
a 'stub' idascript for use by auto-analyze |
viz/all-crossrefs-idb.py |
a simple script, to be used with the other script in viz/ , that exports all the static analysis cross-references from an .idb to a json structure |
A good candidate for using auto-analysis scritps is apk-recipe which will unpack and auto-analyze all binaries (.so and .dex in an apk).
$ git clone https://github.com/cloakware-ctf/idascripts.git
[...]
$ mkdir -p target-apk/downloads
$ mv target.apk target-apk/downloads/
$ cd target-apk/
$ ../idascripts/apk-recipe .
At this point any and all binaries in the .apk will have been extracted and auto-analyzed by IDA Pro. All .idb databases will be in well-named folders under analysis. Note that the script will execute the IDA Pro analysis in parallel (using GNU parallel) to save time.
The other interesting feature of these scripts is to produce hive-plot vizualizations using viz/do. This script will hunt for any IDA pro databases and create hive plot vizualiations of the xrefs therein. Continuing the above analysis:
./viz/do analysis/
At this point there will be a standalone .html file containing an interactive hive-plot for each .idb in analysis/.
Hive Plots:
- Add a duplicate axis (like in Mike Bostock's example) for cases where we want to see intra-group links
- Auto-orient the axes so that the most-central axis is most-horizontal
- Enable dragging of axis to rotate axis
- Enable dragging nodes to re-order
- For panel-plots (not pictured in this repo) keep pan independent, but still tie scale
- Use
ttfor node names - add floating labels at border of visible areas on edges when they are selected or active
- dark theme
Pull requests are welcome.