First commit

.gitignore

@@ -0,0 +1,12 @@
+*.d
+*.o
+*.pyc
+.gdbinit
+config.h
+pyrebox.conf
+*.so
+docs/build
+qemu
+volatility
+pyrebox-i386
+pyrebox-x86_64

BUILD.rst

@@ -0,0 +1,25 @@
+Building PyREBox
+================
+
+- Installing dependencies
+
+  * For Debian based distributions: 
+
+    ``apt-get install build-essential zlib1g-dev pkg-config libglib2.0-dev binutils-dev libboost-all-dev autoconf libtool libssl-dev libpixman-1-dev libpython-dev python-pip``
+
+  * Required python packages:
+
+    ``ipython>=5,<6 sphinx sphinx-autobuild prettytable pefile capstone distorm3 pycrypto pytz``
+
+  * You can install them with pip: 
+
+    ``pip install -r requirements.txt``
+
+- Project configuration and building
+
+  ``./build.sh``
+
+Installing PyREBox
+==================
+
+PyREBox package installation is not yet supported.

Dockerfile

@@ -0,0 +1,54 @@
+#-------------------------------------------------------------------------------
+#
+#   Copyright (C) 2017 Cisco Talos Security Intelligence and Research Group
+#
+#   PyREBox: Python scriptable Reverse Engineering Sandbox 
+#   Author: Jonas Zaddach
+#   
+#   This program is free software; you can redistribute it and/or modify
+#   it under the terms of the GNU General Public License version 2 as
+#   published by the Free Software Foundation.
+#   
+#   This program is distributed in the hope that it will be useful,
+#   but WITHOUT ANY WARRANTY; without even the implied warranty of
+#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#   GNU General Public License for more details.
+#
+#   You should have received a copy of the GNU General Public License
+#   along with this program; if not, write to the Free Software
+#   Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
+#   MA 02110-1301, USA.
+#   
+#-------------------------------------------------------------------------------
+
+FROM ubuntu:16.04
+MAINTAINER Jonas Zaddach
+
+ENV PREFIX /home
+
+#Install packages
+RUN apt-get update
+RUN apt-get install -y build-essential zlib1g-dev pkg-config \
+                       libglib2.0-dev binutils-dev libboost-all-dev \
+                       autoconf libtool libssl-dev libpixman-1-dev \
+                       libpython-dev python-pip \
+                       git curl vim
+
+#clone pybox
+RUN git clone https://github.com/Cisco-Talos/pyrebox pyrebox
+WORKDIR pyrebox
+RUN pip install -r requirements.txt
+RUN ./build.sh
+
+#FIXME: Ugly hack because pip is installing the library in the wrong place
+RUN mv /usr/local/lib/python2.7/dist-packages/usr/lib/python2.7/dist-packages/capstone/libcapstone.so /usr/local/lib
+RUN ldconfig
+
+#OPTIONAL: Copy VM in. Left as an example
+
+#RUN mkdir /images
+#ADD files/template_pybox_vm_winxp.tar.gz /images
+#RUN echo "./start_i386.sh /images/xpsp3.qcow2" > ~/.bash_history
+
+EXPOSE 5900
+ENTRYPOINT ["/bin/bash"]

Makefile

@@ -0,0 +1,76 @@
+#-------------------------------------------------------------------------------
+#
+#   Copyright (C) 2017 Cisco Talos Security Intelligence and Research Group
+#
+#   PyREBox: Python scriptable Reverse Engineering Sandbox 
+#   Author: Xabier Ugarte-Pedrero 
+#   
+#   This program is free software; you can redistribute it and/or modify
+#   it under the terms of the GNU General Public License version 2 as
+#   published by the Free Software Foundation.
+#   
+#   This program is distributed in the hope that it will be useful,
+#   but WITHOUT ANY WARRANTY; without even the implied warranty of
+#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#   GNU General Public License for more details.
+#
+#   You should have received a copy of the GNU General Public License
+#   along with this program; if not, write to the Free Software
+#   Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
+#   MA 02110-1301, USA.
+#   
+#-------------------------------------------------------------------------------
+
+include qemu/config-host.mak
+DEFINES=-I. -I..
+
+CC=gcc
+CPP=g++
+PYTHON_CFLAGS = -I./pyrebox/ -I/usr/include/python2.7 -I/usr/include/python2.7 -fno-strict-aliasing -DNDEBUG -fwrapv  -fstack-protector --param=ssp-buffer-size=4
+PYTHON_LIBS = -L/usr/lib/python2.7/config -lpthread -ldl -lutil -lm -lpython2.7 -Wl,-O1 -Wl,-Bsymbolic-functions -Xlinker -export-dynamic
+CFLAGS=-Wall -O2 -g -fPIC -MMD -std=c++11 -std=gnu++11 
+CFLAGS+=$(PYTHON_CFLAGS)
+LDFLAGS=-g -shared 
+LDFLAGS+=$(PYTHON_LIBS)
+
+#Triggers for i386-softmmu
+
+%-i386-softmmu.o: %.c 
+	@$(CC) $(CFLAGS) $(DEFINES) -I./qemu/i386-softmmu -c -o $@ $< ; \
+	echo $(CPP) $@ ;
+
+%-i386-softmmu.o: %.cpp
+	@$(CPP) $(CFLAGS) $(DEFINES) -I./qemu/i386-softmmu -c -o $@ $< ; \
+	echo $(CPP) $@ ;
+
+%-i386-softmmu.so: %-i386-softmmu.o pyrebox/trigger_helpers-i386-softmmu.o pyrebox/utils-i386-softmmu.o
+	@$(CPP) -Wno-builtin-macro-redefined  $(CFLAGS) $(DEFINES) -I./qemu/i386-softmmu -shared -o $@ $^ ; \
+	echo $(CPP) $@
+
+#Triggers for x86-64-softmmu
+
+%-x86_64-softmmu.o: %.c 
+	@$(CC) $(CFLAGS) $(DEFINES) -I./qemu/x86_64-softmmu -c -o $@ $< ; \
+	echo $(CPP) $@ ;
+
+%-x86_64-softmmu.o: %.cpp
+	@$(CPP) $(CFLAGS) $(DEFINES) -I./qemu/x86_64-softmmu -c -o $@ $< ; \
+	echo $(CPP) $@ ;
+
+%-x86_64-softmmu.so: %-x86_64-softmmu.o pyrebox/trigger_helpers-x86_64-softmmu.o pyrebox/utils-x86_64-softmmu.o
+	@$(CPP) -Wno-builtin-macro-redefined  $(CFLAGS) $(DEFINES) -I./qemu/x86_64-softmmu -shared -o $@ $^ ; \
+	echo $(CPP) $@
+
+clean-triggers:
+	rm -f triggers/*.so triggers/*.o triggers/*.d
+
+documentation: 
+	$(MAKE) -C ./docs/ html 
+
+#We place these 2 rules so that we can compile more comfortably from this directory
+all: 
+	@$(MAKE) -C ./qemu $@
+
+clean: clean-triggers	
+	@$(MAKE) -C ./qemu $@
+

README.rst

@@ -0,0 +1,140 @@
+.. image:: docs/media/pyrebox_logo_light_bg.png 
+
+.. _QEMU: http://qemu.org/
+.. _DECAF: https://github.com/sycurelab/DECAF 
+.. _S2E: https://github.com/dslab-epfl/s2e
+.. _AVATAR: https://github.com/avatartwo 
+.. _PANDA: https://github.com/panda-re/panda
+.. _Volatility: http://www.volatilityfoundation.org/ 
+.. _BUILD: BUILD.rst 
+.. _here: https://github.com/Cisco-Talos/pyrebox/issues
+.. _scripts: https://github.com/Cisco-Talos/pyrebox/tree/master/scripts
+
+PyREBox is a Python scriptable Reverse Engineering sandbox. It is based on QEMU, and its goal is 
+to aid reverse engineering by providing dynamic analysis and debugging capabilities from a 
+different perspective. PyREBox allows to inspect a running QEMU VM, modify its memory or 
+registers, and to instrument its execution, by creating simple scripts in python to automate 
+any kind of analysis. QEMU (when working as a whole-system-emulator) emulates a complete 
+system (CPU, memory, devices...). By using VMI techniques, it does not require to perform any 
+modification into the guest operating system, as it transparently retrieves information from 
+its memory at run-time.
+
+
+Several academic projects such as DECAF_, PANDA_, S2E_, or AVATAR_, have previously leveraged QEMU 
+based instrumentation to overcome reverse engineering tasks. These projects allow to write plugins 
+in C/C++, and implement several advanced features such as dynamic taint analysis, symbolic execution, 
+or even record and replay of execution traces. With PyREBox, we aim to apply this technology focusing 
+on keeping the design simple, and on the usability of the system for threat analysts.
+
+Documentation
+=============
+
+
+Goals
+=====
+
+- Provide a whole system emulation platform with a simple interface for inspecting the emulated guest system.
+
+  * Fine grained instrumentation of system events.
+  * Integrated Virtual Machine Introspection (VMI), based on volatility. No agent or driver needs to be installed into the guest.
+  * An IPython based shell interface.
+  * A Python based scripting engine, that allows to integrate into the scripts any of the security tools based on this language (one of the biggest ecosystems).
+- Have a clean design, de-coupled from QEMU. Many projects that are built over QEMU do not evolve when QEMU gets upgraded, missing new features and optimizations, as well as security updates. In order to achieve this, PyREBox is implemented as an independent module that can be compiled together with QEMU requiring a minimal set of modifications. 
+- Support for different architectures. Currently, PyREBox only supports Windows for x86 and x86-64 bit architectures, but its design allows to support other architectures such as ARM, MIPS, or PowerPC, and other operating systems as well.
+
+IPython shell
+=============
+
+Starting a PyREBox shell is as easy as typing the ``sh`` command on QEMU’s monitor. It will immediately start an IPython
+shell. This shell records the command history as well as the defined variables. For instance, you can save a
+value and recover it later at a different point of the execution, when you start the shell again. PyREBox takes
+advantage of all the available features in IPython such as auto-completion, command history, multi-line editing, and
+automated command help generation.
+
+PyREBox will allow you to debug the system (or a process) in a fairly stealthy way. Unlike traditional debuggers which stay
+in the system being debugged (even modifying the memory of the debugged process to insert breakpoints), PyREBox stays
+completely outside the inspected system, and it does not require the installation of any driver or component into
+the guest.
+
+.. image:: docs/media/breakpoint.gif 
+
+PyREBox offers a complete set of commands to inspect and modify the state of the running VM. Just type ``list_commands``
+to obtain a complete list. You can run any volatility plugin just by typing ``vol`` and the corresponding volatility command.
+For a complete list of available volatility plugins, you
+can type ``list_vol_commands``. This list is generated automatically, so it will also show any volatility plugin you
+install on PyREBox's ``volatility/`` path. 
+
+You can also define your own commands! It is as simple as declaring a function in a script, and loading it.
+
+If you need something more expressive than a command, you can write a Python snippet leveraging the API. For a detailed
+description of the API, see `Documentation`_ or type ``help(api)`` in the shell.
+
+.. image:: docs/media/stack.gif
+
+Scripting
+=========
+
+PyREBox allows to dynamically load scripts that can register callback functions that are called when certain events
+occur, like instructions executed, memory read/written, processes created/destroyed, and so on. 
+
+Given that PyREBox is integrated with Volatility, it will let you take advantage of all the volatility plugins for
+memory forensics in your python scripts. Many of the most famous reverse engineering tools are implemented in Python or
+at least have Python bindings. Our approach allows to integrate any of these tools into a script.
+
+Finally, given that python callbacks can introduce a performance penalty on frequent events such as
+instructions executed, it is also possible to create *triggers*. *Triggers* are native-code plug-in’s (developed in C/C++)
+that can be inserted dynamically at run-time on any event just before the Python callback is executed. This allows to
+limit the number of events that hit the python code, as well as to precompute values in native code.
+
+In this repository you will find example scripts_ that can help you to write your owncode . Contributions are welcome!
+
+Install
+=======
+
+A build script is provided. For specific details about dependencies, please see BUILD_. We also provide a Dockerfile.
+
+Starting a VM
+=============
+
+PyREBox is based on QEMU, so in order to start a VM withing PyREBox, you need to run it exactly as you
+were booting up a QEMU VM. A couple of example scripts are provided: ``start_i386.sh``, ``start_x86_64.sh``,
+you can use them as an example.
+
+The only QEMU monitor option supported currently is *stdio* (``-monitor stdio``).
+
+Roadmap
+=======
+
+- VM image configuration and management console.
+- Support for ARM, MIPS, and other architectures.
+- Support for GNU/Linux guest systems.
+
+Acknowledgement
+===============
+
+First of all, PyREBox would not be possible without QEMU_ and Volatility_. We thank to their developers and
+mantainers for such a great work.
+
+PyREBox is inspired by several academic projects, such as DECAF_, or PANDA_. In fact, many of the callbacks
+supported by PyREBox are equivalent to those found in DECAF_, and the concepts behind the instrumentation
+are based on these works. 
+
+PyREBox benefits from third-party code, which can be found under the directory pyrebox/third_party. 
+For each third-party project, we include an indication of its original license, the original source
+code files taken from the project, as well as the modified versions of the source code files (if applicable),
+used by PyREBox. 
+
+
+Bugs and support
+================
+
+If you think you've found a bug, please report it here_.
+
+This program is provided "AS IS", and no support is guaranteed. That said, in order to help 
+us solve your issues, please include as much information as possible in order to reproduce the bug:
+
+- Operating system used to compile and run PyREBox.
+- The specific operating system version and emulation target you are using.
+- Shell command / script / task you were trying to run.
+- Any information about the error such as error messages, Python (or IPython) stack trace, or QEMU stack trace.
+- Any other relevant information