Merge pull request #54 from cisco-sas/next

version 0.6.8

CHANGELOG

@@ -1,24 +1,34 @@
+Version 0.6.8 (2016-05-04)
+==========================
+
+* bugfix: [DataModel] Delimiter field's __init__ didn't receive nor passed an encoder
+* bugfix: [DataModel] condition - added copy function and fixed invalidate function
+* bugfix: [DataModel] rendering of default values in containers
+* enhancement: [WebInterface] html - improvements in index and reports display
+* new feature: [DataModel] new field - Float
+* new feature: [DataModel] new field - Switch
+
 Version 0.6.7 (2016-04-18)
 ==========================
 
-* new feature: [KittyActor] a new class, KittyActor, is now the base class for monitors and controllers
-* new feature: [KittyActor] a new API is_victim_alive can be used to check if the victim is alive, it is called in a loop in pre-test
-* enhancement: [BaseFuzzer] default logging level is now INFO, hence less messages are printed, can be controlled from cmd line args
 * bugfix: [BaseFuzzer] update test info before the calling target's pre_test
-* new feature: [Report] use 'status' field instead of 'failed' field in the report. currently support FAILED, PASSED and ERROR.
-* enhancement: [WebInterface] html - improvements of caching, window size, cell overflow
-* new feature: [WebInterface] html - report table now shows test status and reason
-* bugfix: [kitty-tool] fixed command line option parsing
-* new feature: [kitty-tool] added checks for mutation indices in "generate"
-* new feature: [kitty-tool] print progress and current field path in "generate"
 * bugfix: [DataModel] fixed "field path" in field info
-* new feature: [DataModel] new fields - Offset & AbsoluteOffset - can hold the offset of a specific field inside the template
-* new feature: [DataModel] new conditions - BitMaskSet and BitMaskNotSet
-* new feature: [DataModel] absolute name resolution of fields
+* bugfix: [kitty-tool] fixed command line option parsing
+* enhancement: [BaseFuzzer] default logging level is now INFO, hence less messages are printed, can be controlled from cmd line args
 * enhancement: [DataModel] perform render in two passes when needed, avoid (some) redundant rendering in between
-* new feature: [DataModel] new container - List - will perform the standard mutations of each of the internal fields, but will also perform mutations on the order and existence of its elements.
 * enhancement: [DataModel] reduce number of mutations for BitField & String field. This cuts the mutation count to about half of the previous count.
+* enhancement: [WebInterface] html - improvements of caching, window size, cell overflow
+* new feature: [DataModel] new conditions - BitMaskSet and BitMaskNotSet
+* new feature: [DataModel] new container - List - will perform the standard mutations of each of the internal fields, but will also perform mutations on the order and existence of its elements.
+* new feature: [DataModel] new fields - Offset & AbsoluteOffset - can hold the offset of a specific field inside the template
+* new feature: [DataModel] absolute name resolution of fields
 * new feature: [DataModel] now String and BitField have a human friendly mutation description in the info
+* new feature: [KittyActor] a new class, KittyActor, is now the base class for monitors and controllers
+* new feature: [KittyActor] a new API is_victim_alive can be used to check if the victim is alive, it is called in a loop in pre-test
+* new feature: [kitty-tool] added checks for mutation indices in "generate"
+* new feature: [kitty-tool] print progress and current field path in "generate"
+* new feature: [Report] use 'status' field instead of 'failed' field in the report. currently support FAILED, PASSED and ERROR.
+* new feature: [WebInterface] html - report table now shows test status and reason
 
 Version 0.6.6 (2016-04-03)
 ==========================

CONTRIBUTING.rst

@@ -0,0 +1,105 @@
+Contributing to Kitty
+=====================
+
+If you look here - thanks!
+
+This document describes the guidelines for contribution for Kitty.
+This is not to discourage you from contributing -
+we only benefit from contribution.
+However, we want to keep Kitty stable, documented and easy to read.
+
+At this point, this guide is rather short, so please read it from start to end.
+it will 
+
+Code
+----
+
+- **Python 3 compatibility**
+
+  Kitty is written in Python 2.7, we do plan to add python3 compatibility
+  at some point, so please make any new addition python 2 and python 3
+  compatible.
+
+- **Coding conventions**
+
+  We do try to make the pass pylint checks (although it is still far from it)
+  don't make it harder for us, check your code with pylint before you issue
+  a pull request.
+  Specifically, you should not have redundant spaces/line breaks in your code,
+  nor lines longer than 160 characters.
+
+- **Comments**
+
+  We do try not to comment the code, in most cases, comments tell us that the
+  code is not clear enough to begin with.
+  If this is the case - please refactor your code.
+  Of course, if the code is unclear but there's no way to make it better,
+  put a comment there.
+
+- **logging**
+
+  There should be no calls to ``print()`` in your code.
+  Kitty uses python's logging infrastructure,
+  and every object that derives from ``KittyObject`` have a member ``self.logger``
+  just for you.
+  Kitty's default logging level is ``INFO``,
+  and in this mode there shouldn't be too much logging to the terminal,
+  Please use the appropriate logger functions.
+
+Tests
+-----
+
+We try to keep Kitty tested, tests run on each push and pull request.
+The tests are located at the **tests** directory,
+and ``python runner.py`` should run all of them.
+We have two requests:
+
+- Run the tests before you open a pull requests.
+- Add tests for every new module, feature, class or code that you create.
+  if your pull request is meant to fix a bug in Kitty,
+  it means that we are missing a test there.
+  Add such a test with your fix.
+- If you create a new test file, add it to the runner,
+  so it will run with the rest of the tests.
+
+Documentation
+-------------
+
+We also try to keep Kitty documented.
+We use Sphinx to generate the documentation.
+Sphinx generates documentation from both code and documentation files.
+So we require a few things in pull requests.
+
+- **Docstrings**
+
+  Add docstrings to your code - modules, classes and public methods.
+  Take a look at other modules (mainly in **kitty/model/low_level**)
+  to get an idea of how we document the code.
+
+- **New modules**
+
+  If you add a new file to kitty, you should:
+
+  - Add a class description file to **docs/source**.
+    Take a look at **kitty.model.low_level.aliases.rst**
+    as an example.
+
+    Your file name should match the ``import`` of this file,
+    for example,
+    if you add the file **kitty/model/low_level/mymodule.py**
+    the documentation file will be called **kitty.model.low_level.mymodule.rst**
+
+  - Add this description file to the TOC of the package.
+    If we keep the example from the last bullet,
+    Open the file **kitty.model.low_level.rst** and add
+    the line **kitty.model.low_level.mymodule** to the TOC tree.
+
+- **New features**
+
+  If you add a new feature to kitty,
+  consider adding a more thorough documentation and a tutorial.
+  Add them to the TOC tree of **index.rst** and **tutorials/index.rst**
+  (respectively).
+
+
+That's it, and thanks for your help!

README.rst

@@ -101,9 +101,10 @@ What's Next?
 
         pip install kittyfuzzer
 
-- Read some of the documentation at `ReadTheDocs <https://kitty.readthedocs.org>`_.
+- Read some of the documentation at `ReadTheDocs <https://kitty.readthedocs.io>`_.
 - Build your fuzzer :-)
 - Need some help - ask at our google group: [email protected]
+- You can also find us on freenode #kittyfuzzer
 
 Contribution FAQ
 ----------------
@@ -126,7 +127,7 @@ Contribution FAQ
 .. |docs| image:: https://readthedocs.org/projects/kitty/badge/?version=latest
     :alt: Documentation Status
     :scale: 100%
-    :target: https://kitty.readthedocs.org/en/latest/?badge=latest
+    :target: https://kitty.readthedocs.io/en/latest/?badge=latest
 
 .. |travis| image:: https://travis-ci.org/cisco-sas/kitty.svg?branch=master
     :alt: Build Status

bin/kitty_template_tester.py

@@ -50,9 +50,21 @@ def process(self, t):
 
 class TemplateTreePrinter(TemplateProcessor):
 
+    def _pad(self):
+        return '  ' * self.depth
+
+    def _print_structure(self, structure):
+        print('%s%s(name="%s")' % (self._pad(), structure['field_type'], structure['name']))
+        if 'fields' in structure:
+            self.depth += 1
+            for field in structure['fields']:
+                self._print_structure(field)
+            self.depth -= 1
+
     def process(self, t):
-        t.num_mutations()
-        print(t.get_tree())
+        self.depth = 0
+        structure = t.get_structure()
+        self._print_structure(structure)
 
 
 class TemplateTester(TemplateProcessor):

bin/kitty_tool.py

@@ -19,12 +19,11 @@
 Tools for testing and manipulating kitty templates.
 
 Usage:
-    kitty-tool generate [--verbose] [-s SKIP] [-c COUNT] [-o OUTDIR] [-f FORMAT] <FILE> <TEMPLATE> ...
+    kitty-tool generate [options] <FILE> <TEMPLATE> ...
     kitty-tool list <FILE>
     kitty-tool --version
 
 Commands:
-
     generate    generate files with mutated payload
     list        list templates in a file
 
@@ -34,6 +33,7 @@
     --out -o OUTDIR         output directory for the generated mutations [default: out]
     --skip -s SKIP          how many mutations to skip [default: 0]
     --count -c COUNT        end index to generate
+    --path -p FIELDPATH     generate mutations only for the field with the given path
     --verbose -v            verbose output
     --filename-format -f FORMAT  format for generated file names [default: %(template)s.%(index)s.bin]
     --version               print version and exit
@@ -114,31 +114,31 @@ def handle(self, template):
 
 class FileGeneratorHandler(Handler):
 
-    def __init__(self, outdir, skip, count, template_names, filename_template, logger):
+    def __init__(self, opts, logger):
         super(FileGeneratorHandler, self).__init__(logger)
-        self.outdir = outdir or 'out'
-        if skip is not None:
+        self.outdir = opts['--out'] or 'out'
+        if opts['--skip'] is not None:
             try:
-                self.skip = int(skip)
+                self.skip = int(opts['--skip'])
             except:
                 raise Exception('skip should be a number')
         else:
             self.skip = 0
-        self.count = count
+        self.count = opts['--count']
         if self.count:
             try:
-                self.count = int(count)
+                self.count = int(self.count)
             except:
                 raise Exception('count should be a number')
-        self.template_names = template_names
-        self.filename_template = filename_template
+        self.template_names = opts['<TEMPLATE>']
+        self.filename_format = opts['--filename-format']
         try:
-            self.filename_template % {
+            self.filename_format % {
                 'template': 'hello',
                 'index': 1
             }
         except:
-            raise Exception('invalid filename template: %s' % (self.filename_template))
+            raise Exception('invalid filename template: %s' % (self.filename_format))
 
     def start(self):
         if os.path.exists(self.outdir):
@@ -162,7 +162,7 @@ def handle(self, template):
 #            step = max(step, 2)
             max_line_length = 0
             while template.mutate():
-                template_filename = self.filename_template % {'template': template_name, 'index': template._current_index}
+                template_filename = self.filename_format % {'template': template_name, 'index': template._current_index}
                 with open(os.path.join(self.outdir, template_filename), 'wb') as f:
                     f.write(template.render().tobytes())
                 metadata_filename = template_filename + '.metadata'
@@ -200,11 +200,7 @@ def _main():
             file_iter = FileIterator(
                 opts['<FILE>'],
                 FileGeneratorHandler(
-                    outdir=opts['--out'],
-                    skip=opts['--skip'],
-                    count=opts['--count'],
-                    template_names=opts['<TEMPLATE>'],
-                    filename_template=opts['--filename-format'],
+                    opts,
                     logger=logger
                 ),
                 logger