v3.0.0b2
Pre-release
Pre-release
Added
- Support for asyncio within the framework and plugins
- Better type checking support
self.log
may be leveraged from within all plugin classes- Add new
Error()
class for standardizing errors from stoQ and plugins
Error()
will track plugin name, error message, and payload_id (optional) - Add configuration properties from
[Core]
and[Documentation]
to each plugin object when loaded PayloadMeta
now has ashould_scan
boolean.
Allows payloads to be logged and archived, but not scanned by worker plugin.Payload
is now updated as results are completed.
Results from completed scans will be available to other plugins instantlyRequest()
class is passed to all dispatchers, workers, and archiver plugins.
TheRequest
object contains all payloads, request metadata, results, and errors from
all other completed plugins. This will allow for all neccessary plugins to have a full
understanding of the current state of the completeRequest
.WorkerPlugin
s now have a configuration option ofrequired_workers
.
This allows for chained worker dependencies. Ifrequired_workers
is defined, the
parent plugin will not be run until all required plugins are completed successfully. The
parent plugin may then use results from other completed plugins for their respective
scanning tasks.- Duplicate extracted payloads are no longer simply skipped, they are appended to
Payload.results[].extracted_by
andPayload.results[].extracted_from
- Add
StoqConfigParser
tostoq.helpers
to extend options forStoq
and plugin configurations. - Parallelization is performed across all of the plugins that can run in a given round,
instead of parallelizing across all of the plugins to perform on a given payload (#147)
Changed
PayloadResults
is now an object ofPayload.results
, rather than an independent object- Most objects have been removed from
Payload
and are now availabe inPayload.results
,
namelyextracted_by
,extracted_from
,payload_id
,size
,payload_meta
Payload.plugins_run
moved toPayloadResults.plugins_runs
and is now aDict[str, List[str]]
rather thanDict[str, List[List[str]]]
PayloadResults.workers
is now aDict[str, Dict]
rather thanList[Dict[str, Dict]]
PayloadMeta
is now an object ofPayloadResults.payload_meta
PayloadResults.extracted_by
is now aList[str]
rather thanstr
PayloadResults.extracted_from
is now aList[str]
rather thanstr
- Dispatchers run on each payload every round, instead of once per payload. This allows
the dispatcher to take advantage of the request state model. (#147) - Worker plugins can specify additional plugins to run on the payload they scan, effectively giving them dispatch capability.
With YARA, for example, this allows us to directly scan with YARA and dispatch
to other plugins by running YARA once. Otherwise, we would run YARA as a dispatcher,
and then immediately run YARA again as a worker plugin. (#147) - Archivers run at the very end along with connectors and decorators because we no
longer scan a payload to completion at once. (#147) - The default value for max_recursion has increased because the average number of
worker rounds taken to complete a scan is expected to increase. (#147)
Deprecated
- DeepDispatcher plugin class has been removed
Payload.plugins_run
has been removed in favor ofPayloadResults.plugins_run
)Payload.worker_results
has been removed in favor ofPayloadResults.workers
RequestMeta
is no longer passed to plugins, in favor of theRequest
objectplugins_opts
has been removed from plugin__init__
function. All plugin configuration options
are only available inself.config