diff --git a/docs_new/configuration/overview.md b/docs_new/configuration/overview.md index ff003fc9..5ac11021 100644 --- a/docs_new/configuration/overview.md +++ b/docs_new/configuration/overview.md @@ -50,6 +50,24 @@ metrics: - # ... ``` +The order of the sections is not important. + +Durations are used to define time spans in the configuration. +They can be used for the `timeout`-property of the workers or for the `completionTarget`-property of the tasks. +Duration values can be defined as a XSD duration string or as a string with a number and a unit. +The following units are supported: +- `s` or `sec`or `secs` for seconds +- `m` or `min` or `mins` for minutes +- `h` or `hr` or `hrs` for hours +- `d` or `day` or `days` for days + +Some examples for duration values: +```yaml +timeout: "2S" # 2 seconds +timeout: "10s" # 10 seconds +timeout: "PT10S" # 10 seconds +``` + ## Tasks The tasks are the core of the benchmark suite. They define the actual process of the benchmarking suite diff --git a/docs_new/configuration/queries.md b/docs_new/configuration/queries.md index e6433de8..262f1b98 100644 --- a/docs_new/configuration/queries.md +++ b/docs_new/configuration/queries.md @@ -7,17 +7,56 @@ Inside the stresstest task, the query handler is configured with the `queries` p Every worker instance of the same worker configuration will use the same query handler. The `queries` property is an object that contains the following properties: -| property | required | default | description | example | -|-----------|----------|----------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------| -| path | yes | | The path to the queries. It can be a file or a folder. | `./example/suite/queries/` | -| format | no | `one-per-line` | The format of the queries. If not set, the format is determined by the path. | `folder` or `separator` or `one-per-line` | -| separator | no | `""` | The separator that should be used if the format is set to `separator`. | `\n###\n` | -| caching | no | `true` | If set to `true`, the queries will be cached into memory. If set to `false`, the queries will be read from the file system every time they are needed. | `false` | -| order | no | `linear` | The order in which the queries are executed. If set to `linear` the queries will be executed in their order inside a file. If `format` is set to `folder`, queries will be sorted by the file name first. | `random` or `linear` | -| seed | no | `0` | The seed for the random number generator that selects the queries. If multiple workers use the same query handler, their seed will be the sum of the given seed and their worker id. | `12345` | -| lang | no | `SPARQL` | Not used for anything at the moment. | | - -## Separator +| property | required | default | description | example | +|-----------|----------|----------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------| +| path | yes | | The path to the queries. It can be a file or a folder. | `./example/suite/queries/` | +| format | no | `one-per-line` | The format of the queries. | `folder` or `separator` or `one-per-line` | +| separator | no | `""` | The separator that should be used if the format is set to `separator`. | `\n###\n` | +| caching | no | `true` | If set to `true`, the queries will be cached into memory. If set to `false`, the queries will be read from the file system every time they are needed. | `false` | +| order | no | `linear` | The order in which the queries are executed. If set to `linear` the queries will be executed in their order inside the file. If `format` is set to `folder`, queries will be sorted by their file name first. | `random` or `linear` | +| seed | no | `0` | The seed for the random number generator that selects the queries. If multiple workers use the same query handler, their seed will be the sum of the given seed and their worker id. | `12345` | +| lang | no | `SPARQL` | Not used for anything at the moment. | | + +## Format + +### One-per-line +The `one-per-line` format is the default format. +In this format, every query is written in a single line inside one file. + +In this example, the queries are written in a single file, each query in a single line: +``` +SELECT DISTINCT * WHERE { ?s ?p ?o } +SELECT DISTINCT ?s ?p ?o WHERE { ?s ?p ?o } +``` + +### Folder +It is possible to write every query in a separate file and put them all in a folder. +Queries will be sorted by their file name before they are read. + +In this example, the queries are written in separate files inside the folder `./example/suite/queries/`: +``` +./example/suite/queries/ +├── query1.txt +└── query2.txt +``` + +The file `query1.txt` contains the following query: +``` +SELECT DISTINCT * +WHERE { + ?s ?p ?o +} +``` + +The file `query2.txt` contains the following query: +``` +SELECT DISTINCT ?s ?p ?o +WHERE { + ?s ?p ?o +} +``` + +### Separator It is possible to write every query in a single file and separate them with a separator. The separator can be set with the `separator` property. Iguana will then split the file into queries based on the separator. diff --git a/docs_new/configuration/tasks.md b/docs_new/configuration/tasks.md index 8f6dffb1..95162ed3 100644 --- a/docs_new/configuration/tasks.md +++ b/docs_new/configuration/tasks.md @@ -4,9 +4,9 @@ They define the actual process of the benchmarking suite and are executed from top to bottom in the order they are defined in the configuration. At the moment, the `stresstest` is the only implemented task. -Tasks are defined in the `tasks` section of the configuration by the `type` property. +Tasks are defined in the `tasks` section of the configuration and are distinguished by the `type` property. -Example: +## Example ```yaml tasks: - type: "stresstest" diff --git a/docs_new/configuration/workers.md b/docs_new/configuration/workers.md index 244aa926..5fbd2702 100644 --- a/docs_new/configuration/workers.md +++ b/docs_new/configuration/workers.md @@ -1,5 +1,5 @@ # Workers -The stresstest uses workers to execute the queries. +The stresstest uses workers to execute the queries, which are supposed to simulate users. Each worker has its own set of queries and executes them parallel to the other workers. Iguana supports multiple worker types, but currently only the `SPARQLProtocolWorker` is implemented. @@ -26,13 +26,15 @@ The worker can be configured with the following properties: |------------------|----------|-------------|-----------------------------------------------------------------------------------------------------------------| | number | no | `1` | The number of workers that should be initiated with that same configuration. | | queries | yes | | The configuration of the query handler these workers should use. (see [here](./queries.md)) | -| completionTarget | yes | | | +| completionTarget | yes | | Either defines how many queries the worker should send, or how long it should send them. | | connection | yes | | The name of the connection that the worker should use.
(needs to reference an already defined connection) | | timeout | yes | | The duration for the query timeout. | | acceptHeader | no | | The accept header that the worker should use for the HTTP requests. | | requestType | no | `get query` | The request type that the worker should use. | | parseResults | no | `true` | Whether the worker should parse the results. | +Each property is explained in more detail below. + ### Example ```yaml connection: @@ -72,7 +74,7 @@ The query handler configuration is explained in more detail [here](./queries.md) The `completionTarget` property defines when the worker should stop executing queries. The property takes an object as its value that contains either one of the following properties: - `number`: The number of times the worker should execute each query. -- `duration`: The duration the worker should execute queries. +- `duration`: The duration during which the worker should iterate and execute every query. Example: ```yaml @@ -98,38 +100,37 @@ If the timeout is reached, the worker will mark it as failed, cancel the HTTP request and continue with the execution of the next query. The system that's being tested should make sure that it's able -to abort the further execution of the query if the timeout has been reached. -(e.g., by using a timeout parameter for the system, if available) +to abort the further execution of the query if the timeout has been reached +(e.g., by using a timeout parameter for the system, if available). Otherwise, problems like high resource usage or other issues might occur. ### Request Type -The request type is a property that defines the type of the HTTP request that the worker should use. -It can be defined by the `requestType` property. -The `requestType` property is a string that can be one of the following values: - -| request type | method | Content-Type header value | description | -|-------------------------|--------|-------------------------------------|-------------------------------------------------------------------------------------------------------------------| -| `"get query"` | `GET` | | The worker will send a `GET` request with a `query` parameter that contains the query. | -| `"post query"` | `POST` | `application/sparq-query` | The body will contain the query. | -| `"post update"` | `POST` | `application/sparq-update` | The body will contain the update query. | -| `"post url-enc query"` | `POST` | `application/x-www-form-urlencoded` | The body will contain the a url-encoded key-value pair with the key being `query` and the will the query itself. | -| `"post url-enc update"` | `POST` | `application/x-www-form-urlencoded` | The body will contain the a url-encoded key-value pair with the key being `update` and the will the query itself. | +The `requestType` property defines the type of the HTTP request that the worker should use. +It consists of a string that can be one of the following values: + +| request type | HTTP method | Content-Type header value | description | +|-------------------------|-------------|-------------------------------------|----------------------------------------------------------------------------------------------------------------| +| `"get query"` | `GET` | | The worker will send a `GET` request with a `query` parameter that contains the query. | +| `"post query"` | `POST` | `application/sparq-query` | The body will contain the query. | +| `"post update"` | `POST` | `application/sparq-update` | The body will contain the update query. | +| `"post url-enc query"` | `POST` | `application/x-www-form-urlencoded` | The body will contain the a url-encoded key-value pair with the key being `query` and the query as the value. | +| `"post url-enc update"` | `POST` | `application/x-www-form-urlencoded` | The body will contain the a url-encoded key-value pair with the key being `update` and the query as the value. | ### Accept Header The `acceptHeader` property defines the value for the `Accept` header of the HTTP requests that a worker sends to the defined endpoint. -This property also affects the [Response-Body-Processors](./overview#responsebodyprocessor) +This property also affects the [Response-Body-Processors](./overview#Response-Body-Processor) that are used to process the response bodies. ### Parse Results The `parseResults` property defines whether the worker should parse the results of the queries. If the property is set to `true`, -the worker will send the response body to the [Response-Body-Processors](./overview#responsebodyprocessor) for processing +the worker will send the response body to the [Response-Body-Processors](./overview#Response-Body-Processor) for processing and calculate hash values for the response bodies. If the property is set to `false`, the worker will not parse the response bodies and will not calculate hash values for the response bodies. Setting the property to `false` can improve the performance of the worker. -If the property is set to `true`, the worker will temporarily store the response bodies in memory for processing. +If the property is set to `true`, the worker will temporarily store the whole response bodies in memory for processing. If the property is set to `false`, the worker will discard any received bytes from the response.