OSGE Generator Driver

OSGE Generator Driver command line interface

bin/generator osge --help

OSGE Generator
Usage: generator [options]

  -e <value> | --eventsPerSec <value>
        number of events to generate per sec, use this to throttle the generator
  -o <value> | --outputFormat <value>
        format of the string to write to the file defaults to: 'tsv'
	 where,
		tsv - string formatted by tabs in between columns
		csv - string formatted by commas in between columns
		avro - string formatted using avro serialization

  -r <value> | --fileRollSize <value>
        size of the file to roll in bytes, defaults to: Int.MaxValue (~2GB)
  -p <value> | --filePath <value>
        path of the file where the data should be generated, defaults to: '/tmp'
  -t <value> | --totalEvents <value>
        total number of events to generate, default: 1000
  -m | --multiTable
        generate data as multi tables format, default: false
  -b <value> | --flushBatch <value>
        number of events to flush to file at a single time, defaults to: 10000
  --threadsCount <value>
        number of threads to use for write and read operations, defaults to: 1
  --threadPoolSize <value>
        size of the thread pool, defaults to: 10
  --loggingLevel <value>
        Logging level to set, defaults to: INFO
  --help
        prints this usage text

Examples:

1. To generate 100000 events to /tmp

   bin/generator osge --totalEvents 100000 --filePath /tmp
   

2. To generate 100000 events to /tmp in multi table mode (generates to multiple tables/files)

   bin/generator osge --totalEvents 100000 --filePath /tmp --multiTable
   

3. To generate 100M events to /tmp and roll file every 64 MB

   bin/generator osge --totalEvents 100000000 --filePath /tmp --fileRollSize 67108864
   

4. To generate 100M events to /tmp/ concurrently using 5 threads

   bin/generator osge --totalEvents 100000000 --filePath /tmp --fileRollSize 67108864 --threadCount 5
   

5. To generate 100M events to /tmp/ concurrently using 5 threads in 'csv' format

   bin/generator osge --totalEvents 100000000 --filePath /tmp --fileRollSize 67108864 --threadCount 5 --outputFormat csv
   

6. To generate 100M events to /tmp/ concurrently using 5 threads in 'avro' format

   bin/generator osge --totalEvents 100000000 --filePath /tmp --fileRollSize 67108864 --threadCount 5 --outputFormat avro