IETF: Replicate Gym experiments using tng-bench for IETF input

[mpeuster]

See: https://github.com/raphaelvrosa/vnf-bench-model/tree/master/experiments/vnf-bench-01

---

Test PED: tng-bench -p examples/peds/ped_suricata_test.yml -v

• install tango1 as test executor machine
• build and use latest Suricata VNF
  • add feature for different entrypoints to run it with different rulesets (overwrite SON_EMU_CMD and SON_EMU_CMD_STOP: PED-based SON_EMU_CMD start commands #42 )
• test all using iperf probes
• add tcpreplay probes as in Gym experiments
• Monitoring (result collection) to VNF itself (docker-vnfs/suricata-ids/suricata/stats.py)
  • PROBLEM: tng-bench needs to execute stop.sh in VNFs directly, so it happens BEFORE result collection. Sic!
  • Fix: values are 0 error f=0
• create a folder with PEDs, BDs, results (in vnf-bench-model repo)
  • vnf-bench-model/experiments/upb-vnf-bench-01
  • tng-bench -p ped_suricata_experiment_small.yml -v
• build fresh PED files etc. in vnf-bench-model repo to do full experiment
  • Add traffic counter dump script to MPs
  • Gym experiment: raphaelvrosa/vnf-bench-model@ecdf101
  • small PED
  • full PED
• ensure that VNF-BD is created correctly from PEDs
• Add README.md to that folder an document what we did

---

• do a first long running tests (see Bear.app) to check stability for real experiments
  • 36
  • 180
  • 1800(~26h runtime, result folder ~323MB, *.tar.gz ~14MB)
• do the experiment (on tango5 machine (using rdev2 as ctrl?))
  • define experiment to produce example results (see repo, 36 configs)
  • fine tune times on target machine (tcpreplay)
  • Bug: Suricata numbers in results.yml are often 0. Looks like this is a result processing problem only. (I checked the "emty" case where it is normal)
  • check if resource limits are applied correctly-> new issue Calibrate and verify application of resource limits #59
    • Bug: cpuset is ignored. confirmed -> new issue Calibrate and verify application of resource limits #59
  • kick-off execution
  • increase repetitions

---

• fix integration with latest vim-emu master (multi VDU support breaks tng-bench compatibility)

---

• Result analysis? Jupyter? -> new issue Result processing and visualization using Jupyter #60

---

OpenBugs

• network setup (E-LINE) seems to be broken by latest emulator LLCM developments
  • this is the root cause: 5GTANGO LLCM: Add support for E-Line links in multi V/CDU deployments son-emu#292

[mpeuster]

raphaelvrosa/vnf-bench-model#3

[mpeuster]

Private results archive: https://git.cs.upb.de/peuster/tng-bench-result-archive
Private results and experimentation: https://git.cs.upb.de/peuster/tng-bench-experiments
(to store even buggy results and not mess up this repo)