Discuss: End to End testing ideas

[MichaelDimmitt]

High level Ask

Applications are in the following languages the ask is to have 1 test suite that can run them all !

1. python
2. ruby
3. elixir

This test suite could be written in

1. bash (with some sort of network request interceptor)
2. node (with some sort of network request interceptor)
3. headful/headless terminal similar to cypress but for terminal instead of for the browser.

Basic tests needed:

1. intercept network requests and mock a response to the api
   2.intercept network request and mock git clone action.
2. accept an array of url's driven by an organization name.
3. github no response or fail.

What is a good failure message? (catch all errors!)

"Oops! Check the logs here:"at this file" for more info.
If not into nerdy things open an issue on our repo."

alternate error message:
"in my opinion tell the user none found or something went wrong and check the logs at this file for more info."

Nitty Gritty (ideating end to end test solutions)

capture network requests and send a mock response.

tcpdump
fiddler(paid)

ideas around bats

what if bats has a way to intercept netwrok requests instead of sending them.

https://github.com/bats-core/bats-detik
random detik kubernetes test: https://vzurczak.wordpress.com/2019/07/09/end-to-end-tests-for-applications-in-kubernetes/

node solution

found a library nock that might be able to intercept network requests in node process
https://github.com/nock/nock/blob/feaa66fa64d24f95937ef759cdd5a7ca07646f1c/tests/test_ipv6.js#L14

other links:
https://stackoverflow.com/questions/28734511/intercept-mock-http-requests-on-node-js-and-browser

[MichaelDimmitt]

What actually needs to be tested?

1. The application still works based on dependant language versions and dependent package managers.
   This could be added ci to tell the day and version that the latest stable failed for our code implementation.

Python example:

• python -v
• pip -v

Elixir example:

• elixir -V
• erlang -V
• hex -V