Adjust popen timeouts for testing

[fjetter]

This popen utility is often used in our CLI tests. The timeouts here are however not well adjusted to our overall test timeouts. Particularly if this is used inside of an async gen_test / gen_cluster test these timeouts do not match since the gen_cluster default test timeout is 30s.

Therefore, the async test timeout of 30s will always win over this popen timeout but timeouts should be hierarchical since they otherwise overshadow problems. The test test_nanny_worker_port_range_too_many_workers_raises for instance is failing intermittently on CI but there is no way to tell where or why it is stuck because the async timeout hits before anything else is happening.

Generally, 30s and 10s for a process termination / kill is way too generous so I suggest to dial both of them down a bit

[fjetter]

Same goes for distributed.cli.tests.test_dask_spec.test_errors. Since the asyncio timeout hits we don't even get the subprocess output

[github-actions]

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

    25 files  ±0      25 suites  ±0   10h 25m 5s ⏱️ + 8m 12s
 4 129 tests ±0   4 009 ✅  -  9    110 💤 ±0  10 ❌ + 9 
47 698 runs  ±0  45 589 ✅  - 13  2 095 💤 ±0  14 ❌ +13 

For more details on these failures, see this check.

Results for commit d1f5013. ± Comparison against base commit 36020d6.

♻️ This comment has been updated with latest results.

[fjetter]

I think I'm OK if the process does not shut down with just SIGINT so I'm catching this and just print the information

[fjetter]

Thought about this again. I think we should raise after all. Having this not work well is a little unhealthy