Query crash with no apparent reason why

[akoshelev]

https://draft-mpc.vercel.app/query/view/brute-cleft-1952

   Helper 3: adding new Query Query(query_id='brute-cleft-1952', current_step=None, _status=<Status.UNKNOWN: 1>, start_time=None, end_time=None, stopped=False, _log_dir=PosixPath('logs'), _status_dir=PosixPath('status_semaphore')).
Coordinator: adding new Query IPACoordinatorQuery(query_id='brute-cleft-1952', current_step=None, _status=<Status.UNKNOWN: 1>, start_time=None, end_time=None, stopped=False, _log_dir=PosixPath('logs'), _status_dir=PosixPath('status_semaphore'), paths=Paths(repo_path=PosixPath('ipa'), config_path=PosixPath('/home/ec2-user/draft/config'), compiled_id='d2d871c', _test_data_path=None), commit_hash='d2d871c', test_data_file=PosixPath('ipa/test_data/input/events-1000.txt'), size=1000, max_breakdown_key=64, max_trigger_value=7, per_user_credit_cap=64).
Coordinator: Starting: IPACloneStep(skip=False, success=None, command=LoggerOutputCommand(cmd='git clone https://github.com/private-attribution/ipa.git ipa', cwd=None, process=None), repo_path=PosixPath('ipa'))
Coordinator: setting status: status=<Status.STARTING: 2>
Coordinator: Starting: IPAUpdateRemoteOriginStep(skip=False, success=None, command=LoggerOutputCommand(cmd="git -C ipa config --add remote.origin.fetch '+refs/pull/*/head:refs/remotes/origin/pr/*'", cwd=None, process=None), repo_path=PosixPath('ipa'))
Coordinator: setting status: status=<Status.STARTING: 2>
Coordinator: Starting: IPAFetchUpstreamStep(skip=False, success=None, command=LoggerOutputCommand(cmd='git -C ipa fetch --all', cwd=None, process=None), repo_path=PosixPath('ipa'))
Coordinator: setting status: status=<Status.STARTING: 2>
Coordinator: From https://github.com/private-attribution/ipa
Coordinator:  * [new ref]           refs/pull/1104/head -> origin/pr/1104
   Helper 2: adding new Query Query(query_id='brute-cleft-1952', current_step=None, _status=<Status.UNKNOWN: 1>, start_time=None, end_time=None, stopped=False, _log_dir=PosixPath('logs'), _status_dir=PosixPath('status_semaphore')).
   Helper 1: adding new Query Query(query_id='brute-cleft-1952', current_step=None, _status=<Status.UNKNOWN: 1>, start_time=None, end_time=None, stopped=False, _log_dir=PosixPath('logs'), _status_dir=PosixPath('status_semaphore')).
   Helper 1: setting status: status=<Status.KILLED: 7>
   Helper 1: Killing: self=Query(query_id='brute-cleft-1952', current_step=None, _status=<Status.KILLED: 7>, start_time=None, end_time=None, stopped=False, _log_dir=PosixPath('logs'), _status_dir=PosixPath('status_semaphore'))
   Helper 1: adding new Query Query(query_id='brute-cleft-1952', current_step=None, _status=<Status.UNKNOWN: 1>, start_time=None, end_time=None, stopped=False, _log_dir=PosixPath('logs'), _status_dir=PosixPath('status_semaphore')).
   Helper 1: setting status: status=<Status.KILLED: 7>
Coordinator: Starting: IPACheckoutCommitStep(skip=False, success=None, command=LoggerOutputCommand(cmd='git -C ipa checkout -f d2d871c', cwd=None, process=None), repo_path=PosixPath('ipa'), commit_hash='d2d871c')
Coordinator: setting status: status=<Status.STARTING: 2>
Coordinator: HEAD is now at d2d871ca Merge pull request #1101 from akoshelev/more-tracing
Coordinator: Starting: IPACorrdinatorCompileStep(skip=False, success=None, command=LoggerOutputCommand(cmd='cargo build --bin report_collector --manifest-path=ipa/Cargo.toml --features="clap cli test-fixture" --target-dir=ipa/target-d2d871c --release', cwd=None, process=None), manifest_path=PosixPath('ipa/Cargo.toml'), target_path=PosixPath('ipa/target-d2d871c'))
Coordinator: setting status: status=<Status.COMPILING: 3>
Coordinator:     Finished `release` profile [optimized] target(s) in 0.13s
Coordinator: Starting: IPACoordinatorGenerateTestDataStep(skip=False, success=None, command=FileOutputCommand(cmd='ipa/target-d2d871c/release/report_collector gen-ipa-inputs -n 1000 --max-breakdown-key 64 --report-filter all --max-trigger-value 7 --seed 123', cwd=None, process=None, output_file_path=PosixPath('ipa/test_data/input/events-1000.txt')), output_file_path=PosixPath('ipa/test_data/input/events-1000.txt'), report_collector_binary_path=PosixPath('ipa/target-d2d871c/release/report_collector'), size=1000, max_breakdown_key=64, max_trigger_value=7)
Coordinator: setting status: status=<Status.COMPILING: 3>
Coordinator: Starting: IPACoordinatorWaitForHelpersStep(skip=False, success=None, query_id='brute-cleft-1952')
Coordinator: setting status: status=<Status.WAITING_TO_START: 4>
Coordinator: setting status: status=<Status.CRASHED: 9>
Coordinator: CRASHING! self=IPACoordinatorQuery(query_id='brute-cleft-1952', current_step=IPACoordinatorWaitForHelpersStep(skip=False, success=False, query_id='brute-cleft-1952'), _status=<Status.CRASHED: 9>, start_time=1717026837.7775867, end_time=None, stopped=False, _log_dir=PosixPath('logs'), _status_dir=PosixPath('status_semaphore'), paths=Paths(repo_path=PosixPath('ipa'), config_path=PosixPath('/home/ec2-user/draft/config'), compiled_id='d2d871c', _test_data_path=None), commit_hash='d2d871c', test_data_file=PosixPath('ipa/test_data/input/events-1000.txt'), size=1000, max_breakdown_key=64, max_trigger_value=7, per_user_credit_cap=64)
Coordinator: adding new Query Query(query_id='brute-cleft-1952', current_step=None, _status=<Status.UNKNOWN: 1>, start_time=None, end_time=None, stopped=False, _log_dir=PosixPath('logs'), _status_dir=PosixPath('status_semaphore')).
Coordinator: setting status: status=<Status.CRASHED: 9>

[eriktaubeneck]

Hmm, looks like Helper 1 got a kill signal, but unclear why. Let's see if this happens again.