fix dead forked pthread_fork children

[henderkes]

closes #2331

[withinboredom]

I'm not sure this actually addresses the issue. The reporter said forking caused it, but then doesn't seem to be actually forking in their code where the issue comes from and only provided a forking reproducer. I think we should wait until the reporter (or someone else) confirms whether it is forking or something else entirely.

It would be worth getting to the root cause of why threads hang around after a fork instead of just exiting. Shutdown functions should still run IIRC, among other things.

[henderkes]

I'm not sure this actually addresses the issue.

Maybe not the one in the issue, but it definitely solves a deadlock issue either way. Not that people should fork in a http context, but if they do it should clean up.

It would be worth getting to the root cause of why threads hang around after a fork instead of just exiting. Shutdown functions should still run IIRC, among other things.

There are multiple paths where it could get stuck:

zend_first_try {
    char *scriptName = NULL;
    while ((scriptName = go_frankenphp_before_script_execution(thread_index))) {

being one, where it waits for the next request to come in. which never happens because the Go threads aren't forked into the child process. But it would deadlock or crash either way once it attempts to do any Go runtime scheduling or cleanup. Forcing a hard exit is the only thing to sidestep go's issues there that I can think of.

We can _exit out after php shutdown handlers (just move it to just before go_frankenphp_after_script_execution, but I'm not sure that's actually correct.

[AlliBalliBaba]

Seems strange that forking would happen in a HTTP context in the first place. Shouldn't FPM have the same problem as it also executes requests in a loop in these child processes?

[henderkes]

Seems strange that forking would happen in a HTTP context in the first place. Shouldn't FPM have the same problem as it also executes requests in a loop in these child processes?

I've not looked at the code extensively, but it seems that the child processes exit and show up as zombies, rather than sleeping (waiting for the next request). They should be reaped at some point, but even if they aren't. they don't consume resources anymore.

[henderkes]

Good golly, can we silence the "requested a deployment" notices?

[henderkes]

I had to revert this to exit out before php shutdown runs. I'm not sure why, but if we exit out after shutdown it crashes.

[dunglas]

Good golly, can we silence the "requested a deployment" notices?

I don't think so... Maybe will I revert this "environement secrets" change because it's very annoying!

[henderkes]

I'm actually toying around with a (probably very bad ) idea of using forking semantics to share current execution state from parent to children. I very often find myself wanting to do something like this in Symfony

public function revenue(DateTime $start, DateTime $end, InvoiceRepository $invoiceRepository): Response {
    $years = [
        new Year($start, $end),
        new Year((clone $start)->modify('-1 year'), (clone $end)->modify('-1 year')),
        new Year((clone $start)->modify('-2 year'), (clone $end)->modify('-2 year')),
    ];

    $futures = array_map(fn(Year $y) => \parallel\run(function () use ($y, $invoiceRepository) {
        foreach ($invoiceRepository->findByDateRange($y->start, $y->end) as $invoice) {
            $m = (int) $invoice->getDate()->format('n');
            ($y->months[$m] ??= new Month($m, 0))->total += $invoice->getTotal();
        }
        return $y;
    }), $years);

    return $this->render('statistics.html.twig', [
        'years' => array_map(fn(Future $f) => $f->value(), $futures),
    ]);
}

Which is... kind of possible with ext-parallel, but very inconvenient. I'd need to create a bootstrap script which creates a new kernel, a service locator and tag all services I need to use (or make them public), refetch every entity and think about a way to serialize the information pulled from them to pass back to the parent. Forking should make it possible to write code like above which captures state using php's default semantics, the only thing that would need to go on in the background is re-estabishing database connections or file descriptors in each child, then even lazy properties passed in should work perfectly well in parallel.

Of course it's not quire as elegant as actually having threads, but I can't think of a way to do it with just threads. There's no ready method for completely cloning a threads entire php execution state.

[henderkes]

1 thread:

8 threads:

Can freely share managed entities from parent to children and back, lazy loading works. Children can use the entire runtime, render with twig, access $this context in arrow functions (their own, but initially the same state as the parents), mailer, queue, mercure, or anything else.