Worker pool handles spawned process

[chromium7]

Fix #651
Ref #650

I can't reproduce the issue in #650, but I did get the same errors in #651
Seems like its an issue with multiprocessing start method in certain OS, as was answered in this stack overflow post: https://stackoverflow.com/questions/46908035/apps-arent-loaded-yet-exception-occurs-when-using-multi-processing-in-django

This solution builds on top of @jackkinsella answer, and can be updated further to handle his specific issue.

[jackkinsella]

FWIW The version of this function we're using in production was.

def run_django_worker(*args: Any, **kwargs: Any) -> None:
    alias = "default"

    try:
        connections[alias].close()
    except Exception:
        logger.error(f"Worker could not get a connection to the database with alias {alias}")

    run_worker(*args, **kwargs)

The exception catching and logging was transient and didn't stop us from using a 16 worker queue to process a few 100k events per day

[jackkinsella]

The django.setup() solution is likely better (but I don't understand it so can't offer feedback)

[jackkinsella]

I just checked production (Heroku - unix) and get_start_method() was "fork". I know that simply executing run_worker() will not work in that environment due to the psycog issues with the DB connection from pre-fork.

(Maybe this explains the failing CI test?)

[chromium7]

Yes, I just realized that db connections are also closed pre-fork in the rqworker management command.

django-rq/django_rq/management/commands/rqworker.py

Lines 95 to 96 in d09421d

	# Close any opened DB connection before any fork
	reset_db_connections()

I called django.setup() because the spawned process doesn't load the installed apps at all. This works fine when I manually tested it, but fails in unit test. Probably its because django.setup() is not setting up the project in testing environment which leads to the process setting up its own DB instance.

I'm currently using this django code for parallel test runner as a reference. I'll try to update the PR in a day or two.
https://github.com/django/django/blob/53719d6b5b745dd99b1ab9315afb242f706ebbf1/django/test/runner.py#L424-L432

[selwin]

Similar to @jackkinsella's comment, we need to use our own run_worker command that calls reset_db_connections before running the original run_worker command:

django-rq/django_rq/utils.py

Line 150 in 025a53f

def reset_db_connections():