Add support for custom database setup #228

millerdev · 2015-08-05T02:55:38Z

Adds two new options to the DatabaseSetUpPlugin (was TestReorderer) plugin:

--db-test-context=CLASS_PATH - override context object for tests that need a database (i.e., ones that extend TransactionTestCase or one of its subclasses).
--non-db-test-context=CLASS_PATH - context object for tests that do not need a database (i.e., ones that do not extend TransactionTestCase or one of its subclasses). This can greatly speed up development if many tests do not require a database, especially when running nose with --stop.

"Non-database" tests are now run first instead of last.

millerdev · 2015-08-05T03:02:29Z

Looks like there's still a bit of work to do as tests are failing. I'll look into that tomorrow.

millerdev · 2015-08-05T12:28:33Z

I see a similar test failure when I run tox with a fresh copy of django-nose:

$ git clone [email protected]:django-nose/django-nose.git
Cloning into 'django-nose'...
remote: Counting objects: 1226, done.
remote: Total 1226 (delta 0), reused 0 (delta 0), pack-reused 1226
Receiving objects: 100% (1226/1226), 295.84 KiB | 0 bytes/s, done.
Resolving deltas: 100% (636/636), done.
Checking connectivity... done.
$ cd django-nose
$ mkvirtualenv djnose
New python executable in djnose/bin/python
Installing setuptools, pip...done.
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean
$ pip install tox
Downloading/unpacking tox
  Downloading tox-2.1.1-py2.py3-none-any.whl
Downloading/unpacking pluggy>=0.3.0,<0.4.0 (from tox)
  Downloading pluggy-0.3.0-py2.py3-none-any.whl
Downloading/unpacking virtualenv>=1.11.2 (from tox)
  Downloading virtualenv-13.1.0-py2.py3-none-any.whl (1.7MB): 1.7MB downloaded
Downloading/unpacking py>=1.4.17 (from tox)
  Downloading py-1.4.30-py2.py3-none-any.whl (81kB): 81kB downloaded
Installing collected packages: tox, pluggy, virtualenv, py
Successfully installed tox pluggy virtualenv py
Cleaning up...
$ tox -e py34-django-18
GLOB sdist-make: /Users/me/code/django-nose/setup.py
py34-django-18 create: /Users/me/code/django-nose/.tox/py34-django-18
py34-django-18 installdeps: coveralls, dj-database-url, Django>=1.8,<1.9
py34-django-18 inst: /Users/me/code/django-nose/.tox/dist/django-nose-1.4.1.zip
py34-django-18 installed: coverage==3.7.1,coveralls==0.5,dj-database-url==0.3.0,Django==1.8.3,django-nose==1.4.1,docopt==0.6.2,nose==1.3.7,PyYAML==3.11,requests==2.7.0,wheel==0.24.0
py34-django-18 runtests: PYTHONHASHSEED='2289617906'
py34-django-18 runtests: commands[0] | ./runtests.sh
PASS (count==6): normal settings
PASS (  --help): normal settings
PASS (count==6): django_nose.run_tests format
PASS (  --help): django_nose.run_tests format
PASS (count==1): via run_tests API
PASS (  --help): via run_tests API
PASS (count==1): with plugins
PASS (  --help): with plugins
PASS (count==4): unittests
PASS (  --help): unittests
PASS (count==4): unittests with testrunner
PASS (  --help): unittests with testrunner
FAIL (test failure): with REUSE_DB=1, call #1
stdout
======
nosetests --verbosity=1

stderr
======
..EE..
======================================================================
ERROR: Test that votes is initialized to 0.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/backends/sqlite3/base.py", line 318, in execute
    return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: no such table: testapp_question

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/me/code/django-nose/testapp/tests.py", line 31, in test_question
    question_text="What is your quest?", pub_date=datetime(1975, 4, 9))
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/models/manager.py", line 127, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/models/query.py", line 348, in create
    obj.save(force_insert=True, using=self.db)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/models/base.py", line 710, in save
    force_update=force_update, update_fields=update_fields)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/models/base.py", line 738, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/models/base.py", line 822, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/models/base.py", line 861, in _do_insert
    using=using, raw=raw)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/models/manager.py", line 127, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/models/query.py", line 920, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 974, in execute_sql
    cursor.execute(sql, params)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/utils.py", line 97, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/utils/six.py", line 658, in reraise
    raise value.with_traceback(tb)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/backends/sqlite3/base.py", line 318, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such table: testapp_question

======================================================================
ERROR: test suite for <class 'testapp.tests.UsesFixtureTestCase'>
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/backends/sqlite3/base.py", line 318, in execute
    return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: no such table: testapp_question

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/nose/suite.py", line 210, in run
    self.setUp()
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/nose/suite.py", line 293, in setUp
    self.setupContext(ancestor)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/nose/suite.py", line 316, in setupContext
    try_run(context, names)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/nose/util.py", line 471, in try_run
    return func()
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/test/testcases.py", line 956, in setUpClass
    'database': db_name,
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/core/management/__init__.py", line 120, in call_command
    return command.execute(*args, **defaults)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/core/management/base.py", line 444, in execute
    output = self.handle(*args, **options)
  File "/Users/me/code/django-nose/django_nose/runner.py", line 378, in _foreign_key_ignoring_handle
    _old_handle(self, *fixture_labels, **options)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/core/management/commands/loaddata.py", line 60, in handle
    self.loaddata(fixture_labels)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/core/management/commands/loaddata.py", line 90, in loaddata
    self.load_label(fixture_label)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/core/management/commands/loaddata.py", line 147, in load_label
    obj.save(using=self.using)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/core/serializers/base.py", line 173, in save
    models.Model.save_base(self.object, using=using, raw=True)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/models/base.py", line 738, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/models/base.py", line 803, in _save_table
    forced_update)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/models/base.py", line 853, in _do_update
    return filtered._update(values) > 0
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/models/query.py", line 580, in _update
    return query.get_compiler(self.db).execute_sql(CURSOR)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 1062, in execute_sql
    cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 840, in execute_sql
    cursor.execute(sql, params)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/utils.py", line 97, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/utils/six.py", line 658, in reraise
    raise value.with_traceback(tb)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/Users/me/code/django-nose/.tox/py34-django-18/lib/python3.4/site-packages/django/db/backends/sqlite3/base.py", line 318, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: Problem installing fixture '/Users/me/code/django-nose/testapp/fixtures/testdata.json': Could not load testapp.Question(pk=1): no such table: testapp_question

----------------------------------------------------------------------
Ran 5 tests in 0.061s

FAILED (errors=2)
ERROR: InvocationError: '/Users/me/code/django-nose/runtests.sh'
___________________________________ summary ____________________________________
ERROR:   py34-django-18: commands failed

I'm not sure what's causing it. Confused.

millerdev · 2015-08-05T13:09:45Z

Also, tests that are failing in travis are passing for me:

$ git checkout fast-first
Switched to branch 'fast-first'
$ tox -e py27-django-18
GLOB sdist-make: /Users/me/code/django-nose/setup.py
py27-django-18 inst-nodeps: /Users/me/code/django-nose/.tox/dist/django-nose-1.4.1.zip
py27-django-18 installed: coverage==3.7.1,coveralls==0.5,dj-database-url==0.3.0,Django==1.8.3,django-nose==1.4.1,docopt==0.6.2,nose==1.3.7,PyYAML==3.11,requests==2.7.0,wheel==0.24.0
py27-django-18 runtests: PYTHONHASHSEED='1267070122'
py27-django-18 runtests: commands[0] | ./runtests.sh
PASS (count==6): normal settings
PASS (  --help): normal settings
PASS (count==6): django_nose.run_tests format
PASS (  --help): django_nose.run_tests format
PASS (count==1): via run_tests API
PASS (  --help): via run_tests API
PASS (count==1): with plugins
PASS (  --help): with plugins
PASS (count==4): unittests
PASS (  --help): unittests
PASS (count==4): unittests with testrunner
PASS (  --help): unittests with testrunner
PASS (count==6): with REUSE_DB=1, call #1
PASS (  --help): with REUSE_DB=1, call #1
PASS (count==6): with REUSE_DB=1, call #2
PASS (  --help): with REUSE_DB=1, call #2
/Users/me/code/django-nose/runtests.sh: line 43: gawk: command not found
/Users/me/code/django-nose/runtests.sh: line 43: gawk: command not found
py27-django-18 runtests: commands[1] | coverage combine
___________________________________ summary ____________________________________
py27-django-18: commands succeeded
  congratulations :)

jwhitlock · 2015-08-05T13:20:44Z

I have a laptop without gawk installed, and I'm getting similar results. I think I can assume Python 2 and write a version that doesn't use gawk. That may turn this into a local failure as well.

millerdev · 2015-08-05T14:32:34Z

Not sure I follow. Locally on this PR and master I'm getting:

$ tox -e py34-django-18
...
PASS (count==4): unittests with testrunner
PASS (  --help): unittests with testrunner
FAIL (test failure): with REUSE_DB=1, call #1

and on this PR I'm seeing

$ tox -e py27-django-18
...
PASS (count==4): unittests with testrunner
PASS (  --help): unittests with testrunner
PASS (count==6): with REUSE_DB=1, call #1
PASS (  --help): with REUSE_DB=1, call #1
PASS (count==6): with REUSE_DB=1, call #2
PASS (  --help): with REUSE_DB=1, call #2

It's true, I don't have gawk installed, but are you saying that's the cause of the failure on master?

jwhitlock · 2015-08-05T14:43:36Z

No, it's two separate issues. The first is this bit:

/Users/me/code/django-nose/runtests.sh: line 43: gawk: command not found
/Users/me/code/django-nose/runtests.sh: line 43: gawk: command not found

That's keeping the profile test from running, which is a minor thing. I've pushed a fix to master, because it's a small fixable thing.

The other is that REUSE_DB=1 seems to cause intermittent failures with the fixture tests. Given the many problems with REUSE_DB=1 and FastFixtureTestCase, this can be expected. I'm looking through the nose docs to see if there is a way to mark the test as failures OK, so we can track failures but not fail PRs.

On my system, I did see the "Problem installing fixture" issue once when running runtests.sh, but it went away on the second run. It consistently fails for

COVERAGE=1 RUNTESTS_ARGS="--noinput" tox -e py34-django-18
COVERAGE=1 RUNTESTS_ARGS="--noinput" tox -e py34-django-master

Interesting, but probably has nothing to do with your PR, other than I'm reluctant to add new features when these legacy features are so broken.

millerdev · 2015-08-05T15:01:44Z

Thanks for the clarification.

jwhitlock · 2015-08-05T15:13:59Z

After some experimenting, I pushed another commit that runs the REUSE_DB=1 tests, but doesn't fail runtests.sh if they fail. That seems like the best option until the feature is rewritten or dropped.

millerdev · 2015-08-05T16:21:44Z

Unfortunately I have not been able to fix the issue turned up by the profile plugin test. It's a legit problem with the way this new feature puts groups of tests within contexts, and I'll need to think through how do do that in a way that does not interfere with other plugins (to the extent possible). It might be a while until I get around to fixing that issue.

Custom database setup/teardown can be done by specifying --db-test-context with a custom context class path. Database setup/teardown can be deferred until needed (or completely omitted if no tests require it) with --non-db-test-context and an appropriate class path such as `django_nose.plugin.NullContext`.

Fixes “with profile plugin” tests

millerdev · 2015-10-07T21:15:56Z

I fixed the profile plugin test, but it appears that one of the travis jobs hung. What now?

jwhitlock · 2015-10-07T23:24:35Z

I bumped TravisCI - the job is unstuck and passed

millerdev · 2015-10-10T18:44:33Z

@jwhitlock anything else you'd like to see before this can be merged?

jwhitlock · 2015-10-12T01:21:53Z

I'm reluctant to add new features to this creaky, bug-prone project, and this one seems to touch a lot of code even for people that don't want it. I haven't dived into the code, but my first impression is it would be better suited as a plugin, an alternate to the default TestReorder plugin instead of replacing it. Spinning features out as plugins in separate repos may be the theme of the 1.5 release cycle.

At a minimum, I'm going to try this branch on my own projects, and see if it breaks things or makes them faster.

Also fix missing django-nose options.

Make it possible to use module-level setup/teardown functions in modules with database tests.

jwhitlock added the feature label Aug 5, 2015

millerdev added 7 commits October 7, 2015 15:31

Run non-db tests first

bce933a

Allow custom non-db test context

b9b4ef6

Improve test grouping

50b1cfe

Python 2.6 and flake8

3d5aa62

Document custom database contexts

a4034f8

Restore sys.stdout for db delete prompt

b8568ed

millerdev force-pushed the fast-first branch from 72a35e4 to 3dde874 Compare October 7, 2015 19:33

Make DatabaseSetupPlugin work with other plugins

be176ba

Fixes “with profile plugin” tests

millerdev force-pushed the fast-first branch from abf501a to be176ba Compare October 7, 2015 19:48

millerdev added 2 commits October 20, 2015 22:34

Add --db=(skip|only) for easy test selection

472adb3

Also fix missing django-nose options.

Add support for database tests in context suites

e581fd7

Make it possible to use module-level setup/teardown functions in modules with database tests.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for custom database setup #228

Add support for custom database setup #228

millerdev commented Aug 5, 2015

millerdev commented Aug 5, 2015

millerdev commented Aug 5, 2015

millerdev commented Aug 5, 2015

jwhitlock commented Aug 5, 2015

millerdev commented Aug 5, 2015

jwhitlock commented Aug 5, 2015

millerdev commented Aug 5, 2015

jwhitlock commented Aug 5, 2015

millerdev commented Aug 5, 2015

millerdev commented Oct 7, 2015

jwhitlock commented Oct 7, 2015

millerdev commented Oct 10, 2015

jwhitlock commented Oct 12, 2015

Add support for custom database setup #228

Are you sure you want to change the base?

Add support for custom database setup #228

Conversation

millerdev commented Aug 5, 2015

millerdev commented Aug 5, 2015

millerdev commented Aug 5, 2015

millerdev commented Aug 5, 2015

jwhitlock commented Aug 5, 2015

millerdev commented Aug 5, 2015

jwhitlock commented Aug 5, 2015

millerdev commented Aug 5, 2015

jwhitlock commented Aug 5, 2015

millerdev commented Aug 5, 2015

millerdev commented Oct 7, 2015

jwhitlock commented Oct 7, 2015

millerdev commented Oct 10, 2015

jwhitlock commented Oct 12, 2015