Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Coloured Output #62

Open
JohnPaton opened this issue May 28, 2018 · 3 comments
Open

Coloured Output #62

JohnPaton opened this issue May 28, 2018 · 3 comments
Labels
enhancement New feature or request

Comments

@JohnPaton
Copy link

JohnPaton commented May 28, 2018

Currently I write my tests in unittest, but use pytest as a test runner because the coloured output makes it easier to see what's going on at a glance (plus it just looks better). This could be added to marbles as well, since nicer output is a major feature. Examples I'm thinking of:

  • Green . / red F as tests run:
..........................................F......
  • Green ok / red FAIL in verbose mode:
test_not_found (tests.test_users.TestUpdatePassword) ... ok
test_wrong_password (tests.test_users.TestUpdatePassword) ... ok
test_empty_lists (tests.test_utils.TestDropNones) ... ok
test_empty_strings (tests.test_utils.TestDropNones) ... ok
test_no_nones (tests.test_utils.TestDropNones) ... ok
test_nones (tests.test_utils.TestDropNones) ... ok
test_0_to_plus31 (tests.test_utils.TestFormatPhoneNumber) ... FAIL
test_already_formatted (tests.test_utils.TestFormatPhoneNumber) ... ok
test_bad_input (tests.test_utils.TestFormatPhoneNumber) ... ok
test_dashes (tests.test_utils.TestFormatPhoneNumber) ... ok
test_foreign (tests.test_utils.TestFormatPhoneNumber) ... ok
test_plus (tests.test_utils.TestFormatPhoneNumber) ... ok
test_spaces (tests.test_utils.TestFormatPhoneNumber) ... ok
  • Highlight important bits of failure messages
    • Red FAIL in title to make quick scrolling easier (more visible breaks between mesages)
    • Maybe some color for the actual assertion message ( '+31612345678' != 'this will fail')
    • Syntax highlighting in the source lines or traceback (I might be getting carried away here)
======================================================================
FAIL: test_0_to_plus31 (tests.test_utils.TestFormatPhoneNumber)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/johnpaton/some/path/tests/test_utils.py", line 64, in test_0_to_plus31
    self.assertEqual(output, 'this will fail')
  File "/Users/johnpaton/some/path/venv/subsidy/lib/python3.6/site-packages/marbles/core/marbles.py", line 532, in wrapper
    return attr(*args, msg=annotation, **kwargs)
marbles.core.marbles.ContextualAssertionError: '+31612345678' != 'this will fail'
- +31612345678
+ this will fail


Source (/Users/johnpaton/some/path/tests/test_utils.py):
     63 output = utils.format_phone_number(input)
 >   64 self.assertEqual(output, 'this will fail')
     65 
Locals:
	output=+31612345678
	expected=+31612345678
	input=0612345678


----------------------------------------------------------------------

This might be a good use case for colorama and/or pygments at the expense of another dependency.

@leifwalsh
Copy link
Contributor

Probably termcolor rather than pygments, but yes, this seems like a good idea. Pygments could be used on the source, but we'd have to figure out what to do with the line numbers.

This would probably require implementing our own more customized unittest.TestRunner that does more than just customize the unittest.TestResult to remove the traceback, as we do now: https://github.com/twosigma/marbles/blob/master/marbles/core/marbles/core/main.py

FWIW, pytest does this to customize its runner: https://github.com/pytest-dev/pytest/blob/master/src/_pytest/runner.py. Probably more than we need.

Might also be worth setting up Appveyor to test the colored output on windows. We should probably have windows tests anyway, and I'm not sure how we'd test this in general other than checking for explicit ANSI color sequences, which seems gross, but might be acceptable.

Thanks!

@thejunglejane thejunglejane added the enhancement New feature or request label Jun 1, 2018
@leifwalsh
Copy link
Contributor

@JohnPaton one thing you can do for now is use an alternative test runner like green. I tried it on some marbles tests we have and it does work well, you still get the marbles failure message, but within green's colored output. The stacktrace still shows up though, I wonder how hard it would be to get green to drop that.

@CleanCut
Copy link

CleanCut commented Oct 5, 2018

Not hard at all. I added -e / --no-tracebacks to green 2.13.0 just for you. Enjoy! 😉

Sorry about the delay, by the way. I totally missed the notification for CleanCut/green#191

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants