Busted Output (wip) #86

Mathijs-Bakker · 2021-03-02T09:09:34Z

Yo! Here a draft... which is not perfect yet.
But maybe you could review it and see if I am on the right track.

Related issue: #73

Still have many things to address...

Like:

Show spec time,
Show total elapsed time in spec score,
Exit codes (only show one exit code at the end),
Prettying output,
Fetch last fixes from main repo,
...

I am still a big noob at Lua, so constructive comments are welcome!

Current behavior/screens:

Headless:

Not headless:

Conni2461 · 2021-03-02T09:30:42Z

Okay thats a lot of work. Good job. I will need to take a in depth look at this, this evening. One thing you can do, is update your description with a screenshot how it currently looks :)

Another thing we will probably do, after the conflicts are resolved, moves some files around:

lua/plenary/busted.lua -> lua/plenary/busted/init.lua
lua/plenary/busted_dots.lua -> lua/plenary/busted/dots.lua
lua/plenary/busted/reader.lua -> lua/plenary/busted/reader.lua

require'plenary.busted' will still work and load the init.lua file. I do everything else later today

Mathijs-Bakker · 2021-03-02T09:50:42Z

I tried to respect current code and file structure. But yes... better to move them to it's own busted dir.
(btw... i added some screenies)

Mathijs-Bakker · 2021-03-02T09:57:05Z

Note: Passing tests are still shown. But will be removed from visual output later. You can consider them redundant as you would typically show only the failing tests.

The output of: pending tests and errors, still need to be handled.

Conni2461 · 2021-03-02T20:39:19Z

lua/plenary/busted.lua

    local trace = get_trace(nil, 3, msg)
-    return trace.message .. "\n" .. trace.traceback
+    -- return trace.message .. "\n" .. trace.traceback
+    return trace.message


We should keep the traceback

kkharji · 2021-03-02T20:40:01Z

That looks really nice and clean @Mathijs-Bakker . I'd suggest to take out the path in front of Expected ... since it mentioned above it.

Conni2461 · 2021-03-02T20:46:14Z

We can make the path relative with plenary.path

local new_path = Path:new(fullpath):make_relative()

I did look at the code and gave this a try. My 52 test file didn't run. So i am not sure if this works as expected. Would be great if you could point me to code that doesn't work.

Thats my output of the file i just ran:

Its one spec file but i get this multiple ENDOFSPEC things. Maybe that helps :) Also not all tests in plenary.nvim are running.

Edit: thats what is happening on master

Mathijs-Bakker · 2021-03-03T08:49:31Z

Thanks, I will get into it.

Conni2461 · 2021-03-04T09:42:21Z

I am sorry, i am really busy lately. I will do a productive review this evening :)

Mathijs-Bakker · 2021-03-04T10:22:00Z

I am sorry, i am really busy lately. I will do a productive review this evening :)

Nothing new... I only synched with origin and solved a merge conflict.
When I pushed this to my own fork/branch this draft gets auto updated too.
(Fun fact: I was googling for a solution on 'how to update/push to a remote draft'. 🤣)
Gonna create a new branch to deviate from this one... So participants don't get notified on every push I do.

Mathijs-Bakker · 2021-03-06T09:50:18Z

Ok, it's weekend, time to move on with this thing...

I need some advise on this:

For drawing the dots and score results, we need to count the 'fail', 'err', 'pending' and passed tests.

So we need to get the tests first.

The stdout of theJob in test_harness.lua spits everything out line-by-line. (output = vim.schedule_wrap(_, line, _, _))
That's why I choose to concat the lines and store them as tests in a table.

To identify a test and it's status:
The "{SPEC: FAIL}", "{SPEC: SUCC}", etc. are added into busted.lua.
So we can find an occurrence of the specific string in a line.

E.g:

if not ok then
     to_insert = results.fail
     test_result.msg = msg

     print(FAIL, "||", table.concat(test_result.descriptions, " "))
     print(indent(msg, 12))

     print("{SPEC: FAIL}")
     local spec = get_file_and_line_number()

     print(FAIL, " → " .. color_string("cyan", spec.file) .. " @ " .. color_string("cyan", spec.linenumber) .. "\n")

     print(bold_string(table.concat(test_result.descriptions)))
     print(indent("\n" .. msg, 7))

     print("{ENDOFSPEC}")

It works, it's fast but it feels kind of hackie to me,
what do you think... maybe you know a better way?
@Conni2461

Conni2461 · 2021-03-06T11:29:50Z

I need to think about this. I will answer you this evening or tomorrow morning :) Sorry that i am not really a help right now

Conni2461 · 2021-03-07T10:12:00Z

lua/plenary/busted.lua

-    print(HEADER)
+    print(HORIZONTALRULER)
+
+    os.exit(2)


This shouldn't be here

I am looking at the whole code now. Give me between 30 ~ 60 mins for an answer. I wanna play around with the code for a bit

Okay i am trying to get nvim --headless --noplugin -u scripts/minimal.vim -c "PlenaryBustedFile tests/plenary/simple_busted_spec.lua" to work first. After that i am trying to deal with PlenaryBustedDirectory and test harness

Oh i see what you doing. I am not sure if i like it i kinda wanna keep the current way of starting a test, with just executing busted itself. We not always wanna go thought test_harness

That's exactly the reason for reaching out.
I don't like it either.
I'm afk for +/- 2 hours. So will respond later.

Here is what i think we could do. When executing the file with PlenaryBustedFile we have basically the same output, a little bit cleaned up and for Directory we can then get the SUCCESS, FAILURE and PENDING outputs and count them and also convert them to dots.

For stacktrace and error, the line always starts with a and those always come after a FAILURE so, if we read failure we go in another state that captures all further lines that start with and memorizes them for later output. And on exit we just draw it. Colored dots, Failures and summarize. We also can have simple_busted print the summarize and just realize that its a summarize and capture it that way.

If you wanna have a more detailed talk, you can write me on gitter in private.

Conni2461 · 2021-03-07T10:22:53Z

lua/plenary/busted.lua

+    local end_index = info.traceback:find(': in')
+    info.traceback = info.traceback:sub(start_index, end_index)


What are subbing away here, only busted traceback?

Conni2461 · 2021-03-07T10:26:56Z

lua/plenary/busted.lua

+
+    print(FAIL, " → " .. color_string("cyan", spec.file) .. " @ " .. color_string("cyan", spec.linenumber) .. "\n")
+
+    print(bold_string(table.concat(test_result.descriptions)))


missing " " for concat

tjdevries · 2021-04-21T18:33:21Z

@Mathijs-Bakker do you still want to continue on this PR? I'm trying to help out w/ some of these :)

Mathijs-Bakker · 2021-04-21T18:55:04Z

@tjdevries Yes definitely! I'm feeling bad for @Conni2461 that I didn't respond earlier...
There's another project that that needed my attention first.
Priorities...
It will take a few days before I am able to get back into this.
And any help from the man himself will be welcome! X)
Cheers!

Mathijs-Bakker · 2021-04-22T20:55:36Z

@Mathijs-Bakker do you still want to continue on this PR? I'm trying to help out w/ some of these :)
K, I'm getting back into it this tomorrow/weekend.

Conni2461 · 2021-10-04T19:52:11Z

I took a break too from maintaining this project 😆

If the man himself didnt help, maybe i pick this up soon :)

Conni2461 · 2022-05-13T06:00:19Z

Ill close this PR as abandoned. Ill probably fix this issue but i start from scratch.

Still thanks for looking into it and if you still work on it you can just reopen :)

Mathijs-Bakker added 16 commits February 19, 2021 13:09

Add: 'Write testfile before running' behavior, to PlenaryTestFile

d5170de

Preserve old keymap PlenaryTestFile and add PlenaryWriteAndRunTestFile

8a2486e

Update formatting of test results.

b5b4122

Changes to visual output

03bf6b7

Add file

f4d7dcc

Refactor path names

629cbe1

wip

0fb0709

Add some more eye candy

598ce0a

wip - Add filename and fail/error line in the test output header

416dffe

Wip - Dump stdout lines into a test_result table, for better filtering

e13cd17

Wip

a7393b9

Refactor

df605c3

Add busted score... bullets and textual

a99ba27

Refactor

f6befef

Add space between separators in score text

96dc3e0

Small changes to nvim output

07cccdc

Conni2461 reviewed Mar 2, 2021

View reviewed changes

Mathijs-Bakker added 2 commits March 4, 2021 09:50

Merge branch 'master' into bustin-makes-you-feel-good

aa01f56

Solve merge conflicts and fix 'output' path to be relative

adf0574

Conni2461 reviewed Mar 7, 2021

View reviewed changes

Conni2461 mentioned this pull request Oct 4, 2021

Is there a way to have a more concise output of PlenaryBustedDirectory? #237

Open

Conni2461 closed this May 13, 2022

SidOfc mentioned this pull request May 7, 2023

feat(busted): Compact test output format #488

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Busted Output (wip) #86

Busted Output (wip) #86

Mathijs-Bakker commented Mar 2, 2021 •

edited

Loading

Conni2461 commented Mar 2, 2021 •

edited

Loading

Mathijs-Bakker commented Mar 2, 2021 •

edited

Loading

Mathijs-Bakker commented Mar 2, 2021

Conni2461 Mar 2, 2021

kkharji commented Mar 2, 2021

Conni2461 commented Mar 2, 2021 •

edited

Loading

Mathijs-Bakker commented Mar 3, 2021

Conni2461 commented Mar 4, 2021

Mathijs-Bakker commented Mar 4, 2021

Mathijs-Bakker commented Mar 6, 2021 •

edited

Loading

Conni2461 commented Mar 6, 2021

Conni2461 Mar 7, 2021

Conni2461 Mar 7, 2021 •

edited

Loading

Conni2461 Mar 7, 2021

Mathijs-Bakker Mar 7, 2021

Conni2461 Mar 7, 2021

Conni2461 Mar 7, 2021

Conni2461 Mar 7, 2021

tjdevries commented Apr 21, 2021

Mathijs-Bakker commented Apr 21, 2021

Mathijs-Bakker commented Apr 22, 2021

Conni2461 commented Oct 4, 2021

Conni2461 commented May 13, 2022

		local end_index = info.traceback:find(': in')
		info.traceback = info.traceback:sub(start_index, end_index)


		print(FAIL, " → " .. color_string("cyan", spec.file) .. " @ " .. color_string("cyan", spec.linenumber) .. "\n")

		print(bold_string(table.concat(test_result.descriptions)))

Busted Output (wip) #86

Busted Output (wip) #86

Conversation

Mathijs-Bakker commented Mar 2, 2021 • edited Loading

Conni2461 commented Mar 2, 2021 • edited Loading

Mathijs-Bakker commented Mar 2, 2021 • edited Loading

Mathijs-Bakker commented Mar 2, 2021

Conni2461 Mar 2, 2021

Choose a reason for hiding this comment

kkharji commented Mar 2, 2021

Conni2461 commented Mar 2, 2021 • edited Loading

Mathijs-Bakker commented Mar 3, 2021

Conni2461 commented Mar 4, 2021

Mathijs-Bakker commented Mar 4, 2021

Mathijs-Bakker commented Mar 6, 2021 • edited Loading

Conni2461 commented Mar 6, 2021

Conni2461 Mar 7, 2021

Choose a reason for hiding this comment

Conni2461 Mar 7, 2021 • edited Loading

Choose a reason for hiding this comment

Conni2461 Mar 7, 2021

Choose a reason for hiding this comment

Mathijs-Bakker Mar 7, 2021

Choose a reason for hiding this comment

Conni2461 Mar 7, 2021

Choose a reason for hiding this comment

Conni2461 Mar 7, 2021

Choose a reason for hiding this comment

Conni2461 Mar 7, 2021

Choose a reason for hiding this comment

tjdevries commented Apr 21, 2021

Mathijs-Bakker commented Apr 21, 2021

Mathijs-Bakker commented Apr 22, 2021

Conni2461 commented Oct 4, 2021

Conni2461 commented May 13, 2022

Mathijs-Bakker commented Mar 2, 2021 •

edited

Loading

Conni2461 commented Mar 2, 2021 •

edited

Loading

Mathijs-Bakker commented Mar 2, 2021 •

edited

Loading

Conni2461 commented Mar 2, 2021 •

edited

Loading

Mathijs-Bakker commented Mar 6, 2021 •

edited

Loading

Conni2461 Mar 7, 2021 •

edited

Loading