Improving the docs: browser testing!

[marteki]

The testing guidelines have very little information about what functional browser testing looks like in D&D, and the stuff that's there now might be a bit out-of-date.

Also, in different code reviews of the same browser test spec, I've received conflicting information from reviewers about what "best practice" should be for a particular line. Maybe we could add some more guidance around what our best practices are for browser testing?

Current behavior

• From the "Browser testing" section of the "Testing" guidelines page:

  Most CFPB projects use behave for browser testing.

• From the code reviews I've described: no guidance on whether we have a clear preference for toBe.(true) vs. toBe­Tru­thy() vs. toBeTrue(), and when we might choose one over another.

Expected behavior

• Inclusion of protractor in the description of what we use. It's used across consumerfinance.gov in general (and particularly on some specific tools).
• If there are coding best practices that we have a clear preference for at the Bureau, we state what they are. For example, if we prefer to use .toBeTrue() or .toBe(true) to evaluate the value produced by something like .isDisplayed() rather than toBe­Tru­thy() (because .isDisplayed() will always deliver a Boolean promise), we could state that. And if we truly want it up to the discretion of each individual programmer, we state that.

[KimberlyMunoz]

Thanks for bringing this up!

There is a difference between truthy and true. toBeTruthy (Jasmine) / to.be.ok (Chai BDD) and toBeTrue() (Jasmine) /``to.be.true (Chai BDD) in how they'll evaluate out and we've tried to use true over truthy whenever we can because it is more strict. I think guidance on that is helpful!

I don't think we should issue guidance on what type of syntax to use. It looks like all the syntax above is in Jasmine, which is the best supported tool for Protractor, but there's been talk about switching to Mocha Chai to keep it consistent with our unit tests. We also want to leave ourselves open in case there's a better tool for testing other things like React. I think we should describe what is used rather than offer guidelines here. (Also, I'm not sure what syntax uses toBeTrue(), I tried searching in projects to research it but came up empty.)

[marteki]

There is a difference between truthy and true. toBeTruthy (Jasmine) / to.be.ok (Chai BDD) and toBeTrue() (Jasmine) /to.be.true (Chai BDD) in how they'll evaluate out and we've tried to use true over truthy whenever we can because it is more strict. I think guidance on that is helpful!

Then maybe that's all the guidance needed, rather than syntax standards: "When looking at how things evaluate, be as strict as possible."

(Personal note: when first starting Jasmine/Protractor browser tests at the Bureau, I had tests written using toBe( true) and was advised to change them to toBeTruthy(). Several months later, told by someone else that they should be changed back. Knowing that we lean more towards strict evaluation would be helpful to know when deciding if a code review change request is actually the best choice for a particular situation.)

Finally, toBeTrue() is used in a Jasmine add-on. I hadn't checked if the Bureau was using it specifically, but it could be something that outside folks are using or new folks have experience with. Maybe another vote in the "don't talk about specific syntax" column!

[KimberlyMunoz]

@sebworks has some good starter notes here: https://gist.github.com/sebworks/edb4c89c4e8c244812f2dac3d6f77a89

I'm prodding him to add to them to the repo!