Sending large number of tests results in several not being run.

[krusynth]

I've been trying to basically test every browser back to IE8 for our project, but it looks like about a quarter are just not running. Here is a chunk of them that failed just from trying to run 20 versions of Firefox:

https://saucelabs.com/tests/c7ab1da7edcc48c880cc9baca9ffc5bb
https://saucelabs.com/tests/3e3cb03f8269428c9b21aa6438a04b60
https://saucelabs.com/tests/984ff86faeae43679445e158003268e8
https://saucelabs.com/tests/0f521cfede1547e68fa0a368a40f4182
https://saucelabs.com/tests/d0e9738c4d254df081a0fc7e8de4479b
https://saucelabs.com/tests/2ad48a7e338d4f22bb1ec941f044ac15
https://saucelabs.com/tests/675ab877284a453cb476db46519149d2

It doesn't appear to be a queue-limit issue, and this works fine for small batches (<6). SauceLabs has responded to the issue with:

I am not sure what is happening with Grunt-sauce but from the description of the failure it does seem to be an issue with the way it is handling jobs. Grunt sauce, though somewhat affiliated, is not maintained by Sauce Labs itself. I can see that the VM's are starting but appear to not be receiving any additional information from grunt sauce other than to close the session. With that in mind, I think the best option here would be to contact the grunt sauce project as they will be best equipped to help.

I'm a bit out of my depth here, what's the best way to debug further? Is this maybe related to #160 ?

[Jonahss]

You're going to have to look at each of these tests individually, since there seem to be a lot of different things going on.

Here's some good places to look:

• some of your jobs are failing with javascript errors when the page loads. Click on the "Metadata" tab and click on "firefox log" button. You'll see client side js errors there. Check out this one, or this one
• some of your jobs throw javascript errors on the test reporting code. So the tests pass, but never report their results to the sauce servers. You can see those errors in the returned json for the command. See this job or this one

Whenever you are in doubt, try running a Sauce manual test and visit your testpage. Open the javascript developer console and see if there are any errors. Then see if you can access window.mochaResults

Once we weed out those failures, let's tackle the more troublesome failures.