At one of my old jobs, we had a suite of browser tests that would run on PR. It’d stand up the application, open headless chrome, and click through stuff. This was the final end-to-end test suite to make sure that yes, you can still log in and everything plays nicely together.
Developers were constantly pinging slack about “why is this test broken??”. Most of the time, the error message would be like “Never found an element matching css selector #whatever” or “Element with css selector #loading-spinner never went away”. There’d be screenshots and logs, and usually when you’d look you’d see like the loading spinner was stuck, and the client had gotten a 400 back from the server because someone broke something.
We put a giant red box on the CI/CD page explaining what to do. Where to read the traces, reminding them there’s a screenshot, etc. Still got questions.
I put a giant ascii cat in the test output, right before the error trace, with instructions in a word bubble. People would ping me, “why is this test broken?”. I’d say “What did the cat say?” They’d say “What cat?” And I’d know they hadn’t even looked at the error message.
There’s a kind of learned helplessness with some developers and tests. It’s weird.
My #1 pet peeve is when someone comes to me with a problem, and the solution is in the fucking console output or error message.
On a bad day, if I had unilateral power, I would fire those people on the spot.
At one of my old jobs, we had a suite of browser tests that would run on PR. It’d stand up the application, open headless chrome, and click through stuff. This was the final end-to-end test suite to make sure that yes, you can still log in and everything plays nicely together.
Developers were constantly pinging slack about “why is this test broken??”. Most of the time, the error message would be like “Never found an element matching css selector #whatever” or “Element with css selector #loading-spinner never went away”. There’d be screenshots and logs, and usually when you’d look you’d see like the loading spinner was stuck, and the client had gotten a 400 back from the server because someone broke something.
We put a giant red box on the CI/CD page explaining what to do. Where to read the traces, reminding them there’s a screenshot, etc. Still got questions.
I put a giant ascii cat in the test output, right before the error trace, with instructions in a word bubble. People would ping me, “why is this test broken?”. I’d say “What did the cat say?” They’d say “What cat?” And I’d know they hadn’t even looked at the error message.
There’s a kind of learned helplessness with some developers and tests. It’s weird.