The Almighty Test Case Pt.7

We’re getting there, the end is in sight!

Today, we should be able to clear the final two sections, Labelling and Examples, of what constitutes one of my Test Scripts and that should allow us to do a full wrap up next week.

It’s been quite a journey all the way from the top Test Suite grouping down to the low level Test Script data hub so I feel we need a full overview next week before we can sign this all off.

But before we do that, it’s our second tier of Labelling.

Labelling

If you recall, our Test Case also has a labelling section:

This is crucial. Without consistent bug labels, the entire tracking system would fall apart. This section spells out:

🔹Any naming conventions or tags for test sessions, logs, or test runs
🔹What labels must be applied to bugs found via this Test Case

And Test Scripts do the same but with a more “refined” label if required, otherwise it just reiterates the labels found on the Test Case itself.

Why duplicate? Because labels are that important! Didn’t you read the “Is It Trustworthy?” series?

But on top of that duplication, having labelling in Test Scripts really allows for finer data analysis. For example, the CR1000 Test Case would tell you to include the label “QA-Gen_Crash” on all CR1000 bugs. CR1000C would say to have “QA-Gen_Crash” & “QA-Gen_Hang”, CR1000E would have “QA-Gen_Crash” & “QA-Debug”.

Nothing new, nothing crazy but important nonetheless.

Now the next section may be new to some people but it is key to giving a test or a dev a foot up in their work.

Pass & Failure Examples

Now, what I have seen outside of Sony has been all bare Test Cases. You get your Test Case, you perform the steps and log any bugs found. That’s it. One in, one out (of test).

Like a fleeting wisp of info, lost on the QA breeze. (Which, back in 2008 was a rather pungent breeze but things have improved since then.)

Now, a tester could look through Jira for bugs that previously failed the Test Case but that’s tedious and time consuming and so in our Test Scripts, we cherry pick interesting bugs! This is a trick I bring with me from Sony.

Pass Examples

Why would someone want to know what passed a Test Case previously?

Loads of reasons!

🔹Shows interpretation of generic steps
🔹Shows historical coverage
🔹Helps devs
🔹Helps new testers
🔹Clarifies ambiguous cases

Pass Examples future-proof and develop institutional knowledge.

These are Generic Test Cases, the Test Methods are not super strict. They’re open to some interpretation to allow for that tester creativity. But when there’s room for interpretation, sometimes, it’s helpful to know how someone else interpreted it.

Maybe there was a subtle change between builds that meant a specific Test Script looked like it no longer applied but it had previously been tested. A little screenshot and a caption clears that hesitation right up.

Never, don’t forget, our Generic Test Suites are an information hub that will move between projects, not just builds. The intention being that they will last years. These “Pass Examples” effectively become a guide for Best Practices for devs. Imagine they got caught out by a saved data bug, they can look up the Test Case and see how other devs addressed it on previous projects.

Or even simply, you have a new, fresh to QA, Junior Tester on the team and they’re unsure of how to apply your teams nomenclature to the types of games they’re been playing for years with their own naming scheme.
“Objective Delivery System, what’s one of those?… Ohhhhh, thanks Pass Example!”

That’s exactly what they would say.

Simple but powerful and all it requires is, once a bug is written, just pop a screenshot and summary onto the Test Script documentation. Minimal work, almost maximum info sharing.

Failure Examples

Can you guess how this one goes? Yeah, it’s basically the same only with one extra bit of info.

Obviously, well hopefully, any issue that results in a bug logged under the Test Case will be fixed at some point. In our failure examples, we log the fix. We just ask the dev for a little summary and we can have under the screenshot.

Failure Example 1 - Project XYZ

Players found mounting the ramp disorientating.

Dev Note - fixed by implementing a camera dolly system that lerps instead of being fixed. Changeset 3021337.

Such little effort, much power.

Need I say more? If you’re not getting this info into your Test Suites, Cases and Script, you’re just throwing knowledge down the drain.

Wrapping Up

OK, damn, 7 pages of explaining my Test Cases. I hope you found something useful!

One more post next week to bring it all together. Maybe in interactive page??? Who knows?

Take it easy!

Blog Roll | About | Table Of Contents