uTester Shares Software Testing World Cup Experience, Offers Advice

20140613_174308_resized

Mark and his team during the Software Testing World Cup.

Marek (Mark) Langhans is a Gold-rated tester and former Forums Moderator in the uTest Community, and hails from Prague. Mark has tested information systems, web, mobile and desktop applications of domestic financial institutions for a couple of years now. In June, he participated in the Europe Preliminary round of the Software Testing World Cup (STWC), and shares his experience here, along with some advice for future STWC participants.

Before I go into any details about the competition, let me thank the entire team and all the judges and product owners behind the Software Testing World Cup (STWC).

If testing needed a push into the general public eye, this event was the right way to go about it. Not only it has given us testers a way to compete and connect with each other, and to see our limitations, strengths and weaknesses, but it has also put the testing profession into a whole new perspective. Testing has been made cool, and that is very rare to do.

Gameday

On Friday, the 13th of June, three of my colleagues and I participated in the Europe Preliminary round of the STWC. Even though we had a pretty awesome base in our firm HQ’s basement, at the end, we didn’t end up near the top. However, with the things we have taken from it and it has given to us, you just can’t put a price tag on that. In three hours, you learn so much about yourself and your testing capabilities than you may have in your whole testing career.

The competition started on time. Thirty minutes before the official start, we all had received emails introducing us to the software to test (Sales Tool — for more details, check out the YouTube stream), the scope, and, of course, some tips what we should focus on. The email contained a link to the application so we could move around with it before the actual competition. These thirty minutes flew by, as the application was something none of us had come across before, and so we tried to figure out what we actually could test and how.

The email also contained a link to a YouTube channel which, in real-time, streamed a Google hangout of the judges, customer and product owner. On the YouTube channel, in the comments area, we could ask questions regarding the scope, the application and everything about the competition. We could also use Twitter to ask these questions. My team and I were focused more on the application, as it was an unknown to us, so we didn’t pay much attention to the stream, even though we had a huge screen just for it in front of us. We listened to it with only one ear, but that probably was a mistake, in retrospect.

The application wasn’t that complex, so there weren’t that many features, but the ones we could cover weren’t that easy to test. We found a few issues, and everything we came across was reported in the Agile Manager, the official bug reporting tool. Before the competition, we prepared a bug report template so we all had the same information contained in the reports, but we weren’t consistent…another mistake, as this was taken into consideration in judging.

There was one huge issue with the reporting tool, as other teams reported that they could see other teams’ bug reports. This made the competition a little uneven, to be honest, but I do not think it made it any difference in the end.

After the 2 1/2 hours testing the application, we moved into creating the test report. We had prepared a template for it, so we just filled in our findings. For non-English speaking teams, this was the hardest part, I guess, to make it understandable without any major grammatical mistakes. We followed a few tips from Global Jury member Matt Heusser: Keep it simple on few pages with clearly visible sections, have a summary at the top, and below, go into some details about your findings, with more details about those findings and how you came to them.

We sent out the report five minutes after 12. We immediately received an auto-reply that the report was received, and after that, we packed our stuff and went home. Of course, our heads were filled with what we had done, we could have done better and what we could have done better next time.

Advice for the Prospective STWC participant

When I look back at the whole experience and compare it to uTest, for instance, I originally thought this would be very similar in many ways, but it wasn’t.

Having three hours to test something isn’t that bad — you get to used to it when testing applications here at uTest. But with three hours, you just report bugs and that is it. Maybe you’ll fill in a review whenever you have time. But for the competition, there is a responsibility to report back to the customer your findings and recommend having the application released or postponed. And to do all this in three hours is quite different from just testing and then moving on to another project.

This contest wasn’t all about how well you test and how many valuable bugs you log, but also about communication, both within your team, and also with other teams and the team behind STWC. This was mentioned many times by Matt, and teams that were visible publicly were given additional points which may have helped few of them to get a better position at the end.

As this contest was made for testers and our profession, we should have given something small back in return — at least promoting it on social media and so on. A colleague of ours Tweeted about our team a few times, but when I look back, it wasn’t enough. It wasn’t all about the points, but about being part of something greater and helping to achieve something.

If you decide to be part of a STWC team next year, take the time and sit down with your team before the competition, and set a few strategies. We did so only once, and within the time we only came up with bug and test report templates. We didn’t cover some sort of strategy on how to go about the application under test, or at least make a list what to test for both mobile and web applications. In just three hours, there isn’t much time to be creative and try to think of something, and when you have no guidelines, you may start panicking.

Three hours is a very short time, so don’t try to test everything, the entire application and all its features. It is more about making compromises and focusing on some key areas, each team member on something different, or just working in pairs in the same area.

From what I have taken from the competition, communication within our team helped us understand the SUT very quickly, but we tried to test every feature available, and thus ended up scratching only the surface and not going deep enough. When you compromise, you have at least something to write about in your test report, where you can detail what you covered and what you would cover if you had more time.

Comments

  1. damian synadinos says

    I thought that was a very nice summary of your experience. Thanks for sharing!

    I was particularly glad to see the thought and learning that occurred after the competition. In my opinion, the prize was not to money, tickets, or trophies, but learning and improvement. And by that standard, I think you ranked near the top.

    As a judge, I learned a lot, too! It was interesting to see how quickly my judging algorithms failed, and I ended up replying on heuristics instead.

  2. Lucas Dargis says

    Great post Marek!

    I’m glad you were able to represent uTest at the competition. Sounds like it was a fun learning experience for everyone involved. I’m very sorry I missed out. Maybe next year.

Trackbacks

Leave a Reply