TAOGD Lenses: Lens #3 “The Lens of Fun”

This post is a continuation of the series examining the lenses of game design described by Jesse Schell in his book The Art of Game Design. Previous posts in this series can be found here.

Fun is desirable in nearly every game, though sometimes fun defies analysis. To maximize your game’s fun, ask yourself these questions:

  • What parts of my game are fun?
  • What parts need to be more fun?

Fun isn’t normally one of the characteristics we consider as testers for most software apart from games. Still, there’s at least 3 ways we should be looking at the concept of fun as we’re testing:

  1. Application is fun when intended to be

    The most obvious one is when we’re testing software that’s designed to be fun. If we’re testing a game or recreational app, we should certainly be raising issues if that game loses its shine. I find this to be particularly true in the mobile space these days (though that may be because that’s where I play most software-based games). There’s been a large number of games that I downloaded, played once and abandoned because I quickly lost interest in them. The game just wasn’t enough fun to hold my interest. Maybe some portion of the challenge required more effort than I was willing to invest and I got frustrated. Maybe it wasn’t challenging enough. Maybe there was a lot of repetition or actions spent just getting from place to place. Getting the right balance is a critical point for developing a game – if the game’s target audience doesn’t get engaged in the game, the time spent developing it may end up being a waste. As testers, it’s not uncommon to get tired of or even frustrated with the app we’re testing. We see it constantly for extended periods of time, warts and all. If we find ourselves getting bored with a game we’re testing, we need to determine whether it’s overexposure, it’s a case of us not being a member of the target audience or if there’s an actual issue that might cause many of our users to quickly abandon the game. If the latter, that falls into the scope of information we should be raising to the team for discussion.
  2. Application is not anti-fun

    This is getting into the realm of usability testing. While an application may not be designed to be fun, we generally don’t want it to be a frustrating experience either. Non-intuitive UIs, overly complicated actions, or a lack of flow through the task can all leave a user muttering under their breath and potentially even lead them to stopping using the system we’ve built. Rather than using a lens of looking where the fun should be, we need to look at the overall system through the lens of where the fun ISN’T. This requires going beyond the scope of the individual feature we’re investigating and looking at how it interacts with the other parts of the system. It requires an understanding of the problems our users want to solve with our software and the contexts they want to solve them in.
  3. Testing itself should be fun

    The last place where fun should come in in our analysis is in an examination of our testing process itself. Tasks that aren’t enjoyable should be periodically reviewed. Is the task even still providing value? (If not, perhaps the solution is to drop the task.) Is there another way that we could approach the task – automation (fully or in part), restructuring, adding a gamification element or something else that could make it more enjoyable? Is it a task that could be performed by someone else – perhaps the current task owner is just burned out on the task, and a new perspective might revitalize the task? With the constant schedule pressure many testers face these days, it can be hard to find the time to step back and evaluate the work we’re doing. We get into ruts, repeating things we’ve done before simply because that’s what we did before, and never notice that we’re no longer efficiently providing value. Our sense of dread for a task or boredom during the task can be an early warning sign that it might be time to take a step back and re-evaluate.
  4. What other ways do you see the lens of fun applying to testing?

    Facebook Twitter Linkedin Digg Reddit Stumbleupon Tumblr Posterous Email
Posted in Testing | Tagged , , | 1 Comment

I believe…

We’ll resume the Art of Game Design series shortly (with a far smaller gap than between lenses #1 & 2), but first an interlude.

I mentioned in the last post that I had recently done some training for a client. Part of that training sprung from a conversation I had with the client’s QA Director about my testing philosophy. We were doing an assessment and planning the training, and I knew elements of my philosophy were going to come out in both and I wanted to talk to him about those elements in a less formal context. The end result was him stating that he wished his team could hear what we talked about, and so the training included a handout that spelled out the elements I’ve so far identified. I wanted to capture that as a stake in the ground, so here it is. This will continue to evolve – I don’t think I’ve captured everything yet, and I”m not sure I’ve got the wording where I want it yet, but I’m pretty happy with what’s here all in all.

  • Quality is subjective by person. What one person values might be another person’s bug
  • Quality is the whole team’s responsibility
  • Software needs to solve the customer’s problems or it is useless
  • Testing is a challenging intellectual process
  • Thus, any testing activity that discourages thinking or questioning while performing it is potentially harmful. Test execution off of pre-written scripts is often an example of this.
  • There are no best practices, just good practices for a particular context. Blindly applying a practice to a situation because it worked somewhere else can more harm than help. Understanding the context any practice will be used in is crucial
  • Testing is all about providing information to the team and stakeholders. If testing is failing to provide the information that is needed, it’s wasting of time and resources
  • To provide that information, test reporting needs to clearly tell 3 stories: the story of the product quality, the story of the testing done and not done, and the story of the testing quality (why what was done was or wasn’t sufficient)
  • To be successful, testers need to approach an app trying to prove that it has bugs rather than proving the app is correct. Otherwise, we may fall victim to confirmation bias and miss issues
  • Test cases are only ever as good as their oracles
  • Testers are not the gatekeepers of the project. They should have a voice, but not the sole voice, in release decisions
  • The value of any action has 3 components: The benefit gained by completing the action, the costs incurred by performing the action, and the hidden costs of not getting the benefit from performing other actions which can no longer be performed. Choosing the right action at any point in time means striking a balance among these 3 elements
  • Writing out fully detailed, explicitly defined test cases is rarely the best use of time. Variation in test execution is actually a good thing. Letting the tester executing make decisions that don’t impact the test’s goal leads to better testing
  • Test cases do, however, need to be explicit about the point of the test case and what it is trying to test
  • Writing test cases at the start of a cycle of testing effort means we’re creating those tests at the point where we have the least knowledge about what we’re testing that we’ll have during the cycle
  • Communication is essential
  • Variety in tests is critical
  • Conventional software testing metrics are often misleading and can lead to undesired behavior. However, they’re what we have to work with at the moment, so until something better comes along, we need to use them to approximate the information we need. We just need to ensure we use them wisely and be conscious of the impacts and risks in doing so
  • Some things can’t be automated. Some things can, but shouldn’t be. 100% test automation is not a desirable goal in most contexts
  • Automating “stabilized” existing tests as regression tests is probably not the best use of automation for a project
Facebook Twitter Linkedin Digg Reddit Stumbleupon Tumblr Posterous Email
Posted in Automated Testing, Testing | Tagged | 1 Comment

TAOGD Lenses: Lens #2 “The Lens of Surprise”

This post is a continuation of the series examining the lenses of game design described by Jesse Schell in his book The Art of Game Design. Previous posts in this series can be found here.

Surprise is so basic that we can easily forget about it. Use this lens to remind yourself to fill your game with interesting surprises. Ask yourself these questions:

  • What will surprise players when they play my game?
  • Does the story in my game have surprises? Do the game rules? Does the artwork? The technology?
  • Do the rules give players ways to surprise each other?
  • Do your rules give players ways to surprise themselves?

Surprise is a double-edged sword when applied to the broader world of software testing. Users can be surprised in a positive way (as in “Oh wow, this software anticipated a need and has already met it!”) or in a negative way (as in “This software can’t do that task I want to do? What a piece of junk!”). As testers, we provide information to our teams and stakeholders about the quality of the system. We do this by looking at the system through various lenses just as Jesse Schell is advocating for game design. Recently, I’ve done some training for a client where we spent a good chunk of the day talking about the use of lenses. We used the Test Heuristics Cheat Sheet that Elisabeth Hendrickson, James Lyndsay, and Dale Emery created and focused on the heuristic section in particular. Each heuristic is another lens that we can look at our software through – asking ourselves things like what does the Goldilocks Heuristic (Too big, too small, just right) mean in the context of the functionality I’m testing, or applying the Count heuristic (None, One, Many). From there we moved on to Rikard Edgren’s Software Quality Characteristics poster and talked about non-functional lenses. This principle of surprise is yet another lens – what’s going to surprise my users when they use this feature? Will it be a pleasant surprise for them or a negative surprise?

For me, a good example of surprise comes from my days working as a tester on the Microsoft Expression Blend product. I was working with the graphics importer team and my job was the testing of the foundational code that read Photoshop files, parsed out all the data about the layers and graphical effects and made all that data available to a converter that made the images available in Blend. We had a lot of Photoshop files in our test suite, including some that were corrupted. Photoshop wouldn’t open these corrupted files, but we took the approach of salvaging as much of the file as we could. We couldn’t always get everything (some of our corruptions were due to truncating the file from which there was no way to recover data) but we’d show what we could get. We never publicized this feature, nor do I know of any user ever trying it. None the less, it’s easy to imagine a user feeling a degree of panic when their file doesn’t open in Photoshop and trying Blend in a last hope way to salvage their work and being pleasantly surprised to find they could recover at least something. From the time of working on that feature, one of the lens I carry with me in my testing is to look at the feature from the perspective of my users and see if I can identify any potential surprises. The negative ones then lead to additional test cases and/or bug reports to reduce their impact.

What surprises your users about your application?

Facebook Twitter Linkedin Digg Reddit Stumbleupon Tumblr Posterous Email
Posted in Testing | Tagged , | 1 Comment

TAOGD Lenses: Lens #1 “The Lens of Essential Experience”

In the my earlier post, I talked about Jesse Schell‘s book, “The Art of Game Design”, promising to go lens by lens through the book. This is the first post looking at a lens. More to come!

Jesse Schell’s Lens #1 in The Art of Game Design is “The Lens of Essential Experience”. This lens is about the experience of the player playing the game.

The Lens of Essential Experience
To use this lens, stop thinking about your game, and start thinking about the experience of the player. Ask yourself these questions:

  • What experience do I want the player to have?
  • What is essential to the experience?
  • How can my game capture that essence?

— From The Art of Game Design: A Deck of Lenses

From a game standpoint, the experience of playing is the main reason for the existence of most games. Games aren’t generally trying to solve a problem – they exist because the players want to have fun. In the broader world of software systems, there is generally a problem the system we’re testing is designed to solve. That problem might be making some process (like tracking finances or managing orders) more efficient or it might be any number of other things for a particular user base. Even when the goal of the software is something other than the experience of using it, however, the experience a user has makes a big difference on how they perceive the application. We’ve all likely had the experience of using software with a poor interface, and it’s unlikely that we see that system of being of high quality, particularly when compared to systems that are easy to use.

This lens invites us to look at our system from the perspective of that end-user. It moves us away from determining whether each feature works the way it should in isolation and focuses us on the solution that the software is providing and how the user will interact with our system. Think about the tasks that a user will be trying to carry out. Will the software make it easy for them to achieve their goals?

One aspect of this is the straightforwardness of the action. Extraneous clicks and dialogs or hidden interface elements can lead to frustrated users. For example, at the beginning of 2013, my company gave each employee a Surface RT tablet. This was the first time I’d encountered the Windows 8/RT interface and I didn’t know that swiping in from the sides of the screen would bring up toolbars and options. I was initially frustrated that simple actions like Searching or even finding settings for the apps I was trying weren’t available. It was only after searching online for how to accomplish a task that I found how to access the charms. Until I learned that, my user experience was lacking.

On the other hand, making actions too easy to access can also cause problems. Some apps put the button to delete an item near other non-delete buttons and a misguided click (or tap on a mobile device) can result in the user losing work. I ran into this recently as well, when I was checking out at a grocery store. The checker misremembered the code for onions and entered the code for cucumbers instead. In trying to fix the issue, he ended up voiding the entire transaction (seemingly without confirmation that he wanted to do that) and had to rescan every item. Neither he nor I were fond of that user experience.

There aren’t rules to guide every experience. Testing the experience our software creates requires us to think about how a user will interact with our system, to put ourselves in their shoes. It involves recognizing the things the user will want to easily do and enabling those things while preventing (or restricting) the things that will get in the way of those tasks. It involves thinking about the emotions we want to invoke in our users and understanding the essentials of those emotions. We may want to delight our users when they find we anticipated their needs in unexpected ways or we may want to make a task as efficient as possible to let the user do their task quickly and get on with the rest of their work. Whatever our goal is for our experience, we can use the Lens of Essential Experience to focus our testing on determining whether we have met that goal.

Facebook Twitter Linkedin Digg Reddit Stumbleupon Tumblr Posterous Email
Posted in Testing | Tagged , | 1 Comment