Great Products Consulting

Tag: user acceptance testing (UAT)

  • What are acceptance criteria?

    What are acceptance criteria?

    Acceptance criteria define what the system needs to be able to do in order to be considered ‘complete’ from a software development point of view. In essence, they’re a set of tests that you conduct to confirm that your platform or app does what you want it to.

    In this article, I’ll look at what they look like and how you use them.

    What are acceptance criteria?

    For each requirement that you have for your platform or app, you should also have one or more acceptance criteria. These criteria describe how your system is expected to work – both in terms of a “normal” scenario, but also when a user does something that they shouldn’t have.

    A commonly used structure for framing your acceptance criteria is as follows:

    <b>Given</b> [a particular scenario]

    <b>When</b> [the following steps are completed]

    <b>Then</b> [the expected outcome happens]

    This handy approach ensures that you fill in each part for a complete statement of acceptance.

    Examples of acceptance criteria

    Let’s say you’re building a marketplace to connect freelancers with potential customers. For the search function, you might have the following acceptance criteria:

    Given I am looking for a developer in Australia,
    When I enter “developer” in the keyword search field, select “Australia” from the drop down field, and click the search button,
    Then the search results screen displays with a list of developers from Australia, their rating, and their indicative hourly rate; with the results sorted by rating.

    To go even further, your acceptance criteria may also include the following:

    Given that I have searched for Australian developers,
    When I click on the header of the hourly rate column,
    Then the search results are sorted by hourly rate from lowest to highest.

    You’ll want to write acceptance criteria for each feature. This will ensure that you’ve covered all parts of the process. However, you may not cover every single scenario that occurs – simply because your acceptance criteria would go on forever!

    How do you use them?

    So now that you know what acceptance criteria are, how should you use them? Most obviously, you’re using acceptance criteria to test what your developers have done. You’re making sure that your platform or app works the way that you expect.

    By having your acceptance criteria written out, you now have a systematic way to test – and retest your platform or app. This also allows you and your developers to know what you did if something goes wrong. If you just went into your system, started playing around, it would be very difficult to recreate the problem.  With written criteria, you can repeat the test to see if the same thing goes wrong again.

    Thinking about your acceptance criteria is also a useful tool to help you define what you want your product to do. If you start thinking about what your acceptance criteria might be when you write your requirements, it actually helps your mind work through the things that the product needs to do. This will give you a more complete set of requirements.

    Acceptance criteria are an important part of the software and product development process. Along with giving you some help in writing your requirements and a systematic approach to testing, acceptance criteria also give you and your developers a benchmark for what you expect your system to be able to do. So, take the time to write out your acceptance criteria as they’re a critical component for building your platform or app.

    Looking for a developer?
    Get my 10 essential tips for hiring the right developer for your project.
    Download it now for free!
  • How do I make sure my platform or app works?

    How do I make sure my platform or app works?

    Testing is the step in the development process for making that your online web platform or mobile app works before you launch it. There are different types of testing to look at different aspects of your product, and then there is a process to actually plan and execute the testing. 

    In this week’s article, I’ll look at 5 steps for planning and executing your testing. These 5 steps primarily apply to application (system) testing, integration testing, user acceptance testing and beta testing. Performance testing is a different beast, as it requires some technical tools to complete.

    The 5 steps to make sure your platform or app works properly are:

    1. Creating a test strategy
    2. Writing test scripts
    3. Preparing for testing
    4. Executing the testing
    5. Reviewing the results
    1. Creating a test strategy

    For each type of testing, you want to start by creating a test strategy. This outlines all of the pieces required to do the testing including:

    • What you’re going test
    • When you’re going to do the testing
    • What your goals are
    • How you’re going to do the testing
    • Where you’re going to test
    • What you’ll test each day of the testing
    • Who needs to be involved
    • How will problems be resolved
    • What the entry/exit criteria are (e.g. what must be completed before the testing can start/end)
    • How you’ll track the progress of the testing

    This can be a list of bullet points or it can be a detailed paragraphs. The point is that you think about these things before you start your testing.

    Your developer may already have tools and processes in place for some of these things, but it’ll be up to you to find out what these are and to see if they are adequate for your testing.

    There’s also such as thing as “over-testing” – where you test until things are perfect. While it seems like the ideal thing to do, one thing you need to know is that most software is launched with known bugs and issues – they’re just not serious ones. As you create your test strategy, factor this into your entry and exit criteria so that you don’t end up testing forever and never launching!

    2. Writing test scripts

    Once you’ve created your strategy, you need to write some test scripts (or “acceptance criteria”) for what you’re going to test. These scripts describe what you’re testing, step-by-step instructions for completing an activity and what you expect the outcome to be. You need to create scripts for all of the functionality that you describe in your original product definition. This includes all of the “normal” scenarios and any “exception” scenarios.

    Doing this is going to be a big task, but it makes things a lot easier during the testing as you know exactly what you need to do and what you’re expecting your platform or app to do.       

    3. Preparing for testing

    This step is about looking at your test strategy and making sure all of the pieces are in place – have you got the resources that you need? Has the issue management process been set up? Is the testing site ready? Have the entry criteria been met?  You want to hit the ground running when the testing starts.  You don’t want to waste time and delay the testing because something isn’t ready.

    Preparing for testing is extremely important for beta testing if you’re coordinating a lot of people.  You need to make sure that they have the right setup and proper instructions, so they can perform the testing.  It doesn’t inspire confidence if people have to wait around for you or if your testers aren’t given the right direction! 

    4. Executing the testing

    Execution is about completing your test scripts according to the schedule that you’ve defined. You follow the steps in your scripts and if something doesn’t work, you notify your developer to fix it. You should document your findings – including screen captures – so that if something does go wrong, you can go back and look at what you did.

    During the testing, you also need to track and monitor your progress to make sure you meet your timelines, and to make sure your platform or app is working correctly.

    5. Review the results

    This last step is an important one.  You want to make sure you’re happy with the outcome of your testing. As you track and monitor your testing, you get a feel for how well your platform or app is working. You look at the number of issues that you encounter, and see how many test scripts that you’ve been able to complete successfully. If there are lots of issues, then you may want to pause the testing until some of the issues have been fixed.  You should keep testing until your exit criteria have been met.

    Why do I need to do all of this?

    As discussed last week, you want to make sure that your product works! You want to ensure the best experience for users, and broken functions don’t inspire confidence or positive references. 

    Another reason for going through this process is you need to be clear about what you want to test, so you can be happy your platform or app works properly! You also need to align with your development team on what you expect so that they can meet those expectations.

    Finally, in order to be effective, testing needs to be done in a way that is specific and repeatable. If you do something and it doesn’t work, the first thing that the developer is going to want to do is recreate the problem. If you can’t tell them what you did, then they can’t figure out what went wrong. Also, when the problem is fixed, you need to re-test the functionality in the same way to make sure that the fix works.

    Testing – despite it’s tediousness – is actually quite rewarding! You get to see your finished product in action, which is always exciting. You also assure yourself that your product works the way that you want it to.

    Stuck on how to test your app? Sign up for a free 30-minute discovery session* to get you on the right track.

    * When you sign up for a discovery session, we’ll also add you to our mailing list so you can receive emails from us (with your consent). These emails will include news and updates, occasional offers and promotions, and exclusive content and resources to help you on your development journey. You may also receive follow-up emails in relation to the discovery session. We will not spam you. If you don’t wish to hear from us anymore, simply select the unsubscribe option at the bottom of any email that we send to you. To view our privacy policy, click here.

  • What are the different types of testing?

    What are the different types of testing?

    Did you know that there are different types of testing that should be done on your new online web platform or mobile app? Each type of testing has an important role to play in making sure that your end product works the way that you expect it to. In this article, I’ll describe the different types of testing so that you know what you have to do for your platform or app.

    Imagine that your launch your product and your first users can’t sign in. Imagine you have a massive growth spurt and your product can’t support the number of users trying to access the platform. In the life of a new product, this is bad news! 

    There is a good bet that your development team will not test your product in a way that will ensure that it works the way that you want it to – probably because you haven’t paid them to. Testing is a specialist activity so your developers are not likely to have the skills to do parts of it properly (their specialty is coding!) This means it’s up to you to drive the testing phase of your project.

    So, before you launch your product to the world, what types of testing do you need to do?

    At a high-level, there are 5 different types of testing that you’ll want to look at:

    • Application (or System) testing is about making sure everything within the platform/app works correctly from a features and functions point of view.
    • Integration testing is for making sure that the platform/app talks with all other applications correctly (think payment gateways, email automation, CRMs, etc.)
    • Performance testing is used to check the speed and responsiveness of the platform/app under different conditions.
    • User acceptance testing allows the business (e.g. you) to check that the platform/app works in the way that it’s expected to in view of all of the surrounding operational processes.
    • Beta testing puts your platform/app in the hands of “real” users so they can put it through the rigours of everyday use.

    Traditionally, in large corporates, the technology department will do the system, integration and performance testing. In your case, you’ll have to own this and you’ll probably combine system, integration and user acceptance testing all into one. The developers should be able to run performance testing (as it needs specialised tools to do this), but you’ll have to tell them what you expect should happen.

    There is one more type of testing that you’ll have to do. This is called “regression testing”. Regression testing is about making sure that any changes to a platform or app don’t affect what’s already there. So, if your developer fixes something, you’ll want to test that what was fixed now works, but you’ll also want to re-test some of the functionality that is related to that feature to make sure that the fix hasn’t broken something else.

    Testing under a waterfall methodology

    If you’re developing your platform or app using a waterfall methodology, then testing occurs after the “build” phase. You’ll probably do a combined system, integration and user acceptance test at that time for all of the functionality that has been built. In the meantime, your development team should do some performance testing to make sure your platform or app can deal with the volumes of users that you expect. Once all of that’s completed, you’ll want to do some beta testing as well.

    Testing under an agile methodology

    If you’re using an agile methodology to build your platform or app, then you’ll be doing some testing along the way. As each component is built, you’ll do some system testing and maybe some integration and user acceptance testing to make sure that each piece works. What you test will depend on the component being built and what has been built before it. For example, if you want a user to be able to create an account and then you want to send those details to an email automation tool, you can only test the “integration” after both of those pieces have been built.

    You’ll only be able to test your operational processes as part of your user acceptance testing (e.g. sending a welcome email) after you’ve added that to your email automation too. What this means is that you should do some integration and user acceptance testing after all of the coding has been completed. This will ensure that everything works together. Finally, look at doing some beta testing, so that you can get real people to put your product through its paces.

    In summary…

    You need to test your product to make sure that it works! There is nothing worse that launching a product only to have key features and functionality that don’t work properly.  

    There are different types of testing to do and each has a different purpose. Testing is a lot of work and the amount of effort should not be overlooked as you build your product. It’s such an important step to ensuring a successful launch. You’ll need to decide what types of testing that you want to do, so make sure you factor it into your timelines.

    Next week, we’ll look at how you actually do the testing.

    Want to learn more about how to test your platform or app? Sign up for a free 30-minute discovery session* to make sure your users can use your app!

    * When you sign up for a discovery session, we’ll also add you to our mailing list so you can receive emails from us (with your consent). These emails will include news and updates, occasional offers and promotions, and exclusive content and resources to help you on your development journey. You may also receive follow-up emails in relation to the discovery session. We will not spam you. If you don’t wish to hear from us anymore, simply select the unsubscribe option at the bottom of any email that we send to you. To view our privacy policy, click here.