Great Products Consulting

Category: Test

  • What are acceptance criteria?

    What are acceptance criteria?

    Acceptance criteria define what the system needs to be able to do in order to be considered ‘complete’ from a software development point of view. In essence, they’re a set of tests that you conduct to confirm that your platform or app does what you want it to.

    In this article, I’ll look at what they look like and how you use them.

    What are acceptance criteria?

    For each requirement that you have for your platform or app, you should also have one or more acceptance criteria. These criteria describe how your system is expected to work – both in terms of a “normal” scenario, but also when a user does something that they shouldn’t have.

    A commonly used structure for framing your acceptance criteria is as follows:

    <b>Given</b> [a particular scenario]

    <b>When</b> [the following steps are completed]

    <b>Then</b> [the expected outcome happens]

    This handy approach ensures that you fill in each part for a complete statement of acceptance.

    Examples of acceptance criteria

    Let’s say you’re building a marketplace to connect freelancers with potential customers. For the search function, you might have the following acceptance criteria:

    Given I am looking for a developer in Australia,
    When I enter “developer” in the keyword search field, select “Australia” from the drop down field, and click the search button,
    Then the search results screen displays with a list of developers from Australia, their rating, and their indicative hourly rate; with the results sorted by rating.

    To go even further, your acceptance criteria may also include the following:

    Given that I have searched for Australian developers,
    When I click on the header of the hourly rate column,
    Then the search results are sorted by hourly rate from lowest to highest.

    You’ll want to write acceptance criteria for each feature. This will ensure that you’ve covered all parts of the process. However, you may not cover every single scenario that occurs – simply because your acceptance criteria would go on forever!

    How do you use them?

    So now that you know what acceptance criteria are, how should you use them? Most obviously, you’re using acceptance criteria to test what your developers have done. You’re making sure that your platform or app works the way that you expect.

    By having your acceptance criteria written out, you now have a systematic way to test – and retest your platform or app. This also allows you and your developers to know what you did if something goes wrong. If you just went into your system, started playing around, it would be very difficult to recreate the problem.  With written criteria, you can repeat the test to see if the same thing goes wrong again.

    Thinking about your acceptance criteria is also a useful tool to help you define what you want your product to do. If you start thinking about what your acceptance criteria might be when you write your requirements, it actually helps your mind work through the things that the product needs to do. This will give you a more complete set of requirements.

    Acceptance criteria are an important part of the software and product development process. Along with giving you some help in writing your requirements and a systematic approach to testing, acceptance criteria also give you and your developers a benchmark for what you expect your system to be able to do. So, take the time to write out your acceptance criteria as they’re a critical component for building your platform or app.

    Looking for a developer?
    Get my 10 essential tips for hiring the right developer for your project.
    Download it now for free!
  • How do I make sure my platform or app works?

    How do I make sure my platform or app works?

    Testing is the step in the development process for making that your online web platform or mobile app works before you launch it. There are different types of testing to look at different aspects of your product, and then there is a process to actually plan and execute the testing. 

    In this week’s article, I’ll look at 5 steps for planning and executing your testing. These 5 steps primarily apply to application (system) testing, integration testing, user acceptance testing and beta testing. Performance testing is a different beast, as it requires some technical tools to complete.

    The 5 steps to make sure your platform or app works properly are:

    1. Creating a test strategy
    2. Writing test scripts
    3. Preparing for testing
    4. Executing the testing
    5. Reviewing the results
    1. Creating a test strategy

    For each type of testing, you want to start by creating a test strategy. This outlines all of the pieces required to do the testing including:

    • What you’re going test
    • When you’re going to do the testing
    • What your goals are
    • How you’re going to do the testing
    • Where you’re going to test
    • What you’ll test each day of the testing
    • Who needs to be involved
    • How will problems be resolved
    • What the entry/exit criteria are (e.g. what must be completed before the testing can start/end)
    • How you’ll track the progress of the testing

    This can be a list of bullet points or it can be a detailed paragraphs. The point is that you think about these things before you start your testing.

    Your developer may already have tools and processes in place for some of these things, but it’ll be up to you to find out what these are and to see if they are adequate for your testing.

    There’s also such as thing as “over-testing” – where you test until things are perfect. While it seems like the ideal thing to do, one thing you need to know is that most software is launched with known bugs and issues – they’re just not serious ones. As you create your test strategy, factor this into your entry and exit criteria so that you don’t end up testing forever and never launching!

    2. Writing test scripts

    Once you’ve created your strategy, you need to write some test scripts (or “acceptance criteria”) for what you’re going to test. These scripts describe what you’re testing, step-by-step instructions for completing an activity and what you expect the outcome to be. You need to create scripts for all of the functionality that you describe in your original product definition. This includes all of the “normal” scenarios and any “exception” scenarios.

    Doing this is going to be a big task, but it makes things a lot easier during the testing as you know exactly what you need to do and what you’re expecting your platform or app to do.       

    3. Preparing for testing

    This step is about looking at your test strategy and making sure all of the pieces are in place – have you got the resources that you need? Has the issue management process been set up? Is the testing site ready? Have the entry criteria been met?  You want to hit the ground running when the testing starts.  You don’t want to waste time and delay the testing because something isn’t ready.

    Preparing for testing is extremely important for beta testing if you’re coordinating a lot of people.  You need to make sure that they have the right setup and proper instructions, so they can perform the testing.  It doesn’t inspire confidence if people have to wait around for you or if your testers aren’t given the right direction! 

    4. Executing the testing

    Execution is about completing your test scripts according to the schedule that you’ve defined. You follow the steps in your scripts and if something doesn’t work, you notify your developer to fix it. You should document your findings – including screen captures – so that if something does go wrong, you can go back and look at what you did.

    During the testing, you also need to track and monitor your progress to make sure you meet your timelines, and to make sure your platform or app is working correctly.

    5. Review the results

    This last step is an important one.  You want to make sure you’re happy with the outcome of your testing. As you track and monitor your testing, you get a feel for how well your platform or app is working. You look at the number of issues that you encounter, and see how many test scripts that you’ve been able to complete successfully. If there are lots of issues, then you may want to pause the testing until some of the issues have been fixed.  You should keep testing until your exit criteria have been met.

    Why do I need to do all of this?

    As discussed last week, you want to make sure that your product works! You want to ensure the best experience for users, and broken functions don’t inspire confidence or positive references. 

    Another reason for going through this process is you need to be clear about what you want to test, so you can be happy your platform or app works properly! You also need to align with your development team on what you expect so that they can meet those expectations.

    Finally, in order to be effective, testing needs to be done in a way that is specific and repeatable. If you do something and it doesn’t work, the first thing that the developer is going to want to do is recreate the problem. If you can’t tell them what you did, then they can’t figure out what went wrong. Also, when the problem is fixed, you need to re-test the functionality in the same way to make sure that the fix works.

    Testing – despite it’s tediousness – is actually quite rewarding! You get to see your finished product in action, which is always exciting. You also assure yourself that your product works the way that you want it to.

    Stuck on how to test your app? Sign up for a free 30-minute discovery session* to get you on the right track.

    * When you sign up for a discovery session, we’ll also add you to our mailing list so you can receive emails from us (with your consent). These emails will include news and updates, occasional offers and promotions, and exclusive content and resources to help you on your development journey. You may also receive follow-up emails in relation to the discovery session. We will not spam you. If you don’t wish to hear from us anymore, simply select the unsubscribe option at the bottom of any email that we send to you. To view our privacy policy, click here.

  • What are the different types of testing?

    What are the different types of testing?

    Did you know that there are different types of testing that should be done on your new online web platform or mobile app? Each type of testing has an important role to play in making sure that your end product works the way that you expect it to. In this article, I’ll describe the different types of testing so that you know what you have to do for your platform or app.

    Imagine that your launch your product and your first users can’t sign in. Imagine you have a massive growth spurt and your product can’t support the number of users trying to access the platform. In the life of a new product, this is bad news! 

    There is a good bet that your development team will not test your product in a way that will ensure that it works the way that you want it to – probably because you haven’t paid them to. Testing is a specialist activity so your developers are not likely to have the skills to do parts of it properly (their specialty is coding!) This means it’s up to you to drive the testing phase of your project.

    So, before you launch your product to the world, what types of testing do you need to do?

    At a high-level, there are 5 different types of testing that you’ll want to look at:

    • Application (or System) testing is about making sure everything within the platform/app works correctly from a features and functions point of view.
    • Integration testing is for making sure that the platform/app talks with all other applications correctly (think payment gateways, email automation, CRMs, etc.)
    • Performance testing is used to check the speed and responsiveness of the platform/app under different conditions.
    • User acceptance testing allows the business (e.g. you) to check that the platform/app works in the way that it’s expected to in view of all of the surrounding operational processes.
    • Beta testing puts your platform/app in the hands of “real” users so they can put it through the rigours of everyday use.

    Traditionally, in large corporates, the technology department will do the system, integration and performance testing. In your case, you’ll have to own this and you’ll probably combine system, integration and user acceptance testing all into one. The developers should be able to run performance testing (as it needs specialised tools to do this), but you’ll have to tell them what you expect should happen.

    There is one more type of testing that you’ll have to do. This is called “regression testing”. Regression testing is about making sure that any changes to a platform or app don’t affect what’s already there. So, if your developer fixes something, you’ll want to test that what was fixed now works, but you’ll also want to re-test some of the functionality that is related to that feature to make sure that the fix hasn’t broken something else.

    Testing under a waterfall methodology

    If you’re developing your platform or app using a waterfall methodology, then testing occurs after the “build” phase. You’ll probably do a combined system, integration and user acceptance test at that time for all of the functionality that has been built. In the meantime, your development team should do some performance testing to make sure your platform or app can deal with the volumes of users that you expect. Once all of that’s completed, you’ll want to do some beta testing as well.

    Testing under an agile methodology

    If you’re using an agile methodology to build your platform or app, then you’ll be doing some testing along the way. As each component is built, you’ll do some system testing and maybe some integration and user acceptance testing to make sure that each piece works. What you test will depend on the component being built and what has been built before it. For example, if you want a user to be able to create an account and then you want to send those details to an email automation tool, you can only test the “integration” after both of those pieces have been built.

    You’ll only be able to test your operational processes as part of your user acceptance testing (e.g. sending a welcome email) after you’ve added that to your email automation too. What this means is that you should do some integration and user acceptance testing after all of the coding has been completed. This will ensure that everything works together. Finally, look at doing some beta testing, so that you can get real people to put your product through its paces.

    In summary…

    You need to test your product to make sure that it works! There is nothing worse that launching a product only to have key features and functionality that don’t work properly.  

    There are different types of testing to do and each has a different purpose. Testing is a lot of work and the amount of effort should not be overlooked as you build your product. It’s such an important step to ensuring a successful launch. You’ll need to decide what types of testing that you want to do, so make sure you factor it into your timelines.

    Next week, we’ll look at how you actually do the testing.

    Want to learn more about how to test your platform or app? Sign up for a free 30-minute discovery session* to make sure your users can use your app!

    * When you sign up for a discovery session, we’ll also add you to our mailing list so you can receive emails from us (with your consent). These emails will include news and updates, occasional offers and promotions, and exclusive content and resources to help you on your development journey. You may also receive follow-up emails in relation to the discovery session. We will not spam you. If you don’t wish to hear from us anymore, simply select the unsubscribe option at the bottom of any email that we send to you. To view our privacy policy, click here.

  • Intro to Product and Software Development and Product Management (part 3 – build your app)

    Intro to Product and Software Development and Product Management (part 3 – build your app)

    I hope you’re starting to get a feel for what’s required to build an app. I’m now onto part 3 of my series on introducing concepts in product management, product development and software development. In part 1, I provided a general introduction to these concepts and in part 2, I looked at what happens at the beginning. In this post, I’ll cover what it takes to bring your product to life by focusing on the things that are involved in actually building your app.

    Product Development

    In product development, building your app involves “Develop” and “Test” activities:

    “Develop” involves designing and building all of the components required to deliver and run your product. This is the fun stuff! Here, you take all of your requirements (what you want), you find a solution for them (how it works) and then you build it. This should not only include the technical bits for your product (see below under software development), but all of the manual processes required to support your product (e.g. customer servicing, accounting, etc) and the “administrative” things required to run a business (e.g. logos and branding, bank accounts, business registrations, terms and conditions, privacy policy, etc). These non-technical pieces are just as important as building your app – so please don’t forget about them! The develop phase includes planning your testing activities as well.

    “Develop” activities also includes thinking about how you want to launch your product and to whom.  This will allow you to prepare things like content and advertising ahead of time. You should come up with a plan for your launch; including your launch activities and the tasks required to execute your launch. You may also want to start some prelaunch activities to get the attention of early customers.  Launch activities should not be limited to marketing and PR.  They should include all of the bits and pieces needed to get your business up and running as well (e.g. starting insurance, etc).    

    After your product and operational processes have been built, you need to make sure everything works the way that you want it to and the way it was designed to. This is what “Testing” is all about. You not only have to test “normal” scenarios but also what might happen if things go wrong. This is an important step because you don’t want your new customers to have a bad experience when using your new product! Creating and developing a proper testing process will help you to ensure that nothing gets overlooked and forgotten.

    Software Development

    In software development, building your product involves three activities: “Design”, “Build” and “Test”:

    First though, you’re going to have to find a developer. This involves deciding how they’ll fit into your team (e.g. employee, freelancer, co-founder, etc), finding the right people and getting some estimates (or negotiating a salary or equity). You may also choose to hire a UX (user experience) designer to create the screens for your app.

    The “Design” phase is about coming up with the technical and functional solution for your product. The final look and feel and layout of the screens will be determined. Solutions will also be defined for the processes and functions in your app. Some other technical things will probably also be done to prepare for your app to be built – like finding a hosting site, setting up a code repository, creating developer accounts for mobile apps, etc. 

    It’s important for you to make sure you review and approve your solution. You want to see what you app will look like and understand how it will work.  It’s easier to adjust things at the design stage rather than waiting until after your app has been built!     

    After your app has been designed, you get to the actual “Building” (finally!) Here, the developer codes your app and performs some testing on the code components. You’ll probably be required to provide the content that your app requires and if you’re building a mobile app, you may want a separate website as a marketing tool. It’s also during this phase that you’ll take your solution and determine how to test it.  

    In large corporations, the “Test” phase is broken down into different types of testing. The technology team covers “systems”, “integration” and “performance” testing, while the business team will organise “user acceptance” and “beta” testing. A summary of each type of testing is below:

    • System testing is about making sure everything within the app works correctly from a features and functions point of view.
    • Integration testing is for making sure that the app talks with all other applications (think payment gateways, email automation, CRMs, etc) correctly.
    • Performance testing is used to check the speed and responsiveness of the app under different conditions.
    • User acceptance testing allows the business (e.g. you) to check that the app works in the way that they expected it to.
    • Beta testing puts your app in the hands of “real” users so they can put it through the rigours of everyday use.

    Unless you hire a full-service agency, it’s a good bet that you’ll end up doing all of the testing for your app – except for performance testing. Testing involves a different skillset from coding, so you’ll find that many developers won’t be able to do this properly.

    As the business owner, you’ll need to keep an eye on your development team during these activities.  You’re the “project manager” for your project which means you’re responsible for making sure your product gets delivered! Your development agency may provide a project manager – but that will only be for the technical components. You’ll still have to oversee all of the activities required to build your app – even managing your project manager!

    “Design”, “Build” and “Test” activities can occur simultaneously as your product is broken down into pieces and each piece is completed before going onto the next piece. This collaborative approach often allows you to develop product components a lot faster as team members are working on something together which reduces the amount of back-and-forth time and decisions are made more quickly. However, it does require some experience with the software development process to work this way and you’ll have to be a lot more involved in the day-to-day project activities.

    As you can imagine, there’s lots do here – not just on the technical side, but also on the business side.  It’s easy to feel overwhelmed.  Knowledge is power though – and if you know what to expect, you’re one step ahead!    

    Product Management

    You still haven’t launched yet, so there’s not much managing that you need to do. As mentioned previously, you’ll have to put together a launch plan so that you can get those early customers on board.

    …. and you’re ready for lift-off (almost!)

    So there your have it – you’ve built and tested your app. You should be comfortable with the way the app is working and you have completed all of the preparation activities for your launch.

    In the final article of this series, I’ll look at the last phases in software and product development – the implementation and launch of your app. I’ll then explore what happens next.

    [last updated: April 20, 2018]