Lab 7 - Functional and Non-Functional Testing

Introduction

You are already familiar with the requirements gathering and you also produced several use cases and a project plan. During this lab we will learn how our developed POS software system can be checked against its functional and non-functional requirements (User Stories). In Lab 6, you have already created a plan for the functional (system) testing. In this lab, you shall create a plan for testing some of the non-functional requirements and, in addition, conduct the testing according to both plans. The test outcomes shall be reported.

Homework

Task 1 (3p)

Describe how you are planning to test non-functional requirements for usability. You need to pick 2 non-functional user stories (cf. Lab 1) related to usability in total. You need to offer concrete steps, methods and techniques to test each of the selected non-functional user stories. Make clear how you are going to ensure/guarantee the specified usability (i.e., clarify how you measure the observed usability).

In case you do not have a non-functional user story at hand from Lab 1, please recall how non-functional requirements might look like and could be re-written as user stories:

Example of usability requirement UR-X:

• At most 1 out of 3 users shall encounter a critical problem when using the system and performing tasks A, B, and C. There shall be not more than two cases where a user needs more than the specified maximum duration defined to complete a task.

UR-X could be rewritten as user stories as follows:

• As an end user, I want to be able to perform task A / B / C without encountering a critical problem, so that my usage of the system is not blocked.
• As an end user, I want to be able to perform task A / B / C within 1 minute (each), so that I am not unnecessarily delayed when using the system.

Example of usability test for UR-X:

• Define tasks A, B, and C (e.g., task A could be the entering of a new price for an existing item).
• Define what a "critical" problem is.
• For each task state the start condition and the end condition.
• For each task state the maximum time span in which the task shall be completed.
• Make clear whether the test applies to using the CLI, the GUI, or to both.

Task 2 (7p)

Part 2.1 (5p): In part 2.1 of Task 2, you have to follow the steps proposed in Task 3 of the previous lab (Lab 6) to test your own software system. Report the test cases you executed and whether a test case passed or failed. If it failed, say how the actual result differed from the expected result specified in the test case.

Part 2.2 (2p): In part 2.2 of Task 2, you have to test the system of another group. Which system you have to test will be decided during the assessment lab of homework 7. Once you know which system you have to test, follow the steps proposed in Task 1 of this lab, as described by the team that provides the system you have to test. You must document the application of the usability test steps and the achieved test results, as well as all issues you experience during usability testing (if any). We suggest that each team member performs each of the two usability tests specified by the other team individually. Then you compare what you did and provide the test report.
Note: While you perform the usability test of a system developed by another team, you have to provide your functioning system in a zip file as well as two specified usability test cases (or: test scenarios) so that another team can test the usability of your system.

To generate a distribution of your POS System, we suggest you follow these steps:

1. Make sure the system is using InMemorySalesSystemDAO instead of HibernateSalesSystemDAO. This is to prevent additional database-related problems in other’s computers.
2. Create a zip file with your POS system’s distribution by running gradlew :SaleSystemGUI:distZip or gradlew :SaleSystemCLI:distZip.
3. Make sure the build is working properly. You should test your distribution because you don’t want to share a system that doesn’t work. To do this, unzip the zip file generated in step 2 and just run the script inside the bin folder.
4. Share the distribution. Send the zip file by email, dropbox, or drive to the group that is in charge of testing your system.

Important:

• If you do not provide two defined usability tests at the beginning of the assessment lab, you get 1 point penalty.
• If you do not provide a functioning system in a zip file at the beginning of the assessment lab, you get 1 point penalty.

Submission

Your results must be submitted using your Bitbucket wiki.

NOTE: In the assessment/consultation lab you must be able to show the designs of two usability test scenarios (Task 1) and the test reports (Task 2) of at least two functional requirements. In addition, you must provide your own system as a zip file so another team can test your system as a black-box following your specified usability test steps.

Links

These quite general wiki articles should be considered as a starting point for your further research. There exists many guidelines and tools for the system verification, and you need to become familiar with that field. After this small homework we expect you to be familiar with the main tools and terms used for the V&V and software tesing in general.

During the evaluation you should be able to describe at least major terms like:

• Usability Testing
• Testing levels (check this)
• Test plan
• Verification and Validation procedures
• Software testing in general
• Test plan
• Test case
• Usability testing