Here you find the lecture slides and any other materials handed out during a lecture or related to a lecture and associated lab sessions. The detailed lab instructions can be found under the tab Labs .
Disclaimer: The following schedule might be updated as needed.
SW Testing Lectures:
- Lecture 1 (11.02) - Introduction to Software Testing
- This lecture is related to Quiz 1 and HW 1
- Slides-2021 (all parts)
- Videos:
- Video-2021-Part1 (36:18 min) - Introduction & Motivation (Relevant for Quiz 1)
- Video-2021-Part2 (17:52 min) - Course Information
- Video-2021-Part3 (36:49 min) - Basic Vocabulary (Relevant for Quiz 1)
- Video-2021-Part4 (34:41 min) - Lab 1 (Relevant for homework assignment HW 1)
- Link to an introductory tutorial on testing: Guru99
- Link to an introductory survey of testing concepts: SoftwareTestingFundamentals
- The "Heartbleed Bug" explained: (link) - Think about how a test case that would have found this bug!
- A recent book on debugging by Andreas Zeller: (link)
- Lecture 2 (18.02) - Basic Black-Box Testing Techniques: Boundary Value Analysis & Equivalence Class Partitioning
- This lecture is related to Quiz 2 and HW 2
- Slides-2021 (all parts)
- Videos:
- Video-2021-Part1 (21:32 min) - Black-Box versus White-Box Testing (Relevant for Quiz 2)
- Video-2021-Part2a (23:21 min) - ECP & BVA (Relevant for Quiz 2)
- Video-2021-Part2b (39:09 min) - ECP & BVA (continued); there was a Panopto problem that stopped the recording of Part 2 after 23:21 min (Relevant for Quiz 2)
- Video-2021-Part3 (8:13 min) - Lab 2 (Relevant for homework assignment HW 2)
- Lecture 3 (25.02) - Black-Box Testing (advanced): Cause-Effect Graphing & Combinatorial Testing
- This lecture is related to Quiz 3 and HW 3
- Slides-2021 (all parts)
- Videos:
- Video-2021-Part1 (20:45 min) - Cause-Effect Graphing (Relevant for Quiz 3)
- Video-2021-Part2 (57:32 min) - Combinatorial Testing (Relevant for Quiz 3)
- Video-2021-Part3 (10:28 min) - Lab 3 (Relevant for homework assignment HW 3)
- Two papers and one NIST report on combinatorial testing:
- Lecture 4 (04.03) - Basic White-Box Testing Techniques: Instruction & Control-Flow Coverage
- This lecture is related to Quiz 4 and HW 4
- Slides-2021 (all parts)
- Videos:
- Video-2021-Part1 (32:54 min) - Introduction to Code Coverage (Relevant for Quiz 4)
- Video-2021-Part2a (32:36 min) - Code Coverage Criteria in Detail (Relevant for Quiz 4)
- Video-2021-Part2b (37:13 min) - Code Coverage Criteria in Detail (continued); there was a Panopto problem that stopped the recording of Part 2 after 32:36 min (Relevant for Quiz 4)
- Video-2021-Part3 (4:11 min) - Lab 4 (Relevant for homework assignment HW 4)
- Code Coverage tools:
- Lecture 5 (11.03) - Black-Box Testing (advanced): State-Transition Testing, Random Testing, Metamorphic Testing
- This lecture is related to Quiz 5 and HW 5
- Slides-2021 (all parts)
- Videos:
- Video-2020 (all parts)
- Video-2021-Part1a (24:13 min) - State-Transition Testing (Relevant for Quiz 5)
- Video-2021-Part1b (18:24 min) - State-Transition Testing (continued); there was a Panopto problem that stopped the recording of Part 1 after 24:13 min (Relevant for Quiz 5)
- Video-2021-Part2 (29:11 min) - Random Testing (Relevant for Quiz 5)
- Video-2021-Part3a (6:24 min) - Certification Testing (Relevant for Quiz 5)
- Video-2021-Part3b (19:47 min) - Certification Testing (continued); there was a Panopto problem that stopped the recording of Part 1 after 6:24 min (Relevant for Quiz 5)
- Video-2021-Part4a (28:18 min) - Metamorphic Testing (Relevant for Quiz 5)
- Video-2021-Part4b (4:17 min) - Metamorphic Testing (continued); there was a Panopto problem that stopped the recording of Part 1 after 28:18 min (Relevant for Quiz 5)
- Video-2021-Part5 (6:41 min) - Lab 5 (Relevant for homework assignment HW 5)
- Random Testing Tools:
- Further readings:
- Random Testing (Wikipedia)
- Fuzzing (Wikipedia)
- Fuzzing (online textbook by Andreas Zeller, Rahul Gopinath, Marcel Böhme, Gordon Fraser, and Christian Holler)
- Lecture 6 (18.03) - Test Levels, Test Tools, Test Automation
- This lecture is related to Quiz 6 and HW 6
- Slides-2021 (all parts)
- Videos:
- Video-2020-Part1 (46:50 min) - Test Levels (Relevant for Quiz 6)
- Video-2020-Part2 (42:25 min) - Test Tools (Relevant for Quiz 6 & HW 6)
- Video-2021-Part3 (12:45 min) - Test Automation (Relevant for Quiz 6)
- Video-2021-Part4 (3:43 min) - Test Automation (Relevant for homework assignment HW 6)
- Lists of QA/Test Tools:
- Tool Examples:
- Application Example:
- Lecture 7 (25.03) - Black-Box Testing (advanced): Exploratory Testing / Behaviour Testing
- This lecture is related to Quiz 7
- Slides-2021 (all parts)
- Videos:
- Video-2021-Part0 (4:51 min) - Introduction to Lecture 7 - Please watch this before you watch Parts 1 and 2!
- Video-2020-Part1 (33:32 min) - Exploratory Testing (Relevant for Quiz 7)
- Video-2020-Part2 (39:04 min) - Behaviour Testing (Relevant for Quiz 7 - up to 34:15)
- Note: Last year we had a homework assignment (Lab 7 in the video) on behaviour testing applied to web application testing; in last year's homework assignment, students were supposed to use Behave instead of Cucumber and the target application was programmed in Python; I decided to not cut out from the video the introduction to last year's lab 7 (starts at 34:15) in case some of you are interested in doing lab 7 from last year purely out of interest; if you are interested, you may look at the 2020 course wiki of the Software Testing course and find all relevant materials; note that this is really a completely voluntary thing to do just out of interest; you cannot submit this homework for grading and you won't get any marks for it.
- Gherkin - a business readable Domain Specific Language (DSL) created for describing the behavior of a system
- Cucumber for Behavior Testing with Gherkin
- Exploratory Testing Tools:
- List of ET tools
- Test Gap Analysis with Teamscale (YouTube video 14:21 min)
- Lecture 8 (01.04) - Black-Box Testing (advanced): GUI Testing / Visual Testing / Security Testing / Usability Testing / A/B Testing
- This lecture is related to Quiz 8 and HW 7
- Slides-2021 (all parts)
- Videos:
- Video-2021-Part0 (3:28 min) - Introduction to Lecture - Please watch this before you watch Parts 1-2, 3, and 4-6!
- Video-2020-Part1-2 (46:14 min) - GUI Testing with GUITAR and TESTAR (Relevant for Quiz 8)
- Video-2021-Part3 (9:40 min) - Visual GUI Testing with SikuliX (Relevant for Quiz 8 and Lab/HW 7)
- Video-2019-Part4-6 - (48:31 min) Security, Usability, and A/B Testing (Relevant for Quiz 8) - Note: This video is from 2019; subject matter content in slides has not changed
- GUI Testing / Visual Testing Tools:
- GUITAR - A GUI Testing Framework
- Testar for script-less testing and Testar Manual (2020)
- Link to SikuliX homepage --> To be used in Lab/HW 07
- Sikuli Tutorial
- A good source on A/B Testing: How to build a strong A/B testing plan
- Lecture 9 (08.04) - White-Box Testing (advanced): Data-Flow Testing & Mutation Testing
- This lecture is related to Quiz 9 and HW 8
- Slides-2021 (all parts)
- Videos: Note that the first two videos are from 2020 and, thus, it shows Lab 9 on the slides instead of Lab 8 (which is the lab and HW assignment corresponding to this lecture in 2021)
- Video-2020-Part1 (56:21 min) - Data-Flow Testing (Relevant for Quiz 9)
- Video-2020-Part2 (29:33 min) - Mutation Testing (Relevant for Quiz 9 and Lab/HW 8)
- Video-2021-Part3 (7:33 min) - Lab 8 (Relevant for homework assignment HW 8)
- Readings/Materials:
- Paper on the effectiveness of code coverage criteria:
- Paper by Hemmati, H.: "How Effective Are Code Coverage Criteria?", in Proc. of the International Conference on Software Quality, Reliability and Security (QRS 2015), IEEE, p. 151-156
- Two papers listing mutation operators for Java:
- Paper on the effectiveness of code coverage criteria:
- Tools:
- Mutation testing tool PIT: pitest.org
- YouTube videos on mutation testing:
- Lecture 10 (15.04) - White-Box Testing (advanced): Symbolic Execution, Static Code Analysis, Document Inspections, Code Review
- This lecture is related to Quiz 10 and HW 9 & HW 10
- Slides-2021 (all parts)
- Videos: Note that the videos Part1, Part2a, and Part3 are from 2020 and show an incorrect lab number in the TOC. Last year the lab number was '10' while this year it is '9' (as shown in the new video Part2b)
- Video-2020-Part1 (24:38 min) - Symbolic Execution (Relevant for Quiz 10)
- Video-2020-Part2a (16:38 min) - Static Code Analysis (Relevant for Quiz 10 and Lab/HW 9)
- Video-2020-Part2b (6:00 min) - Lab/HW 9 Introduction
- Video-2020-Part3 (39:45 min) - Review (Relevant for Quiz 10 and Lab/HW 10)
- Readings/Materials:
- Two papers on document inspections & reviews:
- Tool Examples:
- Symbolic PathFinder tool by NASA: Symbolic PathFinder on GitHub
- Static code analysis tool FindBugs: FindBugs web-page
- Static code analysis tool PMD: PMD web-page
- Code review support tool Gerrit (works with Git): Gerrit Tutorial
- Lecture 11 (22.04) - Quality Estimation / Test Documentation, Organisation and Process Improvement (Test Maturity Model)
- This lecture is related to Quiz 11 and HW 10
- Slides-2021
- Videos:
- Video-2021-Part0 (16:03 min) - Administrative information and re-cap of last week's lecture (Relevant for Exam)
- Video-2021-Part1 (49:49 min) - Quality Estimation (Relevant for Quiz 11 and Lab/HW 10)
- Video-2021-Part2 (5:58 min) - Lab/HW 10 Introduction
- Video-2021-Part3-5 - (26:06 min) - Test Documentation / Test Organization / Test Maturity Models
- Readings/Materials/Videos:
- Two papers on the question "What makes a good bug report?":
- Test incident report (IEEE 829-1998):
- Book Chapter by Kit: Organisational approaches to testing
- Industry Guest Lecture by James Bach on the nature of software testing and what makes a good software tester (link)
- Industry Guest Lecture by Almer Mermer on process and people aspects of agile testing (link)
- TMMi Framework
- Lecture 12 (29.04) - Industry Guest Lecture: Security Testing of Mobile Applications by Kristiina Rahkema, Nestri Solutions OÜ
- Slides-KristiinaRahkema (video) -- Note: This is the 2019 Guest Lecture
- Kristina's blog post
- Lecture 13 (06.05.02) - Advanced Topics in Software QA & Testing
- State-of-the-art research presentations (keynotes at ESEC/FSE 20019 in Tallinn):
- Keynote 1: Marta Kwiatkowska, Professor of Computing Systems and Fellow of Trinity College, University of Oxford, UK.
- Presentation: Safety and Robustness for Deep Learning with Provable Guarantees - (video)
- Keynote 2: Joanne Atlee, Professor and Director of Women in Computer Science, University of Waterloo, Canada
- Presentation: Living with Feature Interactions - (video)
- Keynote 1: Marta Kwiatkowska, Professor of Computing Systems and Fellow of Trinity College, University of Oxford, UK.
- State-of-the-art research presentations (keynotes at ESEC/FSE 20019 in Tallinn):
- Lecture 14 (13.05) - Exam Preparation & Q/A Session
- Slides
- Video:
- Video-2021-Part1 (17:27 min) - Info about the exam until Panopto unexpectedly stopped
- Video-2021-Part2 (5:17 min) - Continuation of Part 1