All lectures will be delivered in the classroom (with Panopto video recording). Practice sessions will be conducted physically in the classroom (no video recording).
Here you find the lecture slides and any other materials handed out during a lecture or related to a lecture and associated lab sessions. The detailed lab instructions can be found under the tab Labs .
Disclaimer: The following schedule is still subject to changes.
SW Testing Lectures:
- Lecture 0 (09.02) - Introduction to Software Testing
- This lecture is related to Quiz 0
- Slides-L0-2023
- Video-L0-2023
- Older videos from previous years (content very similar to 2023):
- Video-2021-Part1 (36:18 min) - Introduction & Motivation (Relevant for Quiz 0)
- Video-2022-Part2 - (27:58 min) Course Information (updated to 2022)
- Video-2021-Part3 (36:49 min) - Basic Vocabulary (Relevant for Quiz 0)
- Link to an introductory tutorial on testing: Guru99
- Link to an introductory survey of testing concepts: SoftwareTestingFundamentals
- The "Heartbleed Bug" explained: (link) - Think about a test case that would have found this bug!
- A recent book on debugging by Andreas Zeller: (link)
- Lecture 1 (16.02) - Debugging & Basic Black-Box Testing Techniques: Equivalence Class Partitioning (Part 1)
- This lecture is related to Quizzes 1 & 2 and HWs 1 & 2
- Slides-L1-2023
- Video-L1-2023
- Older videos from previous years (content very similar to 2023):
- Video-2021-Part0 (34:41 min) - Lab 1 (Relevant for homework assignment HW 1) - Note that dates mentioned in the video are from 2021 and, thus, incorrect!
- Video-2021-Part1 (21:32 min) - Black-Box versus White-Box Testing (Relevant for Quiz 2)
- Lecture 2 (23.02) - Basic Black-Box Testing Techniques: Equivalence Class Partitioning (Part 2) & Boundary Value Analysis
- This lecture is related to Quiz 2 and HW 2
- Slides-L2-2023
- Video-L2-2023
- Older videos from previous years (content very similar to 2023):
- Video-2021-Part2a (23:21 min) - ECP & BVA (Relevant for Quiz 2)
- Video-2021-Part2b (39:09 min) - ECP & BVA (continued); there was a Panopto problem that stopped the recording of Part 2 after 23:21 min (Relevant for Quiz 2)
- Video-2021-Part3 (8:13 min) - Lab 2 (Relevant for homework assignment HW 2) - Note that dates are from 2021!
- Lecture 3 (02.03) - Black-Box Testing (advanced): Cause-Effect Graphing & Combinatorial Testing
- This lecture is related to Quiz 3 and HW 3
- Slides-L3-2023
- Video-L3-2023
- Older videos from previous years (content very similar to 2023):
- Video-2022 (all parts)
- Video-2021-Part1 (20:45 min) - Cause-Effect Graphing (Relevant for Quiz 3)
- Video-2021-Part2 (57:32 min) - Combinatorial Testing (Relevant for Quiz 3)
- Video-2021-Part3 (10:28 min) - Lab 3 (Relevant for homework assignment HW 3)
- Two papers and one NIST report on combinatorial testing:
- Lecture 4 (09.03) - Basic White-Box Testing Techniques: Instruction & Control-Flow Coverage
- This lecture is related to Quiz 4 and HW 4
- Slides-L4-2023
- Video-L4-2023
- Older videos from previous years (content very similar to 2023):
- Video-2022 (all parts)
- Video-2021-Part1 (32:54 min) - Introduction to Code Coverage (Relevant for Quiz 4)
- Video-2021-Part2a (32:36 min) - Code Coverage Criteria in Detail (Relevant for Quiz 4)
- Video-2021-Part2b (37:13 min) - Code Coverage Criteria in Detail (continued); there was a Panopto problem that stopped the recording of Part 2 after 32:36 min (Relevant for Quiz 4)
- Video-2021-Part3 (4:11 min) - Lab 4 (Relevant for homework assignment HW 4)
- Code Coverage tools:
- Lecture 5 (16.03) - Black-Box Testing (advanced): State-Transition Testing, Random Testing, Metamorphic Testing
- This lecture is related to Quiz 5 and HW 5
- Slides-L5-2023
- Video-L5-2023
- Older videos from previous years (content very similar to 2023):
- Video-2022 (all parts)
- Video-2021-Part1a (24:13 min) - State-Transition Testing (Relevant for Quiz 5)
- Video-2021-Part1b (18:24 min) - State-Transition Testing (continued); there was a Panopto problem that stopped the recording of Part 1 after 24:13 min (Relevant for Quiz 5)
- Video-2021-Part2 (29:11 min) - Random Testing (Relevant for Quiz 5)
- Video-2021-Part3a (6:24 min) - Certification Testing (Relevant for Quiz 5)
- Video-2021-Part3b (19:47 min) - Certification Testing (continued); there was a Panopto problem that stopped the recording of Part 1 after 6:24 min (Relevant for Quiz 5)
- Video-2021-Part4a (28:18 min) - Metamorphic Testing (Relevant for Quiz 5)
- Video-2021-Part4b (4:17 min) - Metamorphic Testing (continued); there was a Panopto problem that stopped the recording of Part 1 after 28:18 min (Relevant for Quiz 5)
- Video-2021-Part5 (6:41 min) - Lab 5 (Relevant for homework assignment HW 5)
- Video-2020 (all parts)
- Random Testing Tools:
- Further readings:
- Random Testing (Wikipedia)
- Fuzzing (Wikipedia)
- Fuzzing (online textbook by Andreas Zeller, Rahul Gopinath, Marcel Böhme, Gordon Fraser, and Christian Holler)
- Lecture 6 (23.03) - Test Levels, Test Tools, Test Automation
- This lecture is related to Quiz 6 and HW 6
- Slides-L6-2023
- Video-L6-2023
- Older videos from previous years (content very similar to 2023):
- Video-2022 (all parts)
- Video-2020-Part1 (46:50 min) - Test Levels (Relevant for Quiz 6)
- Video-2020-Part2 (42:25 min) - Test Tools (Relevant for Quiz 6 & HW 6)
- Video-2021-Part3 (12:45 min) - Test Automation (Relevant for Quiz 6)
- Video-2021-Part4 (3:43 min) - Test Automation (Relevant for homework assignment HW 6)
- Lists of QA/Test Tools:
- Tool Examples:
- Application Example:
- Lecture 7 (30.03) - Black-Box Testing (advanced): Exploratory Testing / Behaviour Testing
- This lecture is related to Quiz 7
- Slides-L7-2023
- Video-L7-2023
- Older videos from previous years (content very similar to 2023):
- Gherkin - a business readable Domain Specific Language (DSL) created for describing the behavior of a system
- Cucumber for Behavior Testing with Gherkin
- Exploratory Testing Tools:
- List of ET tools
- Test Gap Analysis with Teamscale (YouTube video 14:21 min)
- Lecture 8 (06.04) - Black-Box Testing (advanced): GUI Testing / Visual Testing / Security Testing / Usability Testing / A/B Testing
- This lecture is related to Quiz 8 and HW 7
- Slides-L8-2023
- Video-L8-2023
- Older videos from previous years (content similar to 2023):
- Video-2022 (all parts)
- Video-2021-Part0 (3:28 min) - Introduction to Lecture - Please watch this before you watch Parts 1-2, 3, and 4-6!
- Video-2020-Part1-2 (46:14 min) - GUI Testing with GUITAR and TESTAR (Relevant for Quiz 8)
- Video-2021-Part3 (9:40 min) - Visual GUI Testing with SikuliX (Relevant for Quiz 8 and Lab/HW 7)
- Video-2019-Part4-6 - (48:31 min) Security, Usability, and A/B Testing (Relevant for Quiz 8) - Note: This video is from 2019; subject matter content in slides has not changed
- GUI Testing / Visual Testing Tools:
- GUITAR - A GUI Testing Framework
- Testar for script-less testing
- Link to SikuliX homepage --> To be used in Lab/HW 07
- Sikuli Tutorial
- A good source on A/B Testing: How to build a strong A/B testing plan
- Lecture 9 (13.04) - Security Testing of Mobile Applications (by Kristiina Rahkema, UT and Nestri Solutions OÜ)
- This lecture is related to HW 8 (no Quiz this week)
- Slides-L9-2023
- Video-L9-2023
- Older videos from previous years (content similar to 2023):
- Video-2022
- Slides-KristiinaRahkema (video) -- Note: This is the 2019 Guest Lecture + Video
- Kristina's blog post
- Lecture 10 (20.04) - White-Box Testing (advanced): Data-Flow Testing & Mutation Testing
- This lecture is related to Quiz 9 and HW 9
- Slides-L10-2023
- Video-L10-2023
- Older videos from previous years (content very similar to 2023):
- Video-2022 (all parts)
- Video-2020-Part1 (56:21 min) - Data-Flow Testing (Relevant for Quiz 9)
- Video-2020-Part2 (29:33 min) - Mutation Testing (Relevant for Quiz 9 and Lab/HW 9)
- Video-2021-Part3 (7:33 min) - Lab 9 (Relevant for homework assignment HW 9)
- Readings/Materials:
- Paper on the effectiveness of code coverage criteria:
- Paper by Hemmati, H.: "How Effective Are Code Coverage Criteria?", in Proc. of the International Conference on Software Quality, Reliability and Security (QRS 2015), IEEE, p. 151-156
- Two papers listing mutation operators for Java:
- Paper on the effectiveness of code coverage criteria:
- Tools:
- Mutation testing tool PIT: pitest.org
- YouTube videos on mutation testing:
- Lecture 11 (27.04) - White-Box Testing (advanced): Symbolic Execution, Static Code Analysis, Document Inspections, Code Review
- This lecture is related to Quiz 10 and HW 10 & HW 11
- Slides-L11-2023
- Video-L11-2023
- Older videos from previous years (content very similar to 2023):
- Video-2022 (all parts)
- Video-2020-Part1 (24:38 min) - Symbolic Execution (Relevant for Quiz 10)
- Video-2020-Part2a (16:38 min) - Static Code Analysis (Relevant for Quiz 10 and Lab/HW 10)
- Video-2020-Part2b (6:00 min) - Lab/HW 10 Introduction
- Video-2020-Part3 (39:45 min) - Review (Relevant for Quiz 10 and Lab/HW 11)
- Readings/Materials:
- Two papers on document inspections & reviews:
- White paper on the importance of requirements and their impact on quality (2022): https://www.fortiss.org/fileadmin/user_upload/06_Ergebnisse/Whitepaper/fortiss_Whitepaper_Requirements_Engineering_WEB.pdf
- Tool Examples:
- Static code analysis tool SpotBugs: https://spotbugs.github.io
- Static code analysis tool FindBugs: FindBugs web-page
- Static code analysis tool PMD: PMD web-page
- Symbolic PathFinder tool by NASA: Symbolic PathFinder on GitHub
- Code review support tool Gerrit (works with Git): Gerrit Tutorial
- Survey conducted by JetBrains about code review tools: JetBrains Survey Results
- Lecture 12 (04.05) - Quality Estimation / Test Documentation, Organisation and Process Improvement (Test Maturity Model)
- This lecture is related to Quiz 11 and HW 11
- Slides-L12-2023
- Video-L12-2023
- Older videos from previous years (content very similar to 2023):
- Video-2022 (all parts)
- Video-2021-Part0 (16:03 min) - Administrative information and re-cap of last week's lecture (Relevant for Exam)
- Video-2021-Part1 (49:49 min) - Quality Estimation (Relevant for Quiz 11 and Lab/HW 11)
- Video-2021-Part2 (5:58 min) - Lab/HW 11 Introduction
- Video-2021-Part3-5 - (26:06 min) - Test Documentation / Test Organization / Test Maturity Models
- Readings/Materials/Videos:
- Two papers on the question "What makes a good bug report?":
- Test incident report (IEEE 829-1998):
- Book Chapter by Kit: Organisational approaches to testing
- Industry Guest Lecture by James Bach on the nature of software testing and what makes a good software tester (link)
- Industry Guest Lecture by Almer Mermer on process and people aspects of agile testing (link)
- TMMi Framework
- Lecture 13 (11.05) - Exam Preparation & Q/A Session
- Lecture 14 (18.05) - Advanced Topics in Software QA & Testing (Bonus lecture - not relevant for exam) - Online (Links to videos below)
- State-of-the-art research presentations (keynotes at ESEC/FSE 20019 in Tallinn):
- Keynote 1: Marta Kwiatkowska, Professor of Computing Systems and Fellow of Trinity College, University of Oxford, UK.
- Presentation: Safety and Robustness for Deep Learning with Provable Guarantees - (video)
- Keynote 2: Joanne Atlee, Professor and Director of Women in Computer Science, University of Waterloo, Canada
- Presentation: Living with Feature Interactions - (video)
- Keynote 1: Marta Kwiatkowska, Professor of Computing Systems and Fellow of Trinity College, University of Oxford, UK.
- State-of-the-art research presentations (keynotes at ESEC/FSE 20019 in Tallinn):