Due to the still unstable COVID-19 situation, the February lectures will be conducted fully online. You will find the instructions in Moodle (use the BBB link). Beginning with Lecture 3, all lectures will be delivered in the classroom (with Panopto video recording). Practice sessions will be conducted physically in the classroom (no video recording).
Here you find the lecture slides and any other materials handed out during a lecture or related to a lecture and associated lab sessions. The detailed lab instructions can be found under the tab Labs .
Disclaimer: The following schedule might be updated as needed.
SW Testing Lectures:
- Lecture 1 (10.02) - Introduction to Software Testing - Online (Link in Moodle)
- This lecture is related to Quiz 1 and HW 1
- Slides-2022 (all parts) - video can be found in Moodle
- Videos (with exception of Part3 based on 2021 slides - but same content as in 2022):
- Video-2021-Part1 (36:18 min) - Introduction & Motivation (Relevant for Quiz 1)
- Video-2022-Part2 - (27:58 min) Course Information (updated to 2022)
- Video-2021-Part3 (36:49 min) - Basic Vocabulary (Relevant for Quiz 1)
- Video-2021-Part4 (34:41 min) - Lab 1 (Relevant for homework assignment HW 1) - Note that dates are from 2021!
- Link to an introductory tutorial on testing: Guru99
- Link to an introductory survey of testing concepts: SoftwareTestingFundamentals
- The "Heartbleed Bug" explained: (link) - Think about a test case that would have found this bug!
- A recent book on debugging by Andreas Zeller: (link)
- Lecture 2 (17.02) - Basic Black-Box Testing Techniques: Boundary Value Analysis & Equivalence Class Partitioning - Online (Link in Moodle)
- This lecture is related to Quiz 2 and HW 2
- Slides-2022 (all parts) - video can be found in Moodle
- Videos:
- Video-2021-Part1 (21:32 min) - Black-Box versus White-Box Testing (Relevant for Quiz 2)
- Video-2021-Part2a (23:21 min) - ECP & BVA (Relevant for Quiz 2)
- Video-2021-Part2b (39:09 min) - ECP & BVA (continued); there was a Panopto problem that stopped the recording of Part 2 after 23:21 min (Relevant for Quiz 2)
- Video-2021-Part3 (8:13 min) - Lab 2 (Relevant for homework assignment HW 2) - Note that dates are from 2021!
- No lecture on 24.02. due to Estonian National Holiday -- Beginning with Lecture 3 all lectures take place in the classroom (no BBB virtual session)
- Lecture 3 (03.03) - Black-Box Testing (advanced): Cause-Effect Graphing & Combinatorial Testing
- This lecture is related to Quiz 3 and HW 3
- Slides-2022 + Video-2022 (all parts)
- Videos:
- Video-2021-Part1 (20:45 min) - Cause-Effect Graphing (Relevant for Quiz 3)
- Video-2021-Part2 (57:32 min) - Combinatorial Testing (Relevant for Quiz 3)
- Video-2021-Part3 (10:28 min) - Lab 3 (Relevant for homework assignment HW 3)
- Two papers and one NIST report on combinatorial testing:
- Lecture 4 (10.03) - Basic White-Box Testing Techniques: Instruction & Control-Flow Coverage
- This lecture is related to Quiz 4 and HW 4
- Slides-2022 + Video-2022 (all parts)
- Videos:
- Video-2021-Part1 (32:54 min) - Introduction to Code Coverage (Relevant for Quiz 4)
- Video-2021-Part2a (32:36 min) - Code Coverage Criteria in Detail (Relevant for Quiz 4)
- Video-2021-Part2b (37:13 min) - Code Coverage Criteria in Detail (continued); there was a Panopto problem that stopped the recording of Part 2 after 32:36 min (Relevant for Quiz 4)
- Video-2021-Part3 (4:11 min) - Lab 4 (Relevant for homework assignment HW 4)
- Code Coverage tools:
- Lecture 5 (17.03) - Black-Box Testing (advanced): State-Transition Testing, Random Testing, Metamorphic Testing
- This lecture is related to Quiz 5 and HW 5
- Slides-2022 + Video-2022 (all parts)
- Videos:
- Video-2020 (all parts)
- Video-2021-Part1a (24:13 min) - State-Transition Testing (Relevant for Quiz 5)
- Video-2021-Part1b (18:24 min) - State-Transition Testing (continued); there was a Panopto problem that stopped the recording of Part 1 after 24:13 min (Relevant for Quiz 5)
- Video-2021-Part2 (29:11 min) - Random Testing (Relevant for Quiz 5)
- Video-2021-Part3a (6:24 min) - Certification Testing (Relevant for Quiz 5)
- Video-2021-Part3b (19:47 min) - Certification Testing (continued); there was a Panopto problem that stopped the recording of Part 1 after 6:24 min (Relevant for Quiz 5)
- Video-2021-Part4a (28:18 min) - Metamorphic Testing (Relevant for Quiz 5)
- Video-2021-Part4b (4:17 min) - Metamorphic Testing (continued); there was a Panopto problem that stopped the recording of Part 1 after 28:18 min (Relevant for Quiz 5)
- Video-2021-Part5 (6:41 min) - Lab 5 (Relevant for homework assignment HW 5)
- Random Testing Tools:
- Further readings:
- Random Testing (Wikipedia)
- Fuzzing (Wikipedia)
- Fuzzing (online textbook by Andreas Zeller, Rahul Gopinath, Marcel Böhme, Gordon Fraser, and Christian Holler)
- Lecture 6 (24.03) - Test Levels, Test Tools, Test Automation
- This lecture is related to Quiz 6 and HW 6
- Slides-2022 + Video-2022 (all parts)
- Videos:
- Video-2020-Part1 (46:50 min) - Test Levels (Relevant for Quiz 6)
- Video-2020-Part2 (42:25 min) - Test Tools (Relevant for Quiz 6 & HW 6)
- Video-2021-Part3 (12:45 min) - Test Automation (Relevant for Quiz 6)
- Video-2021-Part4 (3:43 min) - Test Automation (Relevant for homework assignment HW 6)
- Lists of QA/Test Tools:
- Tool Examples:
- Application Example:
- Lecture 7 (31.03) - Black-Box Testing (advanced): Exploratory Testing / Behaviour Testing
- This lecture is related to Quiz 7
- Slides-2022 + Video-2022 (all parts)
- Videos:
- Video-2021-Part0 (4:51 min) - Introduction to Lecture 7 - Please watch this before you watch Parts 1 and 2!
- Video-2020-Part1 (33:32 min) - Exploratory Testing (Relevant for Quiz 7)
- Video-2020-Part2 (39:04 min) - Behaviour Testing (Relevant for Quiz 7 - up to 34:15)
- Note: In 2020, we had a homework assignment (Lab 7 in the video) on behaviour testing applied to web application testing; in the 2020 homework assignment, students were supposed to use Behave instead of Cucumber and the target application was programmed in Python; I decided to not cut out from the video the introduction to 2020's lab 7 (starts at 34:15) in case some of you are interested in doing lab 7 from 2020 purely out of interest; if you are interested, you may look at the 2020 course wiki of the Software Testing course and find all relevant materials; note that this is really a completely voluntary thing to do just out of interest; you cannot submit this homework for grading and you won't get any marks for it.
- Gherkin - a business readable Domain Specific Language (DSL) created for describing the behavior of a system
- Cucumber for Behavior Testing with Gherkin
- Exploratory Testing Tools:
- List of ET tools
- Test Gap Analysis with Teamscale (YouTube video 14:21 min)
- Lecture 8 (07.04) - Black-Box Testing (advanced): GUI Testing / Visual Testing / Security Testing / Usability Testing / A/B Testing
- This lecture is related to Quiz 8 and HW 7
- Slides-2022 + Video-2022 (all parts)
- Videos:
- Video-2021-Part0 (3:28 min) - Introduction to Lecture - Please watch this before you watch Parts 1-2, 3, and 4-6!
- Video-2020-Part1-2 (46:14 min) - GUI Testing with GUITAR and TESTAR (Relevant for Quiz 8)
- Video-2021-Part3 (9:40 min) - Visual GUI Testing with SikuliX (Relevant for Quiz 8 and Lab/HW 7)
- Video-2019-Part4-6 - (48:31 min) Security, Usability, and A/B Testing (Relevant for Quiz 8) - Note: This video is from 2019; subject matter content in slides has not changed
- GUI Testing / Visual Testing Tools:
- GUITAR - A GUI Testing Framework
- Testar for script-less testing
- Link to SikuliX homepage --> To be used in Lab/HW 07
- Sikuli Tutorial
- A good source on A/B Testing: How to build a strong A/B testing plan
- Lecture 9 (14.04) - Security Testing of Mobile Applications (by Kristiina Rahkema, UT and Nestri Solutions OÜ)
- This lecture is related to HW 8 (no Quiz this week)
- Slides-2022 + Video-2022
- Slides-KristiinaRahkema (video) -- Note: This is the 2019 Guest Lecture + Video
- Kristina's blog post
- Lecture 10 (21.04) - White-Box Testing (advanced): Data-Flow Testing & Mutation Testing
- This lecture is related to Quiz 9 and HW 9 -- NB: In 2021 and before, this was Lecture 9 and the related homework assignment was HW 8!
- Slides-2022 + Video-2022 (all parts)
- Videos: Note that the first two videos are from 2020 and, thus, it shows Lab 9 on the slides instead of Lab 8 (which is the lab and HW assignment corresponding to this lecture in 2021)
- Video-2020-Part1 (56:21 min) - Data-Flow Testing (Relevant for Quiz 9)
- Video-2020-Part2 (29:33 min) - Mutation Testing (Relevant for Quiz 9 and Lab/HW 9)
- Video-2021-Part3 (7:33 min) - Lab 9 (Relevant for homework assignment HW 9)
- Readings/Materials:
- Paper on the effectiveness of code coverage criteria:
- Paper by Hemmati, H.: "How Effective Are Code Coverage Criteria?", in Proc. of the International Conference on Software Quality, Reliability and Security (QRS 2015), IEEE, p. 151-156
- Two papers listing mutation operators for Java:
- Paper on the effectiveness of code coverage criteria:
- Tools:
- Mutation testing tool PIT: pitest.org
- YouTube videos on mutation testing:
- Lecture 11 (28.04) - White-Box Testing (advanced): Symbolic Execution, Static Code Analysis, Document Inspections, Code Review
- This lecture is related to Quiz 10 and HW 10 & HW 11 -- NB: In 2021 and before, this was Lecture 10 and the related homework assignments were HW 9 and HW 10!
- Slides-2022 + Video-2022 (all parts)
- Videos:
- Video-2020-Part1 (24:38 min) - Symbolic Execution (Relevant for Quiz 10)
- Video-2020-Part2a (16:38 min) - Static Code Analysis (Relevant for Quiz 10 and Lab/HW 10)
- Video-2020-Part2b (6:00 min) - Lab/HW 10 Introduction
- Video-2020-Part3 (39:45 min) - Review (Relevant for Quiz 10 and Lab/HW 11)
- Readings/Materials:
- Two papers on document inspections & reviews:
- Tool Examples:
- Symbolic PathFinder tool by NASA: Symbolic PathFinder on GitHub
- Static code analysis tool FindBugs: FindBugs web-page
- Static code analysis tool PMD: PMD web-page
- Code review support tool Gerrit (works with Git): Gerrit Tutorial
- Survey conducted by JetBrains about code review tools: JetBrains Survey Results
- Lecture 12 (05.05) - Quality Estimation / Test Documentation, Organisation and Process Improvement (Test Maturity Model)
- This lecture is related to Quiz 11 and HW 11 -- NB: In 2021 and before, this was Lecture 11 and the related homework assignment was HW 10!
- Slides-2022 + Video-2022 (all parts)
- Videos:
- Video-2021-Part0 (16:03 min) - Administrative information and re-cap of last week's lecture (Relevant for Exam)
- Video-2021-Part1 (49:49 min) - Quality Estimation (Relevant for Quiz 11 and Lab/HW 11)
- Video-2021-Part2 (5:58 min) - Lab/HW 11 Introduction
- Video-2021-Part3-5 - (26:06 min) - Test Documentation / Test Organization / Test Maturity Models
- Readings/Materials/Videos:
- Two papers on the question "What makes a good bug report?":
- Test incident report (IEEE 829-1998):
- Book Chapter by Kit: Organisational approaches to testing
- Industry Guest Lecture by James Bach on the nature of software testing and what makes a good software tester (link)
- Industry Guest Lecture by Almer Mermer on process and people aspects of agile testing (link)
- TMMi Framework
- Lecture 13 (12.05) - Exam Preparation & Q/A Session - Online (Link in Moodle)
- Slides-2022 - video can be found in Moodle
- Video:
- Video-2021-Part1 (17:27 min) - Info about the exam until Panopto unexpectedly stopped
- Video-2021-Part2 (5:17 min) - Continuation of Part 1
- Lecture 14 (19.05) - Advanced Topics in Software QA & Testing (Bonus lecture - not relevant for exam) - Online (Links to videos below)
- State-of-the-art research presentations (keynotes at ESEC/FSE 20019 in Tallinn):
- Keynote 1: Marta Kwiatkowska, Professor of Computing Systems and Fellow of Trinity College, University of Oxford, UK.
- Presentation: Safety and Robustness for Deep Learning with Provable Guarantees - (video)
- Keynote 2: Joanne Atlee, Professor and Director of Women in Computer Science, University of Waterloo, Canada
- Presentation: Living with Feature Interactions - (video)
- Keynote 1: Marta Kwiatkowska, Professor of Computing Systems and Fellow of Trinity College, University of Oxford, UK.
- State-of-the-art research presentations (keynotes at ESEC/FSE 20019 in Tallinn):