HW9. Machine Learning III (23.04)
1. Kernel trick in SVM allows to use higher-dimensional mappings φ(x). Provide the mapping φ() for vectors X and Y, when the dot product φ(X)·φ(Y) can be calculated as 〈X · Y〉3. For simplicity - assume 2-dimensional space. (Follow closely the example from slide nr 141 - which was an example for 〈X · Y〉2).
2. Read the article by Domingos: A few useful things to know about machine learning (communications of the ACM, Vol. 55 No. 10, Pages 78-87 doi: 10.1145/2347736.2347755 via ACM Digital library, https://courses.cs.ut.ee/MTAT.03.183/2012_fall/uploads/Main/domingos.pdf). Make a list of key messages with a supporting 1-2 sentence example or clarification of that message (something like short summary of the article)
3. Let's define a circle on 2-dimensional space, with radius 3 and center in (5;5). Provide a function that defines whether a point on a plane is within the circle or outside. Provide the linear classifier formulation for this; and the scoring defined as distance to that separating hyperplane (for how strong is the prediction). Calculate that score for 5 different points - inside the circle and outside; near the hyperplane and far. E.g. use the points (2.8;2.8), (3;3), (6;6), (4;8), (9;9).
Hint: Separating hyperplane in this case is just the circle.
4. and 5. Generate randomly 20, 50, 100 points in the range (0..10,0..10) and label them as positive (within the circle) or negative (outside the circle). Learn 4 different classifier types (e.g. decision tree, random forest, SVM with 2 different kernels), in each case. Label the full area (all points) in (0..10, 0..10) with 0.5 step using the learned classifier (for example if for point (2, 1.5) your classifier predicts positive, you label that area positive). Visualise the "area" learned by classifier with two different colors. (Hint: something similar as the http://scikit-learn.org/stable/ homepage has as an example; but simpler, you can just use dots etc).
(Only after completion of 5):
6. (Bonus 2p) Generate a smiley face like area in first quadrant as the "true class". Generate sample data points as in Task 5, you can make more data points. Use the SVM with radial basis function. Explain the principle of RBF use in the SVM learning. Use some simple forms of smiley face, e.g. similar to http://www.publicdomainpictures.net/view-image.php?image=42226&picture=smiley-silhouette