Distributed Data Processing on the Cloud 2018/19 fall

Practice Sessions

Practice sessions are supervised by Pelle Jakovits (Office: Ülikooli 17, Room - 324, jakovits ät ut . ee)

NB! Practice sessions start from the second week
NB! We strongly encourage you to bring personal laptops to the labs

The lab work in this course involves significant amount of programming. We will mainly work with Java (first half of the course) and Python (second half) programming languages, but we will also briefly touch SQL, R and JavaScript.

Practice sessions are held on:

• Monday at 14.15 - 16.00 in Ülikooli 17 - 115 week 2-16
• Tuesday at 12.15 - 14.00 in Ülikooli 17 - 115 week 2-16

Practice Session schedule

• Practice session 1: Requesting and utilizing Cloud computing resources
  • Deadline 18.09
• Practice session 2: Introduction to Hadoop MapReduce (Java)
  • Deadline 25.09
• Practice session 3: Processing data with Hadoop MapReduce (Java)
  • Deadline 02.10
• Practice session 4: Hadoop MapReduce in Information Retrieval (Java)
  • Deadline 09.10

08.10 & 09.10 - * NO lab session this week - Due to no lecture on 05.10 because of ICS Day

• Practice session 5: Graph Data Processing with Hadoop MapReduce (Java)
  • Deadline 23.10
• Practice session 6: Joins in Hadoop MapReduce (Java)
  • Deadline 30.10
• Practice session 7: Higher level scripting language: Apache Pig (Pig Latin)
  • Deadline 06.11
• Practice session 8: In-memory data processing: Apache Spark (Python)
  • Deadline 13.11
• Practice session 9: SQL in apache Spark (SQL, Python)
  • Deadline 20.11
• Practice session 10: DataFrames in Apache Spark (Python)
  • Deadline 27.11
• Practice session 11: Stream Data Processing: Spark Streaming (Python)
  • Deadline 04.12
• Practice session 12: Graph processing with Spark GraphFrames
  • Deadline 11.12
• Practice session 13: Non-Relational databases (CouchDB, JavaScript)
  • Deadline 18.12
• Practice session 14: Machine Learning in Apache Spark (Python)
  • Deadline 25.12

Lab grading rules

You must submit lab exercise solutions by Tuesday evening (23:59) of the week after to get 100% score

• Up to 1 week late submission gives maximum of 75% grade
• Up to 2 weeks late submission gives maximum of 50% grade

 NB! You need to score at least 50% in total in labs to be allowed to take the exam!