Study Guide

Objective

After completing the course the student is able to:
Understand and describe the data engineering process life cycle

Content

What is Data Engineering
Data Storage and Retrieval
Data Engineering Lifecycle
Extract, Transform and Load (ETL) process
Introduction to Big Data Frameworks

Materials

The learning materials will be introduced during the first lecture.

We may also use the AWS Academy material for studying some parts of the course.

Teaching methods

- Participating in lectures (theory and practice)
- Learning through hands-on programming (classwork assignments)
- Completing homework assignments
- Interacting with the teacher and classmates
- Enhancing knowledge through teamwork projects
- Following the flipped-classroom model (pre-session self-study of theoretical concepts followed by in-class practical application)

Exam schedules

No Exam.

There is a final teamwork project where students must demonstrate their work during a presentation event in week 48.

International connections

The course includes approximately 12 theory sessions and practice sessions where students work with practical tasks.
Additionally, there are 4 online Q&A sessions for extra support.
Furthermore, exercises for classwork and home work that will be partly demonstrated in during contact sessions.

We may also use a flipped-classroom model for some lectures, where students study the theoretical part at home and engage in practical implementation and discussions in class.

Student workload

- Contact teaching:
• We have 12 theory and practice sessions, each lasting 3.5 hours, conducted weekly: 12 x 3.5 = 42
• Additionally, there are 4 online Q&A sessions, each lasting 1 hour.
• Total contact teaching hours per course: 46 hours.

- Homework and teamwork assignment:
• Personal assignments (homework) and independent studies: 69 hours
• Teamwork assignment: 20 hours

Total: approximately 135 hours (5 x 27h)

Content scheduling

Course Topics (pre-planning):
1. Course Overview and Introduction to Data Engineering
2. The Data Engineering Ecosystem
3. Exercise Demo (I)
4. Apache Airflow
5. Data Engineering Life Cycle - Data wrangling and ETL
6. Data Engineering Life Cycle - Data Wrangling and ETL in Airflow
7. Data Engineering Lifecycle - Governance and Compliance
8: Exercise demo
9. Cloud-based data engineering
10 and 11. working on final projects within groups at classroom
12: Final Project presentations

The course includes approximately 12 supervised practice work and theory sessions.
Additionally, 8 personal exercises for homework that will be partly demonstrate in during contact session.
And some classwork assignment that must be submit during the class hours to demonstrate the participation.
Furthermore, the course has a teamwork project that must be done in a group of 4 students.

Note: Exercise work is done individually outside the instructional sessions. The topic of the assignment is specified during the first month of the course.

Further information

Prerequisites:

1- Python Programming:
- Basic Syntax and Control Structures
- Functions and Modules
- Error Handling

2- Object-Oriented Programming (OOP) in Python

3- Pandas Library:
- DataFrames and Series
- Data Manipulation: Skills in reading, writing, filtering, and transforming data using Pandas.

4- Databases


Additionally, experience with a Version Control System is recommended.

Evaluation scale

H-5

Assessment methods and criteria

1) The course is graded on a scale of 0-5

2) Students can achieve 100 points from this course that contains:
- Participation and classwork submission: participating on each lecture and submitting the related classwork assignment during the class hours 3p => 12 X 3 = 36 points.
- Homework assignments: each homework assignment has 5 points. There are 8 homework assignments => 8 x 5 = 40 points.
- Teamwork assignment: 24 points
Note: the teamwork assignment will be graded on scale 0-5 on Itslearning.

3) Evaluation:
50% of total to pass: 50% from participation and classwork + 50% from homework assignments + 50% from the teamwork projects to pass

Note: Grades will be rounded down if they include decimals less than 0.5; otherwise, they will be rounded up. (e.g., 3.4 is rounded down to 3.0, but 3.5 or higher is rounded up to 4.0)

Assessment criteria, fail (0)

The student did NOT get at least 50% of the points in teamwork assignment OR did not get at least 50% of the points in the homework assignments OR did not get at least 50% of the points in participation and classwork submission.

Assessment criteria, satisfactory (1-2)

The student got 50-65% of the points for the homework assignments AND got 50-59% of the points for the participation and classwork assignments submission AND got a grade of 1 - 3 for the teamwork assignment.

Assessment criteria, good (3-4)

The student got 66-85% of the points for the homework assignments AND got 66-85% of the points for the participation and classwork assignments submission AND got a grade of 4 for the teamwork assignment.

Assessment criteria, excellent (5)

The student got at least 86% of the points for the homework assignments AND got at least 86% of the points for the participation and classwork assignments submission AND got a grade of 5 for the teamwork assignment.