Python Programming Foundations for Data Engineering

By the end of the course, participants will be able to:

• Write, run and debug Python programs.
• Use variables, expressions, strings and built-in data types.
• Work with lists, tuples, sets and dictionaries.
• Apply conditions, loops and comprehensions.
• Create reusable functions, modules and packages.
• Process text, CSV and JSON files.
• Handle errors and write reliable programs.
• Use object-oriented programming where appropriate.
• Manage virtual environments and third-party packages.
• Use logging, testing and type hints to improve code quality.
• Call web APIs and process structured responses.
• Build a practical Python project relevant to data-processing workflows.

Module 1

Python Environment and Developer Workflow

Participants prepare a modern Python development environment and learn how Python programs are created, executed and organised.

Key Topics

• Python 3 installation and interpreter basics
• Command-line fundamentals
• JupyterLab and notebook workflow
• Visual Studio Code and Python extensions
• Scripts, notebooks and interactive execution
• Virtual environments using venv
• Package installation with pip
• Introduction to Git and source control

Module 2

Python Syntax, Variables and Core Data Types

This module introduces Python syntax and the built-in data types used in everyday programming and data-processing tasks.

Key Topics

• Identifiers, keywords, comments and indentation
• Variables and assignment
• Integers, floating-point numbers and Boolean values
• Strings, indexing, slicing and string methods
• Formatted strings and output formatting
• Type conversion and input validation
• Operators and expressions
• Basic type hints

Module 3

Python Collections and Data Structures

Participants learn how to organise, search, update and transform data using Python collection types.

Key Topics

• Lists and list operations
• Tuples and immutable data
• Sets and uniqueness
• Dictionaries and key-value structures
• Nested data structures
• Sorting and searching collections
• Copying and mutability
• Choosing the appropriate data structure

Module 4

Control Flow and Comprehensions

This module develops the ability to express business rules, repeat processing steps and transform collections efficiently.

Key Topics

• Comparison and logical operators
• if, elif and else statements
• for and while loops
• break, continue and loop control
• Nested conditions and loops
• List, set and dictionary comprehensions
• Generator expressions
• Readable alternatives to overly complex expressions

Module 5

Functions, Modules and Reusable Code

Participants learn to organise programs into reusable and testable components.

Key Topics

• Defining and calling functions
• Parameters, return values and default arguments
• Keyword arguments and variable-length arguments
• Scope and namespaces
• Docstrings and code documentation
• Lambda functions and appropriate use cases
• map, filter, zip and built-in iteration tools
• Creating and importing modules
• Organising a small Python package

Module 6

Files, Paths, CSV and JSON Processing

This module focuses on the file formats and data-handling tasks most frequently encountered before moving into Data Science and Big Data tools.

Key Topics

• Reading and writing text files
• Context managers and safe file handling
• Paths and directories using pathlib
• File and directory inspection
• CSV reading and writing
• JSON serialisation and deserialisation
• Processing nested JSON structures
• Encoding, delimiters and malformed data
• Basic file permissions and operational considerations

Module 7

Errors, Exceptions, Logging and Debugging

Participants learn how to detect failures, communicate errors clearly and produce programs that can be operated and supported.

Key Topics

• Syntax, runtime and logical errors
• try, except, else and finally
• Raising built-in and custom exceptions
• Validation and defensive programming
• Python logging levels and handlers
• Debugging with breakpoints and tracebacks
• Assertions and sanity checks
• Writing useful error messages

Module 8

Object-Oriented Programming and Data Models

This module introduces object-oriented design as a practical way to model entities and organise larger programs.

Key Topics

• Classes, objects, attributes and methods
• Constructors and instance state
• Encapsulation and validation
• Inheritance and composition
• Special methods and object representation
• Dataclasses for structured records
• Choosing between dictionaries, dataclasses and classes
• Simple domain modelling

Module 9

APIs and External Data Sources

Participants learn how Python applications exchange information with modern systems using HTTP and structured APIs.

Key Topics

• HTTP requests and responses
• REST API concepts
• Using the requests library
• Query parameters, headers and authentication concepts
• Processing JSON responses
• Status codes and failure handling
• Pagination and rate-limit concepts
• Saving API data for later processing

Module 10

Testing, Code Quality and Project Structure

The final technical module introduces the practices required to maintain reliable Python code in professional environments.

Key Topics

• Readable Python and PEP 8 principles
• Functions with clear responsibilities
• Type hints and static-analysis concepts
• Unit testing with pytest concepts
• Test cases, fixtures and expected results
• Project folders and configuration files
• Dependency files and reproducible environments
• README files and technical documentation