Julia programming for ML

Welcome to the Julia programming for Machine Learning course offered by Prof. Klaus-Robert Müller's Machine Learning Group at TU Berlin.

Table of Contents

Goals

The goal of this course is to give you an introduction to the Julia programming language and its Machine Learning ecosystem.

After taking this class, you should be able to write reproducible, unit-tested Julia code and do Machine Learning research in Julia.

Prerequisites

No knowledge of the Julia programming language is required: this course only assumes knowledge of common programming concepts like for-loops and arrays. Occasionally, differences and similarities to Python will be pointed out. If you don't know Python, you can safely ignore these.

Contents

Lectures

The course is taught in five weekly sessions of three hours. In each session, two lectures are taught:

WeekLectureContentExam / Homework
10General Information, Installation & Getting Help
1Basics 1: Types, Control-flow & Multiple Dispatch
22Basics 2: Arrays, Linear Algebra
3Plotting & DataFrames
34Basics 3: Data structures and custom types
5Classical Machine Learning
46Automatic Differentiation
7Deep Learning
58Workflows: Scripts, Experiments & Packages
9Profiling & Debugging

The first three weeks focus on teaching the fundamentals of the Julia programming language. These weeks consist of longer lectures, followed up by shorter, "guided tours" of the Julia ecosystem, including plotting, data-frames and classical machine learning algorithms.

Week four is all about Deep Learning: A comprehensive lecture on automatic differentiation (AD) sheds light on differences between Julia's various AD packages, before giving a brief overview of Flux's Deep Learning ecosystem.

Finally, week five is all about starting your own Julia project, taking a look at the structure of Julia packages and different workflows for reproducible machine learning research. This is followed up by a demonstration of Julia's debugging and profiling utilities.

The lectures and the homework cover the following packages:

PackageLectureDescriptionPython equivalent
LinearAlgebra.jl2Linear algebra (standard library)numpy
Plots.jl3Plotting & visualizationsmatplotlib
DataFrames.jl3Working with and processing tabular datapandas
MLJ.jl5Classical Machine Learning methodsscikit-learn
ChainRules.jl6Forward- & reverse-rules for automatic differentiation
Zygote.jl6Reverse-mode automatic differentiationJAX, PyTorch
Enzyme.jl6Forward- & reverse-mode automatic differentiationJAX
ForwardDiff.jl6Forward-mode automatic differentiation
FiniteDiff.jl6Finite differences
FiniteDifferences.jl6Finite differences
Flux.jl7Deep Learning abstractionsPyTorch, Keras
MLDatasets.jl7Dataset loader
PkgTemplates.jl8Package template
DrWatson.jl8Workflow for scientific projects
Debugger.jl9Debugger
Infiltrator.jl9Debugger
ProfileView.jl9Profiler
Cthulhu.jl9Type inference debugger

Homework

When learning a programming language, it helps to write code. The lectures from week 1 to 4 are accompanied by homework notebooks that test your understanding of the contents. These notebooks contain automated feedback in the form of tests that need to be passed.

For TU Berlin students, the homework will also be evaluated on a second, slightly different test suite to avoid cheating and hard-coded answers.

Designing homework exercises takes a lot of time, so please don't upload any answers to the homework to the internet.

Information for TU Berlin students

Julia Programming for Machine Learning (3 ECTS credits) is an optional course within one of the following modules:

It's not possible to take the class as a standalone, seminar, or free-choice module.

Homework assignments must be submitted every week. You must be enrolled on ISIS to submit homework. If you do not register on time, you cannot pass the course.

More information and links to the Zoom meetings can be found on ISIS.

FAQ

Why should I learn Julia?

Paraphrasing the official website, Julia is:

If these are features that sound appealing to you, you should learn Julia!

How can I run the notebooks?

Running notebooks is described in the "Opening lectures & homework" section of the Installation notebook.

Alternatively, you can open the notebook on Binder by clicking "Edit or run this notebook". However, Binder can take a prohibitively long time to load. Pluto notebooks show an estimate of the loading times above the "Run in Binder" button.

If you are familiar with Git, you can also clone the GitHub repository of this course. You can then open your local copy of the lectures and homework in Pluto. Just make sure to regularly pull to keep your copy of the course up to date.

I found an error or typo. What should I do?

Please help make this course better!

For small fixes, click the "Edit Julia source code" button on top of a notebook. This will open the source on GitHub, where you can then click the "Edit this file" button (shown with a pencil icon). This will create a pull request on GitHub.

Alternatively, write me on ISIS or open an issue on GitHub and tell me what needs to be fixed. Ideas and feedback are also more than welcome.

What does the symbol ⁽⁺⁾ in some slide titles mean?

This symbol indicates optional–often more advanced–content that can be skipped.

Acknowledgements

The format of this website as well as the contents of this course are influenced by the following lectures:

Many thanks to

Last modified: December 18, 2023.
Website, code and notebooks are under MIT License © Adrian Hill.
Built with Franklin.jl, Pluto.jl and the Julia programming language.