Pattern Recognition

This is the collection of notebooks for the course FYS-3012 - Pattern Recognition, taught at UiT The Arctic University of Norway. You can view and execute the notebooks by clicking on the buttons below.

Slides are Marimo notebooks rendered with Marimo and hosted on GitHub Pages. Exercises are classic Jupyter notebooks that you open and work in locally — click the exercise badge to view the notebook on GitHub, then clone the repo (or download the file) to fill in your solutions.

📑 Content

1. Introduction to pattern recognition

   • What is pattern recognition?
   • Applications of pattern recognition
   • Practical information

   

2. Bayes decision theory

   • Motivation
   • Naïve Bayes assumption
   • Bayes classifier
   • Probablity of error
   • Dealing with risk

   

3. Density estimation I

   • Parametric density estimation
   • Likelihood function
   • Maximum-likelihood estimation
   • Non-parametric density estimation
   • Histogram approach
   • Parzen windowing

   

4. Density estimation II

   • K nearest neighbor
   • Mixture models

   

5. Linear classifiers I

   • Mean squared error (MSE) classifier
   • Least sum of squares classifier
   • Widrow-Hoff algorithm
   • Perceptron

   

6. Linear classifiers II

   • Support vector machines

   

7. Non-linear classifiers I

   • Multilayer Perceptron
   • Backpropagation
   • Gradient descent

   

8. Non-linear classifiers II

   • Activation functions
   • The vanishing gradient problem
   • Gradient descent with momentum
   • Multiclass classification with neural networks
   • Softmax function
   • Cross-entropy loss
   • Weight initialization

   

9. Non-linear classifiers III

   • Neural networks as representation learning
   • A vector perspective on backpropagation
   • Connecting neural networks and support vector machines

   

10. Non-linear classifiers IV

• Function approximation for non-linear classification
• Radial basis function (RBF) networks
• Prototypical learning

11. Non-linear classifiers V

• Kernel methods
• Mercer's theorem
• Kernels
• Non-linear SVMs

12. Data transformations and dimensionality reduction I

• Fisher discriminant analysis
• Principal component analysis (PCA)

13. Data transformations and dimensionality reduction II

• Kernel PCA
• Laplacian Eigenmaps

14. Ranking

• Ranking
• PageRank algorithm

15. Clustering I

• Sequential clustering

16. Clustering II

• Hierarchical

17. Clustering III

• K-means
• Spectral clustering

18. Clustering evaluation

💻 How to code locally

The repo is packaged with uv — a single static binary that installs Python and project dependencies in one step. It's orders of magnitude faster than a hand-rolled pip/venv setup (Rust-based, resolves against PyPI), produces a reproducible lockfile (uv.lock), and doesn't require a separate Python distribution or activation dance.

Install uv once (standalone installer), then from the repo root:

uv sync
uv run jupyter lab

uv sync reads pyproject.toml and installs every dependency (marimo, numpy, matplotlib, scipy, scikit-learn, plotly, pandas, ucimlrepo, ipykernel, ipywidgets) into a project-local virtualenv. Subsequent runs reuse that venv.

To run a single Marimo notebook (with the per-notebook deps declared in the PEP 723 header at the top of each .py file):

uv run marimo edit notebooks/01/bayes_decision_theory.py

uv reads the inline script metadata and resolves those deps into an ephemeral venv on first run, so you don't need to uv sync first.

📝 How to work with the exercises

Most lectures ship with a companion exercise notebook named *_exercise.ipynb (the Content block above links to it from each lecture). Unlike the slides — which are Marimo WASM apps you just click — exercises are classic Jupyter notebooks that you open and edit locally.

Workflow:

1. From the repo root, run uv run jupyter lab (after the one-time uv sync from the "How to code locally" section). This opens the Jupyter Lab UI in your browser with the whole repo available on the left.
2. Open the exercise notebook from the Content block — e.g. notebooks/01/bayes_decision_theory_exercise.ipynb.
3. Each notebook starts with a # <Title> — programming exercises heading and a short ## Programming exercises intro cell, followed by alternating question / starter-code pairs.
4. Add your own code cells below each starter — there are no # TODO placeholders; the question text and the starter cell are the only signal for where to work.
5. Save your progress in your working copy of the repo. To get a fresh copy of an exercise, re-download the file from the GitHub link in the Content block above (or git checkout -- notebooks/<dir>/<name>_exercise.ipynb if you have the repo cloned).

Datasets used:

• Synthetic samples generated inline (Gaussians, uniform, XOR, cosine, hand-picked 2-D points).
• sklearn.datasets.make_moons and make_swiss_roll (no download required).
• sklearn.datasets.load_wine (no download required).
• The Iris dataset via ucimlrepo.fetch_ucirepo(id=53) — this does require internet on first run; cached afterwards.

Figure stability:

A few notebooks call np.random.seed(0) at the top so figures look the same every time you re-render. If you change the seed or remove it, figures will reshuffle on each run.

Theoretical exercises:

Not every assignment is a programming exercise. Theoretical exercises exist but, due to copyright on the course textbook, are not published in this public repository. They are distributed separately through a private course repository (see your course portal for access).

🎞 Slides

A growing set of lectures is published as Marimo notebooks running entirely in the browser via WebAssembly:

• Introduction to pattern recognition (link)
• Bayes decision theory (slides) · exercises
• Density estimation I (slides) · exercises
• Linear classifiers I (slides) · exercises
• Linear classifiers II (slides) · exercises
• Non-linear classifiers I (slides) · exercises
• Non-linear classifiers II (slides) · exercises
• Non-linear classifiers III (slides) · exercises
• Non-linear classifiers IV (slides) · exercises
• Non-linear classifiers V (slides) · exercises
• Data transformations and dimensionality reduction I (slides) · exercises
• Data transformations and dimensionality reduction II (slides) · exercises
• Ranking (slides)
• Clustering I (slides) · exercises
• Clustering II (slides) · exercises
• Clustering III (slides) · exercises
• Clustering evaluation (slides) · exercises

Each .py notebook is content-equivalent to its .ipynb counterpart (same text, same figures) and is exported with marimo export html-wasm --mode run during the same CI run that builds the other decks.

To run any of them locally in edit mode (live code, file watchers, variable explorer):

uv run marimo edit notebooks/00/intro.py
uv run marimo edit notebooks/01/bayes_decision_theory.py
uv run marimo edit notebooks/02/density_estimation.py
uv run marimo edit notebooks/03/linear_classifiers.py
uv run marimo edit notebooks/03/support_vector_machine.py
uv run marimo edit notebooks/04/neural_networks_1.py
uv run marimo edit notebooks/04/neural_networks_2.py
uv run marimo edit notebooks/04/neural_networks_3.py
uv run marimo edit notebooks/05/rbf_networks.py
uv run marimo edit notebooks/06/kernel_methods.py
uv run marimo edit notebooks/07/linear_dim_reduction.py
uv run marimo edit notebooks/08/non_linear_dim_reduction.py
uv run marimo edit notebooks/09/ranking.py
uv run marimo edit notebooks/10/seq_clustering.py
uv run marimo edit notebooks/11/hier_clustering.py
uv run marimo edit notebooks/12/clustering.py
uv run marimo edit notebooks/13/clustering_eval.py

The Marimo notebooks trade the slide-stepping UX for a reactive notebook UX — for example, in the Bayes lecture the slider for the overlapping-Gaussians figure is a real mo.ui.slider instead of a baked-in Plotly animation, and the bar-chart example reacts to a word-selection dropdown.

📝 Citation

If you are using this material in your courses or in your research, please consider citing it as follows:

@misc{wick2024prbook,
  author       = {Kristoffer Wickstrøm},
  title        = {Pattern Recognition Course},
  year         = {2025},
  howpublished = {Online},
  url          = {https://github.com/Wickstrom/pattern-recognition-handbook}
}

This repository is heavily based on Time Series Analysis with Python by my collegue at UiT Filippo Maria Bianchi. If you find my repository useful, please head over there and give him a star.

@misc{bianchi2024tsbook,
  author       = {Filippo Maria Bianchi},
  title        = {Time Series Analysis with Python},
  year         = {2024},
  howpublished = {Online},
  url          = {https://github.com/FilippoMB/python-time-series-handbook}
}