Federated Learning System

The Impact of Artificial Client Data Generation Parameters on the Performance of Federated Learning Systems

Overview

This repository contains the implementation developed as part of a master's thesis investigating how artificial client data generation and data distribution parameters affect the performance of federated learning systems.

The project provides a configurable federated learning pipeline for creating heterogeneous client datasets, training image-classification models across multiple federated clients, and evaluating the resulting global model.

The system is implemented in Python using the Flower federated learning framework and PyTorch.

Research status: This repository contains research code developed for thesis experiments. It is intended for research, reproducibility, and portfolio purposes rather than production deployment.

---

Research Objective

Federated learning performance can depend strongly on how data is distributed across participating clients.

This project investigates the effect of different artificial client-data generation approaches, including:

• random client data distribution
• quantity-skewed distributions
• feature-based client splitting
• image manipulation
• image preprocessing and augmentation

The resulting federated models can then be compared using common classification metrics.

---

System Workflow

The main workflow is:

Input dataset
     │
     ▼
Data loading
     │
     ▼
Client data splitting
     │
     ▼
Optional image manipulation
     │
     ▼
Optional image preprocessing
     │
     ▼
Client datasets
     │
     ▼
Federated training with Flower
     │
     ├── Client 0
     ├── Client 1
     ├── ...
     └── Client N
     │
     ▼
FedAvg aggregation
     │
     ▼
Global model
     │
     ▼
Server-side benchmark evaluation
     │
     ▼
Experiment metrics

---

Main Components

Data Loading and Client Generation

The data preparation code is located in:

fl_base_code/data_loader/

The codebase contains implementations for three client splitting approaches:

• random splitting
• quantity-skew splitting using a Dirichlet distribution
• feature-based splitting

The selected splitting configuration is defined in config.yaml.

Image Manipulation

The data manipulation module contains configurable image transformations including:

• contrast adjustment
• brightness adjustment
• white-balance adjustment
• Gaussian noise
• edge filtering
• sharpening and blurring

Manipulation parameters can be configured in config.yaml.

Image Preprocessing

The preprocessing module includes:

• horizontal flipping
• image rotation

Both transformations can be applied to client image datasets.

Federated Learning

Federated learning is implemented using the Flower framework.

The system consists of:

• a Flower server
• multiple Flower clients
• local PyTorch model training
• FedAvg aggregation
• configurable federated training rounds
• configurable number of clients

The run_federated_learning.py script prepares the data and starts the server and client processes.

Models

The model configuration provides two model categories:

• custom_cnn
• transformer

The custom CNN architecture is configurable through parameters such as:

• convolutional layers
• pooling layers
• fully connected layers
• activation functions
• dropout

The transformer configuration includes torchvision model options such as:

• vit_b_16
• vit_l_16
• swin_b
• swin_l

The configuration also allows pretrained weights to be enabled for transformer models.

---

Evaluation

The global model is evaluated on a benchmark dataset on the server side.

The evaluation code calculates:

• loss
• accuracy
• macro precision
• macro recall
• macro F1-score

The evaluation also stores labels and model predictions.

Experiment metrics and the experiment configuration are saved as JSON output inside the experiments/ directory.

---

Repository Structure

Federated-Learning-System/
│
├── data-cleaner-and_exploration/
│   ├── data_cleaner.ipynb
│   └── data_exploration.ipynb
│
├── final_data/
│   └── .gitkeep
│
├── fl_base_code/
│   │
│   ├── benchmark_data/
│   │
│   ├── client/
│   │   ├── __init__.py
│   │   ├── client.py
│   │   ├── model_creator.py
│   │   └── utils.py
│   │
│   ├── data_loader/
│   │   ├── __init__.py
│   │   ├── data_manipulation.py
│   │   ├── data_preprocessing.py
│   │   ├── data_splitter.py
│   │   ├── main.py
│   │   └── utils.py
│   │
│   ├── experiments/
│   │   ├── FairnessAnalysis.ipynb
│   │   ├── MultiClient.ipynb
│   │   ├── client_visualizations.ipynb
│   │   ├── multiple_experiment_analysis.ipynb
│   │   ├── splitcompare.ipynb
│   │   └── visualizations.ipynb
│   │
│   ├── preprocessed_data/
│   │
│   ├── server/
│   │   ├── __init__.py
│   │   ├── MetricSaver.py
│   │   ├── model_creator.py
│   │   ├── server.py
│   │   └── utils.py
│   │
│   ├── __init__.py
│   ├── config.yaml
│   └── run_federated_learning.py
│
├── .gitattributes
├── .gitignore
├── README.md
└── requirements.txt

---

Dependencies

The Python dependencies currently defined in requirements.txt are:

torch~=2.3.1+cu121
pandas~=2.2.2
yaml~=0.2.5
pyyaml~=6.0.1
pillow~=10.2.0
scikit-learn~=1.5.2
torchvision~=0.18.1+cu121
flwr~=1.8.0
numpy~=1.26.4
timm~=1.0.11
opencv-python~=4.10.0.84

---

Installation

Clone the repository:

git clone https://github.com/shuvanon/Federated-Learning-System.git
cd Federated-Learning-System

Create a Python virtual environment:

python -m venv .venv

Activate the environment.

Windows

.venv\Scripts\activate

Linux/macOS

source .venv/bin/activate

Install the dependencies:

pip install -r requirements.txt

The current requirements include CUDA 12.1 builds of PyTorch and torchvision.

---

Dataset Configuration

The repository does not include the source image dataset inside final_data/; the directory currently contains only .gitkeep.

Dataset paths are configured in:

fl_base_code/config.yaml

The main data configuration contains:

data:
  csv_file: ...
  img_dir: ...
  preprocessed_data_dir: ...
  save_split: ...
  clean_data_before_run: ...
  batch_size: ...
  train_split: ...
  label_column_index: ...

The current committed configuration contains local Windows paths, so these paths need to match the dataset location on the machine running the experiment.

---

Benchmark Configuration

Benchmark settings are also stored in config.yaml:

benchmark:
  csv_file: ...
  img_dir: ...
  benchmark_percentage: ...
  save_benchmark: ...

The server loads this benchmark dataset and evaluates the global model during federated training.

---

Experiment Configuration

The main experiment configuration file is:

fl_base_code/config.yaml

Experiment Name

experiment_name: "Baseline"

Client Split Selection

The configuration contains:

use_split_strategy: "random"

and configuration entries for:

random
quantity_skew
feature_based

Image Manipulation

The top-level manipulation selection is configured through:

use_manipulation_technique: "none"

The configuration contains parameters for:

contrast
brightness
white_balance
gaussian_noise
edge_filter
sharpening_blurring

Preprocessing

The preprocessing selection is configured through:

use_preprocessing_technique: "none"

The preprocessing configuration contains:

fixed_flip
rotation

Model

The model type is selected through:

model:
  type: "custom_cnn"

The configuration contains parameters for both:

custom_cnn
transformer

Training

Local model training parameters include:

training:
  epochs: 100
  learning_rate: 0.0001

Federated Network

The configuration also contains:

• server address
• client address
• number of federated rounds
• number of clients

---

Running the Federated Learning Pipeline

After configuring the dataset and experiment settings, run the pipeline from the repository root:

python fl_base_code/run_federated_learning.py

The launcher:

1. executes the data loader,
2. prepares client data,
3. starts the Flower server,
4. starts the configured client processes,
5. waits for the client processes to complete.

---

Experiment Results and Analysis

Experiment output is stored under:

fl_base_code/experiments/

The repository also contains the following analysis notebooks:

FairnessAnalysis.ipynb
MultiClient.ipynb
client_visualizations.ipynb
multiple_experiment_analysis.ipynb
splitcompare.ipynb
visualizations.ipynb

These notebooks are used for visualising and analysing client distributions and federated learning experiment results.

---

Data Exploration

The repository contains separate notebooks for dataset cleaning and exploration:

data-cleaner-and_exploration/
├── data_cleaner.ipynb
└── data_exploration.ipynb

---

Author

Shuvanon Razik

Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)

Email: shuvanon.razik@fau.de