← All work
Machine Learning

Thesis: Distributed development of AI models

My bachelor's thesis focused on distributed machine learning. For this thesis I received the Dean's Award.

CategoryMachine Learning
Updated21. 10. 2025
Repositoryhttps://gitlab.fit.cvut.cz/ledlmart/bachelor-project
Stack
AIMachine LearningDistributed TrainingPyTorchFlask API

Overview

My bachelor's thesis examined how parts of machine learning development, including training, hyperparameter tuning, and inference, can be distributed between several machines. The practical work focused on collaborative hyperparameter tuning.

Thesis Instructions

The development of machine learning models is predominantly executed on single machines with multiple processing units. This work explores the potential for distributing various stages of the development process, such as training, hyperparameter tuning, and inference, across multiple machines connected over the internet. Emphasis is placed on hyperparameter optimization, evaluating methods and algorithms.

The main objectives of the thesis are:

  1. Study the use of distributed computing in machine learning, including model training and inference, with a primary focus on hyperparameter tuning.
  2. Analyze existing hyperparameter optimization methods and algorithms, with consideration of their applicability in distributed environments.
  3. Design and implement an API for collaborative hyperparameter tuning across multiple machines.
  4. Demonstrate the practical use of the API by connecting at least three machines to collaboratively perform hyperparameter tuning for a machine learning model.

Thesis Project

I designed and implemented a system in which a central server coordinates hyperparameter tuning. Connected client machines train and evaluate models with different hyperparameter configurations.

The server provides a REST API built with Flask. WebSockets coordinate the workers during training and allow the server to react when a client disconnects.

Each client requests a configuration, trains the model with those parameters, and returns the result. The server collects the results for analysis and optimization.

Results

I tested the system with three machines. Hyperparameter tuning with all three was approximately 2.5 times faster than with a single machine. The server also handled clients with unstable connections by reconnecting them or assigning their work elsewhere.

Technologies used

  • Python
  • Flask (for building the RESTful API)
  • WebSockets (for real-time communication)
  • Scikit-learn (for machine learning model training)
← All work