RESEARCH
LEONARDO STELLA - DLC Research Group
Dynamic Learning and Control (DLC) Research Group
A list of areas of interest are listed below. The focus of the group is at the intersection of game theory, reinforcement learning, and control. Active projects as well as past projects are listed below.
Game Theory and Control
We study complex dynamics arising in various games, including cooperative and competitive games, evolutionary games, and mean-field games. Our long-term goal is to develop strategies for large multi-agent systems with convergence, stability, and robustness guarantees, with a focus on learning. Applications span from opinion dynamics, network design, or financial markets, to bio-inspired systems such as decision-making in honeybees.
Multi-Agent Reinforcement Learning
Multi-agent reinforcement learning is the most prominent paradigm for learning in systems with many agents. However, there are still open problems in learning policies in a distributed manner, including scalability, safety and efficiency with a focus on theoretical guarantees. Our contribution to address these issues is threefold: we create algorithms that are robust and adaptable, develop theoretical frameworks where the communication between agents, and deploy these approaches in cyber-physical systems such as robotics swarms.
ML in Materials Science
Our goal is to design physics-guided and explainable systems to tackle complex problems in materials science. In particular, our contribution is twofold: reinforcement learning frameworks for the optimisation of process parameters in additive manufacturing where data are used to experimentally validate our approach; and physics-informed neural networks for the characterisation of the manufacturing processes.
PROJECTS
A list of active projects and collaborations that I am working on.
2024 Process Parameter Optimisation for Predictive Manufacturing and Design via Multi-Agent Reinforcement Learning
The goal of this project is look at multi-agent reinforcement learning (MARL) for the optimisation of process parameters for predictive manufacturing. This project is in collaboration with Prof Moataz Attallah.
Role: Beneficiary, Principal Investigator.
Co-Investigator: Prof Moataz Attallah.
PhD Students: Ms Ilaria Lagalante, Mr Francesco Careri.
Research Assistants: Mr Ahmed Faizan, Ms Nicoletta Lambrou, Mr Sam Robbins.
2024 Resource-Constrained MARL for UAV Swarms: A Testbed for Collective Decision-Making
The aim of this project is to investigate multi-agent reinforcement learning (MARL) approaches in the context of swarm robotics under constrained data sharing. Resources and communication between the agents are designed to be kept to a minimum (e.g., sending only state/action pairs and rewards), taking inspiration by the minimal communication in honeybees. In particular, this projects will constitute a testbed for the theoretical development of model-free MARL and will underpin blue skies research through the use of the proposed robots.
Role: Beneficiary, Principal Investigator.
PhD Student: Mr Ziyue Chu.
2023-2026 HuMaT: Reinforcement Learning for Human-Agent Teaming
Reinforcement Learning (RL) has achieved exceptional success in recent years. Recently, a prominent area of research involves the extension to multi-agent reinforcement learning (MARL). The focus of this project is on building reliable and robust human-agent teams. In particular, the aims are: i) explainability – of the agents to the human, and of the human to the agents; ii) team behaviour in multi-human and multi-agent teams.
The goal of this project is look at multi-agent reinforcement learning (MARL) to improve and automate human-agent teaming. This project is in collaboration with the Alan Turing Institute and ARL.
Role: Co-Investigator.
Principal Investigator: Prof Christopher Baber and Prof Andrew Howes.
Research Associate: Dr Aditya Acharya.
PhD Student: Mr Aju Ani Justus.
PREVIOUS PROJECTS
A list of previous projects that I worked on in past years or during my PhD and undergraduate/postgraduate studies.
2021 ECR Development Fund: COVID-19 Virus Propagation Digital Twin
The main goal of this project is to create a digital twin replica of the first floor of the University of Derby as a general university setting for COVID-19 propagation.
The project extends the previous application built for the 2020 ECR, making use of Unity3D to generate the digital twin (virtual reality).
Role: Beneficiary, Principal Investigator.
Research Supervisor: Roisin Hunt (Computer Games Modelling and Animation).
Research Assistants: Petar Cacik, Will Kitchen, Ryan Skull.
The main goal of this project is to analyse the official data and the current research on the COVID-19 pandemic in order to assess the impact of asymptomatic individuals.
We use the official data to estimate the parameters of an epidemiological model that has been formulated to predict the trends of this virus and and assess different containment strategies.
Finally, virtual reality technology is used to provide a visual immersive environment, building on previous research on crowd dynamics and safety compliance with the government policies.
Role: Beneficiary, Principal Investigator.
Research Assistants: Matej Kapinaj, Diego Marti Mason, Alejandro Pinel Martínez.
Innovate UK: Creating Information Models for a Virtual Reality Training Solution
The main goal of this project is to automate the creation of information models to support virtual reality training solutions. The KTP research associate worked in partnership with Bloc Digital, a leading company in XR solutions located in Derby.
Role: Innovate UK KTP Supervisor.
KTP Research Associate: Dr. Claire Palmer.
ProSFeT (Freight Logistics in Urban Context)
The ProSFeT project (http://www.prosfet.eu/PROSFET/) aims at improving logistics operations and local authority planning within the urban freight framework. The main objectives include a review of the urban freight transport in Europe, the utilisation of stakeholders' engagement methods and the use of decision support tools.
My role in the project is to develop a decision tool (.Net/C#) which provides additional information to stakeholders in order to support the feasibility of urban consolidation centres (UCCs) in urban context. Specifically, the tool extends a traditional planner to include costs in terms of personnel, vehicles, orders, distances, etc. A key aspect captured by the tool is the calculation of the advantages of using a UCC in terms of CO2 consumption vs a direct shipping approach.
Decision makers can use the tool to test a set of different scenarios, by placing the UCC according to different sets of rules. By running the planner, the planning results can be used to estimate costs of the UCC in the specified scenario and to obtain some metrics on how to optimise logistics in the considered urban context.
SUPERFLUIDITY (5G)
The SUPERFLUIDITY project (http://superfluidity.eu/) aims at deploying 5G in Europe. As in physics a matter is in the superfluidity state when it behaves like a fluid with zero viscosity, so the main objectives of the project are to achieve the same in the Internet. The key points include fast instantiation of services, easy use of these services independently of where the user is in the network (whether it is in the core, aggregation or edge), and to shift them to new locations.
My role in the project was to study virtualisation through a set of container deployer softwares, like Kubernetes (https://kubernetes.io/) and nomad (https://www.nomadproject.io/). The aim was to instantiate and destroy virtual machines through both orchestrators after designing a method to interact with them on a linux machine. The performances of the proposed methods were therefore tested and compared for results.
SUPERFLUIDITY is aimed at providing "a converged cloud-based 5G concept that will enable innovative use cases in the mobile edge, empower new business models, and reduce investment and operational costs".