Rafael Fernandes Cunha

I am a Lecturer in Artificial Intelligence (AI) and a PhD Candidate at the University of Groningen, the Netherlands, with a research focus on reinforcement learning (RL).

My present research is centered around improving data efficiency in fully cooperative multiagent RL problems through the application of transfer learning techniques. I am particularly interested in algorithm-level investigations and convergence analysis in this area. In a previous project, we leveraged RL to minimize the fuel consumption of a one-dimensional vehicle platoon.

During my master's studies in Electrical Engineering, with a focus on control systems at UNICAMP, I acquired experience in the mathematical modeling of dynamical systems and the resolution of convex optimization problems. This groundwork has helped me understand RL problems on the algorithmic level and discern their connection to output feedback control type problems, for which there are established mathematical tools for analysis.

Here’s a broad overview of my current research interests:

Deep Reinforcement Learning
Multiagent Reinforcement Learning
Transfer Learning
Hierarchical Reinforcement Learning

Check out here the list of open projects that you can enroll in for your bachelor’s or master’s thesis.

News

Jul 13, 2023	New paper accepted on the journal Transactions on Intelligent Transportation Systems!
Apr 1, 2023	Started new position as a Lecturer in Artificial Intelligence at the University of Groningen.
May 12, 2022	I attended the Easter Europe Machine Learning (EEML) summer school in Vilnius, Lithuania.

Selected publications

2023

TITS

Fuel-Efficient Switching Control for Platooning Systems With Deep Reinforcement Learning

Tiago R Goncalves, Rafael F Cunha, Vineeth S Varma, and 1 more author

IEEE Transactions on Intelligent Transportation Systems, 2023

Abs

The wide appeal of fuel-efficient transport solutions is constantly increasing due to the major impact of the transportation industry on the environment. Platooning systems represent a relatively simple approach in terms of deployment toward fuel-efficient solutions. This paper addresses the reduction of fuel consumption in platooning systems attainable by dynamically switching between two control policies: Adaptive Cruise Control (ACC) and Cooperative Adaptive Cruise Control (CACC). The switching rule is dictated by a Deep Reinforcement Learning (DRL) technique to overcome unpredictable platoon disturbances and to learn appropriate transient shift times while maximizing fuel efficiency. However, due to safety and convergence issues of DRL, our algorithm establishes transition times and minimum periods of operation of ACC and CACC controllers instead of directly controlling vehicles. Numerical experiments show that the DRL agent outperforms both static ACC and CACC versions and the threshold logic control in terms of fuel efficiency while also being robust to perturbations and satisfying safety requirements.

2021

ECC
On imitation dynamics in population games with Markov switching

Rafael F Cunha, Lorenzo Zino, and Ming Cao

In 2021 European Control Conference (ECC), 2021

Abs Bib

Imitation dynamics in population games are a class of evolutionary game-theoretic models, widely used to study decision-making processes in social groups. Different from other models, imitation dynamics allow players to have minimal information on the structure of the game they are playing, and are thus suitable for many applications, including traffic management, marketing, and disease control. In this work, we study a general case of imitation dynamics where the structure of the game and the imitation mechanisms change in time due to external factors, such as weather conditions or social trends. These changes are modeled using a continuous-time Markov jump process. We present tools to identify the dominant strategy that emerges from the dynamics through methodological analysis of the function parameters. Numerical simulations are provided to support our theoretical findings.
@inproceedings{cunha2021imitation, title = {On imitation dynamics in population games with Markov switching}, author = {Cunha, Rafael F and Zino, Lorenzo and Cao, Ming}, booktitle = {2021 European Control Conference (ECC)}, pages = {722--727}, year = {2021}, organization = {IEEE}, }

2019

IJSS
Robust partial sampled-data state feedback control of Markov jump linear systems

Rafael F Cunha, Gabriela W Gabriel, and José C Geromel

International Journal of Systems Science, 2019

Abs Bib

This paper proposes new conditions for the design of a robust partial sampled-data state feedback control law for Markov jump linear systems (MJLS). Although, as usual, the control structure depends on the Markov mode, only the state variable is sampled in order to cope with a specific network control structure. For analysis, an equivalent hybrid system is proposed and a two-point boundary value problem (TPBVP) that ensures minimum Hoo or H2 cost is defined. For control synthesis, it is rewritten as a convex set of sufficient conditions leading to minimum guaranteed cost of the mentioned performance classes. The optimality conditions are expressed through differential linear matrix inequalities (DLMIs), a useful mathematical device that can be handled by means of any available LMI solver. Examples are included for illustration.
@article{cunha2019robust, title = {Robust partial sampled-data state feedback control of Markov jump linear systems}, author = {Cunha, Rafael F and Gabriel, Gabriela W and Geromel, Jos{\'e} C}, journal = {International Journal of Systems Science}, volume = {50}, number = {11}, pages = {2142--2152}, year = {2019}, publisher = {Taylor \& Francis} }