Traffic light management based on reinforcement learning: Q-Learning and SARSA
DOI:
https://doi.org/10.15665/q4jsbg56Keywords:
Vehicle traffic control, SUMO, TraCI, Q-learning, SARSA, SUMO-RLAbstract
The accelerated growth of urban traffic has revealed the limitations of traditional traffic lights with fixed timing, which do not adapt to the variability of traffic patterns. In response, this work proposes the use of reinforcement learning algorithms, specifically Q-Learning and SARSA, to implement more efficient and dynamic traffic light control systems. Simulations are developed in the SUMO (Simulation of Urban Mobility) environment to compare the performance of these models against traditional traffic lights, with the average waiting time per vehicle as the main metric. The results allow us to evaluate the adaptability and superiority of intelligent models in most traffic scenarios, demonstrating their potential as an effective solution for managing vehicle flow in urban environments.
References
Ghanadbashi, S., y Golpayegani, F. (2021). "An Ontology-Based Intelligent Traffic Signal Control Model". IEEE, 2554-2561. https://doi.org/10.1109/itsc48978.2021.9564962
Jayanto, N. D., Satwika, I. P., Bayuwindra, A., Cahyono, R. T., y Husni, E. M. (2023). "Traffic Light Optimization using SUMO at the Samsat Intersection". IEEE, 1095-1100. https://doi.org/10.1109/smarttechcon57526.2023.10391623
Kekuda, A., Anirudh, R., y Krishnan, M. (2021). "Reinforcement Learning based Intelligent Traffic Signal Control using n-step SARSA". IEEE, 379-384. https://doi.org/10.1109/icais50930.2021.9395942
William, I., Kozhevnikov, S., Sontheimer, M., y Chou, S. (2024). “Enhancing Urban Traffic Management in Taipei: A Reinforcement Learning Approach”. IEEE, 1-6. https://doi.org/10.1109/scsp61506.2024.10549687
R. Zhang, A. Ishikawa, W. Wang, B. Striner y O. K. Tonguz, “Using Reinforcement Learning With Partial Vehicle Detection for Intelligent Traffic Signal Control”, IEEE Transactions on Intelligent Transportation Systems, vol. 22, núm. 1, pp. 404–415, enero de 2021. https://doi.org/10.1109/TITS.2019.2958859
Deepika, M. S., Mahajan, D., Vibhashree, S. H., Kiran, K., Shenoy, P. D., y Venugopal, K. R. (2024). "Analysing Mobility Patterns: A Comparative Study of SUMO RandomTrips and Duarouter". IEEE, 1-6. https://doi.org/10.1109/indiscon62179.2024.10744313
M. Noaeen, A. Naik, L. Goodman, J. Crebo, T. Abrar, Z. Shakeri Hossein Abad, A. L. C. Bazzan y B. Far, “Reinforcement learning in urban network traffic signal control: A systematic literature review”, Expert Systems with Applications, vol. 199, art. 116830, 2022. https://doi.org/10.1016/j.eswa.2022.116830
M. Miletić, E. Ivanjko, M. Gregurić y K. Kušić, “A review of reinforcement learning applications in adaptive traffic signal control”, IET Intelligent Transport Systems, vol. 16, págs. 1269–1285, 2022. https://doi.org/10.1049/itr2.12208
S. Park, E. Han, S. Park, H. Jeong y I. Yun, “Deep Q-network-based traffic signal control models”, PLOS ONE, vol. 16, núm. 9, art. e0256405, sep. 2021. https://doi.org/10.1371/journal.pone.0256405
H. Wei, G. Zheng, V. Gayah y Z. Li, “Recent advances in reinforcement learning for traffic signal control: A survey of models and evaluation”, ACM SIGKDD Explorations Newsletter, vol. 22, núm. 2, págs. 12–18, diciembre de 2020. https://doi.org/10.1145/3447556.3447565
A. R. M. Jamil, K. K. Ganguly y N. Nower, “Adaptive traffic signal control system using composite reward architecture based deep reinforcement learning”, IET Intelligent Transport Systems, vol. 14, págs. 2030–2041, 2020. https://doi.org/10.1049/iet-its.2020.0443
O. Mejia, R. Ceballos, R. Torres and J. Hoyos, "Adaptive Navigation System for an Autonomous Vehicle in a Goal-Oriented Environment," IEEE Latin America Transactions, vol. 23, no. 10, pp. 848-855, Oct. 2025, https://doi.org/10.1109/TLA.2025.11150628
A. Haydari and Y. Yılmaz, “Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey”, IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 1, pp. 11–32, Jan. 2022. https://doi.org/10.1109/TITS.2020.3008612
J. Clifton and E. Laber, “Q-Learning: Theory and Applications,” Annual Review of Statistics and Its Application, vol. 7, pp. 279–301, 2020. https://doi.org/10.1146/annurev-statistics-031219-041220
M. Corazza and A. Sangalli, "Q-Learning and SARSA: A Comparison between Two Intelligent Stochastic Control Approaches for Financial Trading," SSRN Electronic Journal, 2015. https://dx.doi.org/10.2139/ssrn.2617630
H. van Seijen, A. R. Mahmood, P. M. Pilarski, M. C. Machado, and R. S. Sutton, “True Online Temporal-Difference Learning,” *J. Mach. Learn. Res.*, vol. 17, no. 145, pp. 1–40, 2016. https://www.jmlr.org/papers/v17/15-599.html
Lucas N. Alegre, “SUMO-RL,” GitHub repository, 2019. [Online]. Available: https://github.com/LucasAlegre/sumo-rl
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Juan Pablo Hoyos, Jose Joaquin Vergara Cartagena, Leober Arturo Duran Raudales
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The authors to publish in this journal agree to the following conditions:
- The authors transfer the copyright and give the the journal first publication right of the work registered with Creative Commons Attribution License, which allows third parties to use the published work on the condition of always mentioning the authorship and first publication in this journal.
- The authors may perform other independent and additional contractual arrangements for the non-exclusive distribution of the version of the article published in this issue (E.g., Inclusion in an institutional repository or publication in a book), it must be indicated clearly that the work was first published in this journal.
- It allows and encourages the authors to publish their work online (eg institutional or personal pages) before and during the review and publication process. It can lead to productive exchanges and greater and faster dissemination of the published work (see The Effect of Open Access)
Instructions to fill out Certificate of Originality and Copyright Assignment
- Click here and get the forms of Certificate of Originality and Copyright Assignment .
- In each field to fill out, click and complete the corresponding information.
- Once the fields are filled out, at the end of the form copy your scanned signature or digital signature. Please adjust the size of the signature on the form.
- Finally, you can save them as pdf files and send them through the OJS platform as an attachment.
