Internship M2 in Engineering (H-F)

Title : Bike sharing system rebalancing by reinforcement learning algorithms

Keywords:  bike sharing system, bike repositionning,  reinforcement learning

Description

This internship project focuses on a specific component of a broader initiative to improve the dynamic rebalancing of bike-sharing systems [1,2,3]. The problem is addressed in two stages. Based on data at the station and travel needs at a given moment t, the number of bicycles available and needed will be predicted at time t+1. Points of origin and destination can be grouped together to improve the performance of spatio-temporal calculations of flow gradients from the micro scale at the station to the city scale [3,4].

This approach will thus make it possible to predict more quickly the number of bicycles used on the network and at stations in order to obtain a quasi-dynamic description of the system [6,7]. In a second stage, using these new estimated input data, real-time rebalancing is deployed. A reinforcement learning algorithm is then used and trained to propose and refine the dynamic redistribution strategy for bicycles [8,9]. The advantage of this approach lies in its ability to adapt to contextual disturbances and to resolve issues on a large scale. However, this performance comes at a cost and is detrimental to ensuring the most optimised solution is achieved.

This internship will focus on the first stage of the project, which concerns the prediction and modeling of bicycle availability and demand dynamics. The objective will be to design and evaluate predictive models capable of capturing both spatial and temporal dependencies in the bikeshare system. The intern will explore and compare different machine learning approaches, such as time series forecasting, graph neural networks, or spatio-temporal convolutional architectures, to estimate short-term variations in bicycle flows at the station and network levels by using clustering, for example [10].

The performance of the models will be evaluated against real operational data, and the results will serve as input for the reinforcement learning framework used in the second phase of the project .

Depending on the progress and interests of the intern, additional exploration may include studying the integration of uncertainty quantification in predictions or the use of online learning methods to adapt models in real time as new data become available.

The internship will provide the opportunity to gain hands-on experience in data science, spatio-temporal modeling, and urban mobility systems, while contributing to an innovative research topic with potential real-world applications.

Objectives :

O1. Develop predictive models to estimate short-term bicycle availability and demand at both the station and network levels using spatio-temporal data.

O2. Analyze and preprocess heterogeneous datasets, including trip records, station metadata, weather conditions, and temporal factors, to create robust inputs for modeling.

O3. Implement and compare different machine learning approaches (e.g., time series forecasting, graph neural networks, spatio-temporal models) to capture flow dynamics in the bikeshare system.

O4. Evaluate the performance and scalability of predictive algorithms under realistic conditions, using metrics relevant to operational decision-making in mobility systems.

O5. Provide data-driven inputs for the reinforcement learning module, enabling the development of adaptive and real-time rebalancing strategies in the second phase of the project.

O6. Integrate uncertainty quantification to assess the confidence of predictions and their impact on rebalancing decisions.

O7. Explore online or incremental learning techniques to enable continuous model adaptation as new data streams become available.

Expected scientific/ technical production

The internship is expected to lead to both methodological and applied outcomes, including:

O1. A cleaned and structured dataset integrating multimodal information (trip data, station metadata, weather, temporal and spatial context) suitable for spatio-temporal modeling.

O2 .A set of predictive models (baseline statistical models and advanced machine learning architectures) for short-term demand and availability forecasting in bikeshare systems. 

O3. A comparative performance analysis report, detailing the accuracy, robustness, and computational efficiency of the different modeling approaches.  

O4. A prototype or simulation tool demonstrating the integration of prediction outputs into a reinforcement learning environment for dynamic rebalancing.  

O5.Technical documentation and reproducible code, following open science practices, to facilitate future extensions and integration into the larger project framework. 

O6. Preparation of a scientific report or publication draft, presenting the methodology, results, and implications for large-scale mobility optimization.

Lab presentation

CESI LINEACT (UR 7527), Laboratory for Digital Innovation for Businesses and Learning to Support the Competitiveness of Territories, anticipates and accompanies the technological mutations of sectors and services related to industry and construction. The historical proximity of CESI with companies is a determining element for our research activities. It has led us to focus our efforts on applied research close to companies and in partnership with them. A human-centered approach coupled with the use of technologies, as well as territorial networking and links with training, have enabled the construction of cross-cutting research; it puts humans, their needs and their uses, at the center of its issues and addresses the technological angle through these contributions.

Its research is organized according to two interdisciplinary scientific teams and several application areas.

• Team 1 "Learning and Innovating" mainly concerns Cognitive Sciences, Social Sciences and Management Sciences, Training Techniques and those of Innovation. The main scientific objectives are the understanding of the effects of the environment, and more particularly of situations instrumented by technical objects (platforms, prototyping workshops, immersive systems...) on learning, creativity and innovation processes.

• Team 2 "Engineering and Digital Tools" mainly concerns Digital Sciences and Engineering. The main scientific objectives focus on modeling, simulation, optimization and data analysis of cyber physical systems. Research work also focuses on decision support tools and on the study of human-system interactions in particular through digital twins coupled with virtual or augmented environments.

These two teams develop and cross their research in application areas such as

• Industry 5.0,

• Construction 4.0 and Sustainable City,

• Digital Services.

Areas supported by research platforms, mainly that in Rouen dedicated to Factory 5.0 and those in Nanterre dedicated to Factory 5.0 and Construction 4.0.

Link to the research axes of the research team involved

CESI Lineact Research Thematic: Management of Multimodal Transport Systems (GSTM)

Bibliography :

[1] Z. Jiang, C. Lei, and Y. Ouyang, “Optimal investment and management of shared bikes in a competitive market,” Transportation Research Part B: Methodological, vol. 135, pp. 143–155, May 2020, doi:https://doi.org/10.1016/j.trb.2020.03.007.

[2] M. Dell’Amico, M. Iori, S. Novellani, and A. Subramanian, “The Bike sharing Rebalancing Problem with Stochastic Demands,” Transportation Research Part B: Methodological, vol. 118, pp. 362–380, Dec. 2018, doi: https://doi.org/10.1016/j.trb.2018.10.015.

[3] C. M. Vallez, M. Castro, and D. Contreras, “Challenges and Opportunities in Dock-Based Bike-Sharing Rebalancing: A Systematic Review,” Sustainability, vol. 13, no. 4, p. 1829, Feb. 2021, doi: https://doi.org/10.3390/su13041829.

[4] Randriamanamihaga, A. N., Côme, E., Oukhellou, L., & Govaert, G. (2014). Clustering the Vélib׳ dynamic Origin/Destination flows using a family of Poisson mixture models. Neurocomputing, 141, 124-138

[5] Yunlong Feng, Roberta Costa Affonso, Marc Zolghadri, Analysis of bike sharing system by clustering: the Vélib’ case, IFAC-PapersOnLine, Volume 50, Issue 1,2017, Pages 12422-12427, ISSN 2405-8963, doi.org/10.1016/j.ifacol.2017.08.2430.

[6] Lei Lin, Zhengbing He, Srinivas Peeta; Predicting station-level hourly demand in a large-scale bike-sharing network: A graph convolutional neural network approach, Transportation Research Part C: Emerging Technologies,Volume 97,2018,Pages 258-276,ISSN 0968-090X, doi.org/10.1016/j.trc.2018.10.011.

[7] Wang, Xudong & Cheng, Zhanhong & Trépanier, Martin & Sun, Lijun. (2020). Modeling bike-sharing demand using a regression model with spatially varying coefficients.

[8] Liang, Jiaqi & Liu, Defeng & Jena, Sanjay & Lodi, Andrea & Vidal, Thibaut. (2024). Dual Policy Reinforcement Learning for Real-time Rebalancing in Bike-sharing Systems. 10.48550/arXiv.2406.00868.

Contacts and Application

All qualified individuals are encouraged to apply by sending the following documents (

in one file PDF) :

• A curriculum vitae (CV) detailing academic background, technical skills, and relevant experiences
• A motivation letter (maximum one page) expressing interest in the internship and alignment with the research topic
• Academic transcripts (relevé de notes) from the current and previous years of study
• Any reference letters or supporting documents that may strengthen the application

The Candidat’es Profile

The candidate should be a Master’s student (M2) or in the final year of an Engineering School program, with a background in Computational Mechanics, Applied Mathematics, or Data Science, and an interest in all three fields.

She/He should have some knowledge and experience in a number of the following topics:

• Numerical modeling and simulation of physical or dynamical systems

• Machine learning or statistical data analysis

• Time series forecasting and spatio-temporal modeling

• Optimization and/or reinforcement learning methods

• Programming skills in Python (preferred), including libraries such as NumPy, Pandas, PyTorch, or TensorFlow

• Data visualization and exploratory data analysis

• Familiarity with version control tools (e.g., Git) and collaborative coding practices

• Good written and oral communication skills in English