Recently, the advances in state-of-the-art Computer Vision techniques lead to an increase of AI applications in many industrial fields. We refer to Computer Vision as a set of image-based methodologies designed to perform tasks such as object classification and detection. Mastering these methods allow enterprises in the industry to automate routine tasks, reducing the need for human intervention and ultimately improving standardization and scalability. Among potential applications, image-based intelligent algorithms have a great potential in the predictive maintenance field, allowing maintenance costs and machine downtime reduction and therefore leading to more efficient processes.
The customer is a public company in charge of management, administration and maintenance of all the railway infrastructure in Italy. As well as giving work to about 25.000 people, it owns roughly 16.000 railroad kilometers and 2.200 train stations, distributed among the whole Italian territory.
Following the principles of Industry 4.0, the customer is committed to invest in its IT infrastructure in order to automate low-skill and routinary tasks. Specifically, the first goal is to automate the Asset Management activity, which is about tracking all relevant assets installed on the territory. Tracking means being able to know both the geographical location and the nature of an asset. In the past this task was carried out manually for all 16.000 km of railroads:
- 1. Images of the surroundings of the train were captured by a diagnostic train and periodically inserted into a central database, alongside with GPS coordinates of the image.
- 2. Later on, on-field operators had to manually review each acquisition, identifying relevant assets and updating the inventories.
- 3. They were also responsible for the identification of potential anomalies on the train’s track, such as clay, waste and soil on the rails and the ballad. In fact, these refluxes may potentially obstruct the rails, and make the train’s passage more difficult, and it is part of the customer’s duties to fix them.
An additional complication is that assets are periodically moved depending on business needs, therefore the Asset Management activity must be performed multiple times within a year.
The client’s goal was therefore to speed up and scale the Asset Management and Anomaly Detection activity through AI and automation, reducing the need for on-field operators and ultimately reaching a higher accuracy.
Our solution is made of two image-based algorithms, implemented in the customer’s server. The first one is responsible for detecting and localizing 22 asset categories such as railroad crossing, fences, railroad traffic lights etc., while the second identifies anomalies on the ballad and sends real-time notifications.
The diagnostic train, made of a 360° camera and a GPS-based location module that allows to match each photo with its acquisition coordinates, processes roughly 120.000 photos every 24 hours, corresponding to 40 km of railroads.The acquisitions are sent to the customer’s server, where our predictive model processes them. The architecture used for the model is a Convolutional Neural Network, the state-of-the-art method for object classification in Computer Vision.
Our algorithm looks at all the newly acquired images and predicts the presence of an asset inside of it. The model’s prediction is combined with GPS information about the railroad track to accurately localize the identified object. Finally, the asset inventory is automatically updated in real-time.
In parallel, a separate model checks the same images, and forwards a notification whether there is any kind of anomaly on the ballast.
In order to avoid misclassifications we trained our models on 6 thousand images, optimizing the precision and recall.
For asset recognition, test precision and recall averaged across classes are around 85%, while the anomaly detection algorithm reported 70% of recall and 92% of precision on the test set.
As a result, our solution allowed the client to scale up the Asset Management activity, decreasing the time employed for asset inventory update. Also the dependency on human labour, and the work-related stress. Ultimately, our algorithms outperformed humans both in terms of asset classification accuracy and anomaly detection, showing the possibility to reduce accidents and failures in the long run.
The AI framework we presented shows the potential for real-time, automated Asset Management, providing a tool for more efficient decision-making in the railway business. If you want to know more about our AI solutions for the transportation sector, visit our dedicated webpage.