Road Crack Detection — General Workflow
How to apply deep learning algorithms to detect road crack in Indonesia.
At the plenary session of the first Esri Indonesia User Conference, EIUC 2019, Geo AI director from Esri Inc demonstrated the use of deep learning capabilities in ArcGIS to perform automated road crack detection in American road. The faster the detection process is better because road crack could cause economic and safety impact if it is not fixed immediately. In this article, I will show how to automate the detection process with a use case in Indonesia.
List of contents:
1. Setting up the environment
2. Prepare the data
3. Label the data
4. Train the data
5. Simulation
Setting up evironment
Before we implement the deep learning algorithm for detecting road crack, first we need to set up the environment first. Basically we just need to install tensorflow library for training the model, but for faster training model we must make sure that the tensorflow type is GPU instead of CPU.
I will not explain in detail how to install the environment, but if you want to learn more about it, you could check this link:
Link 1
Link 2
Prepare the data
After we successfully set up the environment, the next thing to do is to prepare the raw data. Because we want to create a model for detecting road crack, we need a bunch of road images with road crack features as the raw data. There are two ways to get images. First, you could take the images by taking photos. Second, you could use the existing images which are shared publicly.
In this article, we will use the second way. I found a Youtube channel which shares a road survey video in Sumatera Selatan. Technically, we need to download the video and then split the video into several images by time (for example: if you have 60 minutes video, you could have 60 images if you divide it with a 1 minute clip rate) or frame (for example: if you have 60 minutes video which consists of 3600 frames, you could have 60 images if you use a 60 frame clip rate). *
*) Make sure that the images have road crack features.
Label the images
Next, we need to label the images by creating a box-shape boundary to the specific features, like the road crack, and then give the label name to the boundary. For labeling application, you could use labelImg and download that in here.
The output of the labeling process is an image (.jpg) with its label (.xml). See the picture below to get a deeper understanding of what I mean.
Train model
As the deep learning model has the same concept as machine learning, to start the learning process we need to split the images dataset into the train set and test set. Create two different folders named test and train, then split the results of the labelling process from the step before. We could use the 80:20 golden rule, for example, if we have 100 images, put 80 images to the train folder and 20 images to test folder.
After we split the images, then the next step is to train the model based on the train images. We will use Faster R-CNN configuration for the tensorflow framework as it is proven for building a good model. Set the number of steps to train the data, the bigger the steps then the results will be better. I set 50,000 steps to train the road crack model.
In order to get understand how good is our model, we need to check the losses of the training process, the more the losses convergence to 0 then the model will be good.
Test model
For the last step, use the model we had from the previous step to the video we want to test. You could see the result from the video below.
For the next phase, I will join the result of the model in this article with GPS data and combine it with Esri’s dashboard solution. This dashboard will give us information about where the crack happened and how does it look.
If you think that the workflow is too general, I have a plan to write a more detailed and technical workflow soon. Stay tuned :)
