When deep learning meets satellite imagery


Hi everyone, welcome to
this second AI Academy video! Last time, we talked about
the data science team at Earthcube, who they are,
what is their mission. Today we are going to talk about satellite imagery
and Artificial Intelligence: how can we combine them,
and where is the challenge? First, let’s define Artificial Intelligence. In the scope of this video, Artificial Intelligence will mean deep learning. And by deep learning, I mean
the use of a deep neural network. Secondly, let’s define satellite imagery. Right now, there are satellites orbiting around the Earth,
at more than 600 km above its surface. Each satellite has its own sensor,
which gives you specific data for each satellite. They are taking pictures of the Earth,
at a given place and at a given time. Now that we have defined both
Artificial Intelligence and satellite imagery, how can we work with both of them? Let’s take the same example
as in the previous video: vehicle detection in satellite imagery. We are going to divide
the subject into three parts. The first part will be
why satellite images are specific data. The second part is
why they are challenging to work with, and finally we are going to see
how deep learning can be the key to solve this challenge. Let’s start with why satellite images
are specific data. First, if you compare a photography
and a satellite image, you will see that usually, a satellite image has much more pixels than a classical photography. And that, considering that a photography,
can already have thousands of pixels. Secondly, if you compare a photography
and a satellite image, you will see that you don’t have
the same number of channels. What do I mean?
In a classical image, you will have three channels: one for the red color, one for the blue color,
and one for the green color. You will have all these channels
in a satellite image, but you may also have others
and dozens of others. Also, when a satellite
is taking an image of the Earth, you may want to know where the place is. So, you will have
to consider geo-referenced images. These images will first be recorded
in an elliptical frame. And from this elliptical frame, you will have
to take your coordinates into a flat Earth projection. And this allows us
to introduce the ground resolution. When a satellite is looking down
at the Earth surface, each of its sensor’s pixels is able to record
a certain distance on the surface. Depending on the satellite,
it could be 30cm or 50cm for instance. And then you just have
to pick the satellite of your choice according to the use case
you want to work on. All this information is recorded
in the metadata of the image. But in this metadata, you may also find the sun angle,
atmosphere conditions, the satellite position… for a few examples. At Earthcube, we have a database
that considers all these metadata and aggregates them
for all the single images we have. But we do even more:
in our database, we also have the land cover. The land cover is what you will see
on the ground surface when looking at a specific area. For instance, it could be the sea,
but also a desertic area, or an urbanised area. Now you know everything about satellite images,
let’s see why they are challenging to work with. First thing, when you take a satellite image
and look at the small objects in it, these small objects are for instance cars,
while cars are supposed to be big objects for us. A car in a satellite image is just a few pixels. If you take a look at an aerial image, it will be easier, because the resolution may reach 10cm,
but it’s not the case on a satellite image. Secondly, when a satellite records
an image of a specific area, at a specific time, it only has one shot.
What does it mean? It means that if the weather is bad,
if you have clouds, if you have snow or haze, you will have to deal with it, and your vehicle detector will have to see
the car in the haze or in the snow. Another important thing is
the position of the sun, because your image may be overexposed, or on contrary,
you may have huge shadows in your image, and your vehicle detector will have to be able
to detect vehicles even in the shadows. Lastly, when a satellite looks down at the Earth and when the line between the sensor
of the satellite and the centre of the image makes a right angle with the ground surface,
we say that the satellite is at nadir. This is the position of the sensor where you will get the best resolution. But what happens if the orientation of the sensor changes? Then, the resolution won’t be as good as at nadir, and in the end, you will have buildings, for instance,
hiding objects such as vehicles. Now you know everything about
the specificities of the satellite images, and also about why
they are so challenging. Now, remember, we want to detect
all the vehicles in satellite images. If you want to do that, you may try
with computer vision methods. And of course, you might be able
to detect a few vehicles, but detecting all of them
will he really hard. Why?
First, as we said before, a vehicle in a satellite image will be represented by only
a few pixels and only a few features. So you might not have enough information
to use classical computer vision methods. Secondly, the context around
all these vehicles will always change. And this is where deep learning has proven
to be a great solution for such use cases. And this is why we use deep learning at Earthcube. When working with deep learning and images, usually you will turn towards Convolutional Neural Networks,
or an evolution, such as Capsule Networks. But with satellite images,
you will have to keep in mind three things: The first one, as we said before,
is that our images are really big. This means that your network won’t be able
to process all of it at once, and you will have to divide it into
smaller tiles before putting it in your network. The second thing is that your image
may have more than three color channels, not just red, blue and green colors. Your whole pipeline will have to take
care of all of these channels and to take them into account, or not. And the last thing
and the most important is that you have to consider
that this data is geo-referenced. So in the end, you want to have
your vehicle detection geo-positioned not only in the image
but also on the Earth surface. All your AI pipeline will have to be able
to manage this type of data. If you are able to manage all of this:
the specificities of the data, how they are challenging, and then to build your
whole AI pipeline considering all these specificities, then you will have the good combination
to be able to extract information from satellite images. We are reaching the end of this video, thank you for watching
and we hope you enjoyed it!

2 Comments

Add a Comment

Your email address will not be published. Required fields are marked *