# Ateneo Innovation Center uses ordinary videos as rain monitoring sensors via RGB plots

by Quirino Sugon Jr.

Dr. Greg Tangonan giving a lecture in an Innovation class at Ateneo de Manila University

Two weeks ago, Dr. Greg Tangonan and I met by chance at the College Cafeteria for lunch.

“We have a new project at the Ateneo Innovation Center.  We are studying the rainfall through the blur in the images.” Dr. Tangonan said.

“Yes, the image will blur,” I said. “But how do you measure rain from it?  You’ll need to measure the blur and calibrate it.”

“Of course, we shall calibrate it,” Dr. Tangonan said.  “We are looking at the shift in image per pixel.  Our students are simulating these shifts.   We hope some physics guys will come over and see. ”

I was silent.  I was trying to find some physical model that the students can use.  The theory of random walks that simulates the motion of a pixel like a drunk man whose probability to walk in any direction is the same?  But why will that model work?

“If you have time, come with me to my office, and I shall show you our project,” said Dr. Tangonan.

“I can go there now,” I said.

We already finished our lunch, so off we went.

When we reached his office at the Ateneo Innovation Center, Dr. Tangonan opened his laptop.

“This is the image of the rain,” Dr. Tangonan said as he showed me a video clip.  It’s an image of an island across the sea.

“Now, I can choose a box in the image and plot its RGB values.”  Using the mouse cursor, Dr. Tangonan selected a square on the image, clicks on a program to run a software, and out came a display for the average intensity of red (R), green (G), and blue (B) as function of time.  The display looks like three snakes–red, green, and blue–going out of their hole on the left and racing towards the right.

“This is the image when there is no rain,” Dr. Tangonan said as he selected an image box above the clouds.

The blue is more intense than the red, so the graph of the blue is higher than that of the red.  This is not obvious in the video, because the sky appears white.  But the predominance of blue is  is expected:  The sky is blue because the air is made of molecules which scatter more blue light than red light.  So what we are seeing is scattered blue light.

“Now, this is the image when there is rain,” Dr. Tangonan said as he selected an image box below the dark clouds.

I looked at the RGB display.  My jaw dropped: the red is now more intense than the blue.

“Unbelievable!” I said.  “This is really something!”

What I am seeing is the  typhoon forecasting counterpart for rain.  More than a hundred years ago, Fr. Federico Faura, SJ of Manila Observatory noticed the drop in height of Mercury in the barometer before the coming of the storm.  Padre Faura assumed that the hurricanes of Antilles and the cyclones of Indian Ocean are no different from the typhoons in the Far East.  So it is only a matter of applying the equations in the former two phenomena, and Padre Faura predicted the first typhoon that hit Manila in November 18, 1879.  But how do we associate the drop in blue and the rise of red to rainfall content?

“Of course, it is something.” said Dr. Tangonan.  “We plan to study the stratification of clouds and determine where rain starts.  We can even use the data of MMDA–those videos of daily traffic–effectively transforming MMDA’s cameras to a rain forecasting instruments.”

I nodded.  Maybe we can also have a constant light source, e.g. a lamp post,  as reference and take its video continuously during sunny and rainy days.  A  rainfall measuring instrument such as a tipping bucket can be placed between the camera and the lamp post.  The inversion of the red and blue intensities in the RGB display can then be correlated with the rainfall volume.  This is straightforward.

But the theoretical physics part is tough.  Rainfall varies in different places.  We need to have a theory to make sense of the data.  We have to know the size distribution of the raindrops and the raindrop distribution density.  We have to determine the scattering of light in the raindrop: how much light is received by the camera and how much light is scattered away from the camera.  The size of the particles would determine the equations used for modeling their scattering properties.  Mie scattering is the electromagnetic wave theory for spheres.  If the raindrops are much bigger than the wavelengths of light, maybe I can use the ray tracing approximation, which treats light not waves but rays.  But even this is also tough, because this requires statistical ray tracing, if we allow for multiple scattering.  Maybe we can use Beer-Lambert’s law instead, and lump up all the complex phenomena of light absorption in a simple exponential decay.

“I am sold to the idea,” I said to Dr. Tangonan.  “This method for rain forecasting and monitoring is really feasible.”

“Of course, it is feasible,” said Dr. Tangonan. “We don’t do stupid things.”