Researchers at the University of Washington have developed an in-depth learning method that can transform still images to move the moment-stiffened scene. If the system receives a single photo of a waterfall, it creates a video in which the water descends. All that is left is the murmur of water and the feeling of water spray for the perfect experience.
The team method can animate any flowing material, including smoke and clouds. This technique results in a short video that loops seamlessly to give the impression of endless movement. The professional audience can meet the procedure for the first time at the Computer Vision and Pattern Recognition conference.
“The special feature of our method is that it does not require any user input or extra information. All it takes is one image. , produces a seamlessly looped video that looks quite like a real video “- all Aleksander Hołyński, a researcher at the Paul G. Allen School of Computer Science & Engineering, lead author of the study.
Developing a method that makes a believable video from a single photo has been a challenge for the art. According to Hołyński, this requires a practical prediction of the future, but in the real world there are almost endless possibilities for what might happen next.
The team system consists of two parts. First, it predicts how things moved when the photo was taken, and then uses that information to create the animation. To estimate the motion, the team trained a neural network with videos of thousands of waterfalls, rivers, oceans, and other liquid-moving materials. The training process consisted of asking the network to figure out the motion of the video when it was only given the first frame. After comparing his prediction with the actual video, the network learned to identify clues — such as rippling a stream — to help predict what would happen next. The system then used this information to determine if and how each pixel should move.
The researchers tried to animate the image using a technique called “splatting.” This method moves each pixel according to the predicted motion. However, this caused a problem. “Think of a river waterfall. If you just move the pixels down the waterfall, after a few frames, there will be no more pixels at the top of the video!” Hołyński said. That’s why the team created “symmetrical spatting”. In essence, the method also predicts the future and past of an image and then combines them into a single animation.
“Looking back at the waterfall example, if we go into the past, the pixels will move up the waterfall. Thus the waterfall We will see a hole near the bottom of the animation. We will integrate the information from both animations so that there will never be extraordinarily large holes in our distorted images, “the researcher highlighted.
Finally, the experts wanted their animation into a seamless loop to give the appearance of continuous movement. The animation network follows a few tricks for clarity, including transitioning different parts of the frame at different times and deciding how fast or slow to blend each pixel depending on their environment.
The team method is to works best with predictable, liquid-moving objects. Currently, it is difficult for technology to predict how reflections move or how water distorts the appearance of objects beneath it.
“When we see a waterfall, we know how the water should behave. The same is true for fire. These types of movements obey the same physical laws, and there are usually signs in the picture that tell you how things should move in. We’d love to extend our work to work with a wider range of objects, such as animating a person’s hair that the wind is blowing. I hope that eventually the images we share with our friends and family will not be static images. Instead, they will all be dynamic animations like the ones our method produces, “said Aleksander Hołyński.
Hardware, software, tests, curiosities and colorful news from the world of IT by clicking here