Google has revealed its AI-based video generation model called Lumiere. It is a multimodal video generation tool that generates five-second-long videos based on text or image prompts. Currently, there are very few models out there that can create videos based on a given description, including Runway Gen-2. Even though the platform isn’t publicly available, here’s everything we know about it.
Lumiere Uses STUNet Architecture For Creating Videos
A research paper mentions that Lumiere uses STUNet architecture to create videos as a whole instead of generating multiple frames and combining them. Due to the new technology, both the objects in the video and the movement appear natural. Even so, Lumiere generates 80 frames per second instead of 25 frames by Stable Diffusion.
As mentioned earlier, one can’t try Lumiere and create videos with it. However, a landing page showcases several videos created by using the tool. The website contains several videos made on the original video but in a different style. Further, the tool can animate a certain video part based on the prompt.
Last but not least, the tool can also generate and fill areas of a video that aren’t available in the source file, which looks both impressive and slightly worrisome at the same time. Moreover, the tool looks impressive. However, there’s no word on when it will be available publicly.
You can follow Smartprix on Twitter, Facebook, Instagram, and Google News. Visit smartprix.com for the most recent news, reviews, and tech guides.
