[ad_1]
Text-to-video is the next frontier in generative AI. Runway, a Google-backed AI startup, has released Gen-2, a commercially available model that generates videos from text prompts or an existing image. However, the model still has limitations. The framerate of the four-second-long videos generated by Gen-2 is low and noticeably so, to the point where it’s nearly slideshow-like in places. Additionally, Gen-2-generated clips tend to share a certain graininess or fuzziness in common, as if they’ve had some sort of old-timey Instagram filter applied. Moreover, there’s the content issue: Gen-2 has a tough time understanding nuance, clinging to particular descriptors in prompts while ignoring others. One limitation of Gen-2 that became immediately apparent is the framerate of the four-second-long videos the model generates.
Subheadings:
– Gen-2 by Runway, A Commercially Available Text-To-Video Model
– Revisiting the Limitations Involved with Gen-2
– Diversity Problem and Bias Test
– Is Gen-2 a Genuinely Useful Tool in Any Video Workflow?
– Incorporating Gen-2 into the Creative Process
– Alleviating Deepfake Concerns with AI and Human Moderation
Gen-2 by Runway, A Commercially Available Text-To-Video Model
Gen-2 is one of the first commercially available text-to-video models. Since several of these models demoed by tech giants remain firmly in the research stages, inaccessible to all but a select few data scientists and engineers, Gen-2 becomes an important distinction in the market.
Revisiting the Limitations Involved with Gen-2
One limitation of Gen-2 that became immediately apparent is the framerate of the four-second-long videos the model generates. Additionally, Gen-2-generated clips tend to share a certain graininess or fuzziness in common, as if they’ve had some sort of old-timey Instagram filter applied. Furthermore, there’s the content issue: Gen-2 has a tough time understanding nuance, clinging to particular descriptors in prompts while ignoring others.
Diversity Problem and Bias Test
Gen-2 passes a surface-level bias test and is slightly more diverse in the content it generated. However, results for any prompt containing the word “nurse” or “a person waiting tables” were less promising though as they consistently showed young white women.
Is Gen-2 a Genuinely Useful Tool in Any Video Workflow?
The takeaway from Gen-2 is that it is more a novelty or toy than a genuinely useful tool in any video workflow. Depending on the video, it may require a lot of post-production work to produce something coherent.
Incorporating Gen-2 into the Creative Process
Runway CEO Cristóbal Valenzuela sees Gen-2 as a way to offer artists and designers a tool that can help them with their creative processes. With some editing work, it may be possible to string together a few clips from various styles like anime and claymation to create a narrative piece.
Alleviating Deepfake Concerns with AI and Human Moderation
Runway says it’s using a combination of AI and human moderation to prevent users from generating videos that include pornography, violent content or that violate copyrights. However, these methods are not foolproof, so only time will tell how effective the moderation will be in practice.
Conclusion:
Runway’s Gen-2 is an impressive feat and effectively beats tech giants to the text-to-video punch. Gen-2 can understand a range of styles that lend themselves to the lower framerate and may be useful for artists and designers in their creative processes. However, Gen-2 still has limitations and will require several iterations down the line before it comes close to generating film-quality footage.
FAQ:
1. What is Runway Gen-2?
Runway Gen-2 is a commercially available text-to-video model that generates videos from text prompts or an existing image.
2. What is one limitation of Runway Gen-2?
One limitation of Runway Gen-2 is the framerate of the four-second-long videos the model generates, which is quite low and slideshow-like in places.
3. Is Runway Gen-2 consistent with respect to physics or anatomy?
No, like many generative models, Runway Gen-2 is not particularly consistent with respect to physics or anatomy.
4. What methods does Runway use to alleviate deepfake concerns?
Runway uses a combination of AI and human moderation to prevent users from generating videos that include pornography, violent content or that violate copyrights.
5. Is Runway Gen-2 a genuinely useful tool in any video workflow?
The takeaway from Runway Gen-2 is that it is more a novelty or toy than a genuinely useful tool in any video workflow, depending on the video, it may require a lot of post-production work to produce something coherent.
[ad_2]
For more information, please refer this link