|
Adobe’s Firefly video generation panel. Screenshot: Adobe |
Adobe has announced Firefly Video, an AI model that can generate video footage that the company says is designed “to be commercially safe” and to “not infringe on others’ IP.” Some of the generative AI features are being integrated into the company’s editing software, Premiere Pro.
Premiere Pro is also getting several updates unrelated to the AI features; click here to jump to the section about those.
Perhaps the most useful feature Adobe showed off in its keynote address at its Max conference is Generative Extend. The feature is meant to continue video or audio clips after the end of the actual recording or cut, generating a few extra seconds of footage or room tone to help fill in gaps. The plan is to build it into Premiere Pro, so you can extend your footage straight from your timeline. Like most of Firefly’s video abilities, the feature will be launched as a beta.
 |
|
You can generate new footage to fix eyelines, fill in gaps, or help smooth out a J/L cut, straight from the timeline. Image: Adobe |
There will initially be a few limitations: the most notable is that the beta version can only generate 720p or 1080p 16:9 video at framerates between 8 and 30fps. It also only works with 8-bit and SDR video.
The company also showed off Firefly’s ability to ‘animate’ still images, turning them into video, though that feature will launch on the web version of the tool, not in Premiere. The tool can also generate video assets along with actual footage. The company showed Firefly-generated light leak artifacts that were added to a conventionally-shot video, as well as AI-generated animation and text graphics.
 |
| ‘Filmic’ light leaks generated by Firefly that could then be composited on top of existing footage. AI-Generated Image: Adobe |
Of course, Firefly Video can also generate footage whole-cloth based on text prompts from the user. The UI gives people several ways to control the video that’s generated, letting them pick an aspect ratio and frame rate, as well as specify what kind of framing, angle, and motion they want. However, Adobe says its customers are generally more interested in how generative AI could help them edit human-created content than using the tech to create something from scratch.
Adobe stresses that it has trained the model “responsibly,” using “licensed content” from Adobe Stock and the public domain. Representatives for the company said it was not trained on data scraped from the web or customer data, practices that have been controversial with other generative AI models.
The company also says that footage generated or edited with Firefly will automatically have Content Credentials attached, tagging it as being created with AI. However, similar to images created using the system, it currently falls to the user to manually check a video’s veracity, as most platforms don’t interpret the Content Credentials metadata.
Premiere Pro updates
 |
| Image: Adobe |
Adobe also announced that it’s adding a new ‘context-aware properties panel,’ which will attempt to surface the most useful tools based on what you’re currently working on.
The company has also updated the Frame.io panel, saying that the new version will make it easier to track reviews and approvals. On that note, Adobe also announced that Canon, Nikon, and Leica are adding support for Frame.io’s Camera-To-Cloud feature, which lets cameras automatically upload footage to the service without needing to manually offload it to a computer. There are currently no details on which camera models will support it.
Premiere is also getting a performance bump, with the company promising that it’ll be able to export to ProRes up to three times faster. The app is also getting a bit of a redesign, which the company says will make it feel “modern and more consistent.”
We’re attending Adobe Max, so keep an eye out for demos of some of these features. If you want to follow along live, check out our Instagram page.