AI Video Generation & Filmmaking

RIP Hollywood? Comparing AI Video Generation With Runway Gen-2, Pika Labs & Stable Video Diffusion

What to expect from AI video generation in 2024?

Tristan Wolff
7 min readJan 2, 2024

--

Image by the author & Midjourney

Ok, although some utopians predict the imminent disappearance of Hollywood, I think it will very likely survive and probably even benefit immensely from the AI paradigm shift.

Don’t forget that major studios and production companies have been using AI in film production for years, and the new wave of generative AI models is far too cost-efficient not to be used in professional productions. So Hollywood will be happy to integrate the new models as soon as they are reliable enough.

How long that might take?

Let’s have a look at where we are. Here’s a comparison of the current leading video generation models: Pika Labs, Gen-2 by RunwayML and Stable Video Diffusion (there are many others, but none of the others can really surpass these three in quality).

Comparing RunwayML Gen-2, Pika Labs and Stable Video Diffusion

We will be looking at the raw generations without any wild parameter settings or additional tooling to manipulate camera movements (features that you will find with Pika Labs and RunwayML).

So, for example, we’d be using a base image like this one from Midjourney:

Midjourney image

Here you see the results of all three, from left to right: Stable Video Diffusion, RunwayML’s Gen-2 and Pika Labs (without further parameters or prompts).

Artificial Intelligence in Filmmaking

The integration of AI is significantly influencing pre-production processes already. Story development departments are utilizing language models for rapid idea iteration, script editing and analysis, and even in…

--

--