Video Formula (WIP)

Prompting can be broken down into three areas:

• Text Prompting
• Image Prompting
• Parameters

Prompt Structure

You MUST follow this structure to get your prompt working:
/create prompt: description of the video you want to create parameters

When it comes to text prompting, there is no single right or wrong way to do it. But generally, prompts follow a similar structure:

MEDIUM: Cinematic Film, 3d Animation, 2D Animation, Etc
STYLE: Action Film, Drama Film, Style By (filmmaker)
SCENE: Who is your subject? Where are they located?
ACTION: What is happening in the shot? Walking/Talking etc.
MODULATE: Are there external effects, rain? fog?
CAMERA: Camera angle shots (refer below)
PARAMETERS: (Neg) Details to Follow

Examples

Cinematic, medium shot, crime film, a man wearing a black suit in new york diner, drinking coffee, 1960s aesthetic

Cinematic scene, dramatic film, close up, a young fisherman on a boat, looks longingly at the woman who he loves, smiling, rain falling

Text prompting can be very hit or miss, depending on what you are looking for. It will take a number of re-rolls and prompt adjustments to get a good result. But text prompting (as opposed to image prompting, can often yield very surprising, and delightfully unexpected results.

Shot Descriptions

Here is a list of various “called shots” that have been known to work.

*These terms are not always guaranteed to work, but you may get results from them, depending on the context of the rest of your prompt

Image Prompting

In addition to text prompts, pika allows you to include an image to provide visual context and inspiration for your generated video. After entering your text prompt, simply upload or paste the url of an image related to what you want pika to create. This image “Seed” gives the AI more to work with, helping it pick up on visual details like colors, lighting, poses, objects, and setting.

Use a text prompt in conjunction with the image prompt to describe action, such as “Man smiles.” remember that these are very simple shots and multiple actions such as “Drinks coffee and stands up” are highly unlikely to work

Alternatively, you can experiment with the new /animate command, which will take an image prompt and automatically animate it. The option to prompt alongside /animate is also available.

Parameters

Negative prompt
Prompting unwanted words, what you don’t want the video to have accepts: words surrounded by quotes “

Negative prompting seems to be very useful, particularly using an old stable diffusion format. Try adding it with the following, if you are running into unstable outputs:

“Ugly, blurry, deformed, multiple limbs, pixelated, static, fog, cartoon, vector art, flat, unclear, distorted, error, still, low resolution, oversaturated, grain, blur”

General tips and advice

Often re-running, or reprompting an output will yield better results. Try re-rolling at least 3 times to see if pika picks up on your prompt. Should that fail, use the remix button to adjust your prompt. You will find these buttons under your video output

Text prompting should be done at an elis (explain it to me like I’m five) level. The model doesn’t understand nuanced language or technical descriptions (yet). For instance, “Cinematic tracking forward” generally won’t get you the results you want, but “View from a moving drone” will.
– tyber white

One thing I’ve found so far is the motion controls work best on a landscape or zoomed out image of a subject. If you try ap- plying motion where the subject takes 33% or more of the composition, I get very little, no movement or unexpected results.
– tom blake

You’re not writing a book, you’re describing a physical scene, and you are only making 3 second videos. You have enough data in that prompt for a 5 minute video. Keep that as your overview sketch and then prompt a lot of 3 second clips. Such as: clip 1: a crow perched on a branch, looking around. Clip 2: a crow flying over the woods. Clip 3: woodlands, wilderness, birds-eye view clip 4: a crow flying over the forest, snow capped distant mountains.
-crystal wizard

I find keeping guidance scale around -12 to -15 is a great spot. And can’t reinforce how important -neg are.A great string. -neg “Blur, blurry, artifact, video artifacting, video noise, distortion, ripping, tearing, distortion” I alter it based off of subject needs. If there are hands, I add that in. For example a portrait. “Zoom in. Cinematic. Portrait. Wide angle. Close to camera. Symmetrical. Centered framed. Subject description. Background description. Then the fun stuff -neg arguments. And I get amazing results
– microphonist