How to make videos with niji・journey
How to make videos with niji・journey
Long time no see! We've been long overdue for an update.
And this one is really impressive. WE MADE AN ANIME OPENING!!! Er, I mean, Niji can now generate videos!!!!!!!!
While I'm generally psyched about each and every feature we make, I can say for sure that video-fying your pictures is the best damn thing we've shipped in a while. I really hope you enjoy it!
To see the video model in action, you can check out this ANIME OPENING we made using our video model!!!!!
(My weeb dreams have come true!! 😭)
This feature has been long in the making, so we are suuuuper excited to finally launch it! In this post, I'm going to give you some simple instructions on how to use it, and then talk through some helpful tips in getting the best quality video ✨
How to make videos using Niji
To make a video with niji. You'll want a starting frame and a prompt. The starting frame describes what is moving, and the prompt describes how it moves.
Selecting a image from your gallery
Go to the "Create" page and click on the small image icon on the far left of the 'Imagine Bar'.
Drag any image into the 'Starting Frame.' This will be the first frame!
Any past images that was uploaded for past image prompt, or style references, or even character references can be found in the panel here. You can use any of them by clicking here.
Once your image is selected, type your prompt into the imagine bar, and let the video model work its magic! If you don't have a prompt in mind, that's alright, too! you can simply submit the image, and Niji will give you a recommended motion.
Tips to getting a stellar video
The image style dictates the motion
Let us demonstrate the effect of the first frame with different styles! Here, we use the same prompt
Tips
a girl reaches up and picks an apple
If the first frame is in the style of TV anime, it will make TV anime motions.
If the first frame is rendered realistically, it will make 3D motions
If the first frame is in the style of illustration, it will make storybook illustration motions
As you can see, choosing the style of the first frame with care is very important to the animation!
Don't make it too complex
Remember that this motion will happen in 5 seconds! So try to avoid making it too complicated.
There are 3 simple rules to writing a good video prompt:
- describe only what you can see, not what the character feels on the inside.
- describe a primary action (what is happening)
- describe a secondary action (what is flopping around as a result of the action)
For example, note the difference between:
Anime prince with blue eyes and blonde hair looking around in confusion
vs.
An anime prince with blonde hair and blue eyes turns his head left and right. His cape billows behind him as he pivots, while loose strands of hair sway across his face as he wears a confused expression.
By paying attention to secondary motion, we can make the scene much more lively!
Of course, all that motion can be distracting. (Sometimes maybe you do just want the guy to turn his head), so be sure to include as much and as little detail as you need.
Tips
If you don't wanna think about all this, you can certainly also put in a picture with no prompt as well! Niji will take a guess at how it will animate.
Remember, the picture is the FIRST FRAME
When we are designing a still image, we often take the most powerful moment: the highest point in the jump, the apex of the motion, the most unfurled part of the grand gesture.
However, if you want to make a pleasing video, it is important to start at the beginning, not the climax.
For example:
If you want a marble statue to open its eyes, you can't put in a picture of the statue with open eyes.
You must put in a picture with closed eyes:
And only then she will open her eyes.
Go for scenes where the action is unresolved
Sometimes, it is difficult to follow the first frame rule. For instance, in the example above, we can't control the shape of the opened eyes.
One solution is to write "a statue closes their eyes" and then play it backwards.
Another solution is to start in the middle of an unresolved action.
For example, if you want to make a scene of a girl catching a piece of toast in her mouth, go for this, where the fate of the toast is uncertain.
Instead of something like this, when the toast is already caught.
In this method, the front of the action is missing, but you can flesh it out with some creative cutting.
The start of a new adventure!
Getting some good cuts is just the beginning! In our next post, we'll do a deep dive into the arrangement and editing of videos, featuring an in-depth look at the making of our anime OP!
I hope you enjoy making your creations move!
Once again, thank you for your support on this journey.
Original article from niji・journey blog