When you walked out of a recent Star Wars film, did you come out saying you were blown away by the special effects? Or did you mean the visual effects? Most people use the two terms interchangeably, but they describe two completely different crafts. In the case of Star Wars both are very much present, so the mix-up isn’t that problematic. But if you’re getting into this field, the difference matters. Let’s get clear on what each term means and cover the essential concepts of visual effects that power almost every frame you see on screen.
The short version: special effects are physical and happen in front of the camera. Visual effects are created or finished after the camera stops, in post-production. Everything below builds on that one distinction.
Table of Contents
Special Effects
Special effects, also known as SFX, SPFX, or FX, are real physical effects that happen at the moment of filming. They can be mechanical or optical, ranging from simple things like wind or rain on set, to very complex and dangerous setups like a series of explosions. If it happens for real in front of the lens, it’s a special effect, not a visual effect.
Mad Max: Fury Road was a film defined by its heavy use of practical special effects. Even there, though, visual effects were quietly doing heavy lifting in the background, cleaning up rigs, extending the desert, and pushing the color grade. That’s exactly the point: the best visual effects are the ones you never notice.
The execution of dangerous special effects generally involves, beyond the team that developed the effect, only the people strictly necessary for the shoot. In general, only the 1st AD is present near what’s about to happen, following instructions from specialized professionals and making sure safety protocols are being observed.
Visual Effects
Visual effects, or VFX, are a series of processes applied to real footage and computer-generated imagery, aimed at integrating them in a seamless and natural way into scenes where filming would be impossible due to budget, safety, or any other reason. Where special effects live on set, visual effects live mostly in post.
Although most of the work happens in post-production, visual effects are thought through and planned well in advance, generally during pre-production and production. The shots that look effortless on screen are almost always the ones that were planned the hardest. A clean plate, the right markers, and good on-set data save weeks of work down the line.
Compositing
Compositing, in broad terms, refers to the combination of different elements in a scene so that it becomes a single unified image that reads that way visually. It’s the discipline that ties almost every other visual effects process together.
Usually you’ll have a scene shot against a green screen or blue screen and you’ll work by inserting different types of elements into it.
During this process you’ll dial in levels and technical details in each of the layers involved and their respective elements, so the end result reads as a single image. Color, light direction, grain, and edges all have to agree across every layer. When one of them is off, the brain notices instantly, even if the viewer can’t say why.
Tracking
Tracking is used to capture the camera’s movement information and then apply it to elements that weren’t present during filming. If you simply drop a tiger into a base shot, for example, it’ll sit static in the 2D space of the frame while the boat moves. That’s why you use tracking and apply it to the static layer.
In simple terms, tracking is a sequence of positioning data calculated frame-by-frame by the software. Depending on the shot, this is either a 2D track that follows points on a flat plane, or a full 3D camera solve, often called matchmoving, that rebuilds the real camera’s movement inside 3D space. For greater precision, tracking points are placed during the shoot, usually small dots or X markers, to create high contrast between two colors and give the software something clear to read.
Rotoscoping
In the world of visual effects, rotoscoping means manually cutting out something to place it in another frame or to add something beneath the image. You’ll also hear artists shorten it to “roto.” For example, when you shoot a scene without a green screen, the final image is one single frame. Automatically cutting something out becomes nearly impossible because there’s no single uniform color to remove.
That’s what makes manual masking necessary. The user creates a mask with the selection they make in the software and adjusts it frame-by-frame through the end of the shot. It’s slow, patient work, and it’s one of the most common ways someone breaks into a visual effects pipeline.
Keying
Keying is one of the most well-known essential processes in VFX, also known as chroma key, among other names. The concept is simple: the shot is filmed in front of a green screen or blue screen and in post you tell the software that specific color needs to be removed. Done. If executed correctly, your subject will have a clean key after a few adjustments.
Small details can make all the difference in your final output. When something isn’t quite right, or it’s physically impossible to execute the chroma perfectly, it can compromise the key entirely. Spill, soft shadows, motion blur, and fine hair are the usual culprits.
And the most common fix? Hours and hours of rotoscoping to get everything right in your composite.
Matte Painting
Matte painting as a film technique goes back a long time. It consists of reproducing environments that don’t exist, or where it would be impossible to actually shoot.
Before the digital era, one of the most common processes was an artist literally painting the image with the correct perspective for the scene. These were historically known as glass shots, painted onto sheets of glass and filmed in-camera. It was the equivalent of building manually the scene that today is created entirely in post, where the same craft is now called digital matte painting. That’s obviously a simplified explanation.
3D Elements
3D elements are practically a world unto themselves, with many stages and details. Professionals generally end up specializing in one of those stages, or they become what’s called a generalist: someone who can execute multiple processes at a very good level for professional work, but not necessarily with the same precision, speed, or depth as a specialist. Some of the stages of 3D production include:
Modeling: the entire process of building objects, usually starting from one or several base primitives until you reach the final product. It can be highly technical, like engine components, or quite artistic, using programs like ZBrush where you essentially sculpt your model.
Textures: once the model is built, the next step is applying textures, because at this point it’s just a colorless mass. Textures can be created in 2D software like Photoshop and applied to the model. The use of shadows and gradients within colors can also create depth effects that weren’t defined in the modeling stage.
Simulation: this happens when you simulate events through the software by controlling variables. When simulating water, for example, you don’t need to think through the mathematics of water movement, the software handles that. Your job is to define other variables to arrive at the simulation you want, which leads to the next topic.
Particles: a widely used effect because it forms the base for building many different phenomena, like water, smoke, and dust. Based on particles and how you define their variables, you can arrive at the result you need.
Materials: the material, working together with the texture, helps you reach the result you’re looking for with the object. When you define the material’s properties, the object starts responding according to those specifications regarding lighting and other characteristics.
Rigging: before a 3D mesh is handed off to animators, it needs to go through the rigging process. Riggers insert bones and joints into the character, creating a movement hierarchy. Rigging a character can take anywhere from hours to weeks, and it’s essential for the animator to be able to work efficiently.
Lighting: your 3D piece needs to be lit, just as we are in real life, and the 3D lighting process is essentially an emulation of the physical world. That’s why some DPs already use 3D models of their sets to define lighting positions and distances. The more photography knowledge you have, the better-lit your 3D object will be.
VFX 101
As mentioned above, these are some of the essential fundamentals in the field of visual effects. In virtually any shot containing VFX, more than one of these processes will be working together to achieve the best result. A single hero shot might lean on tracking, rotoscoping, keying, and compositing all at once, and that’s before any 3D enters the frame.
Some involve more artistic qualities, like matte painting and compositing, while others generally call for a more technical approach, like rotoscoping or keying itself. The field is broad enough that there’s room for both kinds of brain, and most careers in visual effects start by going deep on just one of these processes. If you want to see how seriously the industry takes the craft, the Visual Effects Society is a good place to start.
Worth a read alongside this one: the hidden cost of AI workflows, since AI is creeping into VFX pipelines too. Are you getting into this area? What are the main challenges you’re running into day-to-day? Drop them in the comments, I’d like to hear what’s tripping people up most.
Special thanks to Diogo Augusto Gonçalves for reviewing the 3D elements section.
Pingback: video-editing-tips-get-organized