Projection Man

Introduction

Release 11 now includes Projection Man. Projection Man has been around for a while but was previously only available to studios. It was developed exclusively for Sony Pictures Imageworks and used in blockbuster features such as Polar Express, Open Season and Beowulf.

It is essentially a management system for Digital Mattes / Camera mapped images i.e. a still image is projected on to a background object and some foreground objects from the point of view of a camera. Another camera moves around and the illusion of 3D is created. The whole idea is to make the background image look not so fake but appear to be actual 3D elements. Imagine a city scene with thousands of buildings. It would take forever to model them all. With digital mattes the background buildings would be painted, photographed etc and put into a still image. Only the foreground buildings would be modelled and even then they could be just cubes or other basic shapes. The idea being that a camera will move, rotate or zoom in and out and the scene that started off as a 2D image will appear to be 3 dimensional. Often many of these scenes would be only a few seconds long but help give a movie a sense of depth and realism.

Before we dive into the various options of Projection Man it´s useful to explain the process of Camera Mapping and how it works and what you need to do. This will make understanding Projection Man that much easier. I have to say several of the reviewers really struggled with Projection Man initially as the documentation really wasn´t that good.

It should also be mentioned that there´s nothing really in Projection Man that can´t be achieved without using it. CINEMA 4D has had camera mapping for years. All Projection Man does is make working with complex scenes that use camera mapping quicker plus a bit more tidier and structured.

Camera Mapping

So what is Camera Mapping? Camera Mapping is very similar to Frontal Mapping except with Camera Mapping an image is projected out from the point of view of a camera and stays fixed to that camera. Frontal mapping on the other hand changes so that the projected image always points at you no matter what your point of view or camera is. The following images demonstrate camera mapping. As you can see as soon as you move away from the projected camera view the textures become distorted. For this example I´m using an image created in CINEMA 4D and I´m doing that to save myself time. In reality you would use a photograph or an image generated in a 2D program. The scene used to create the rendered image is completely different than the scene use for the camera mapping. The original scene used a sky object and has a Flag deformer applied to a plane object to get the undulating sand. The scene where I camera mapped the rendered image on to objects has a flat plane for the ground and a plane for the background. Then just 3 pyramids were used to match the original pyramids. As you can see the result when the rendered image is projected on to the objects is very good.

Click on the images for a larger view.

Original 2D Image	Looking through camera (no textures)

Camera Mapped and Rendered	Editor view of camera mapped scene

Editing The Image

In the previous section you saw how the 2D image was projected on to the 3 foreground objects (Pyramids) plus the ground plane and background plane. The goal of what we are trying to achieve is to animate a second camera and have the 2D image combined with the 3 foreground objects look like proper 3D as this second camera is animated to move around. To do that some editing of the original 2D image is required so that areas that were previously hidden and now visible have these areas filled in. This is achieved in a 2D paint program like Photoshop or BodyPaint. Typically the clone brush is used to clone paint from one area to another. For our little project we need to cut out the 3 pyramids and fill in their spaces for the image to be applied to the ground and background objects. Then on the 3 pyramids we need to apply another version of the original 2D image, this time with the background removed using an alpha channel. In addition, depending on what the animated camera will be doing, each pyramid needs its own adjusted version of the original 2D image applied. This is usually on foreground objects behind the very front foreground object where parts of them are hidden and become visible. On these back objects the areas that become visible need patching with the clone brush. Shadows play an important part as well and these often need some work with the clone brush.

Just before we dive in an example of using Projection Man, whenever you´re working with Camera Mapping wherever possible work in Enhanced OpenGL mode. Doing so will display textures more accurately in the viewport. This is particularly so if the objects being mapped are highly segmented. Lowered segmented objects will often look distorted. Increasing the segments or switching to Enhanced OpenGL mode will make things look much better plus be more accurate to work with.

Projection Man

Finally we get to discussing Projection Man. I thought it was pointless discussing Projection Man if readers didn´t have an understanding of what camera mapping is an how it works. The interesting thing about the example above is that even though Projection Man wasn´t used, if you go to the Window menu and selection Projection Man you´ll see everything loaded as if I used Projection Man to create the camera, load the materials and restrict the material to the various objects. Here´s a screen grab of Projection Man.

Projection Man.

In Projection Man, one or more Cameras are used for the projections. When I created this manually I also added a protection tag so that it could not be moved or edited. You can protect cameras in Projection Man from the right click menu. The animated camera is shown. Cameras can be created, duplicated and deleted directly in Projection Man. The objects with materials using Camera Projection (and only camera projection) are listed under the camera that was used for the projection. There is only one projection camera used in this scene but if there were more then each projection camera would be listed. Each projection camera would then have those objects using it for camera mapping listed beneath it. Not shown but if your scene has objects without materials or using materials with different mapping modes then these are listed at the bottom as unassigned objects.

Listed across the top are the various material channels. Strangely these don´t match the same order as how channels are listed in materials. I guess the logic is the 3 most important channels used in camera mapping namely colour, luminance and alpha are listed first. Since I used separate images for the alpha channels you can see these images listed separately. If I used images with built in alpha channels the listing would show both a texture loaded in the luminance channel and the alpha channel all on the same line / row.

Camera Mapping With Projection Man

Let´s use the previous example but this time use Projection Man and do the image editing in Photoshop. I´ll delete the original sky and create a new animated sky. The hardest part is getting a camera set up. To make things easier I´ll use a camera rig available from Cineversity although it is not required. This is loaded into the scene from the Content Browser. More information below. I did consider making a video of me working through things but settings up cameras and patching textures can be a bit slow going.

Step 1 - Setup Plate Camera. I´ve started with a completely blank scene and loaded the Cineversity Plate Camera preset. I then loaded in the original 2D image into the Plate Camera. Next I orientated the camera and changed the aperture size and focal length to match my original image which conveniently I knew. (If you´re using photos, often the file will have embedded meta data containing these values). The screengrab on the right is looking through the Plate camera and the screengrab on the left is an Editor camera view of the scene.

Editor view of Plate camera projection.	Load Original 2D image into Plate Camera.

Step 2 - Match simple objects to Plate image. Just as as I did above I need to model and match simple objects to the reference image. Just a matter of creating 3 Pyramid Primitives and sizing, positioning and rotating to match the view plus add in a ground plane. This time I won´t need a background plane object as I´ll edit out the sky in Photoshop. The good thing with using the Plate Camera preset is that you can easily alter transparency (provided you are in OpenGL mode and with transparency enabled). In the image on the left below you can see the front image plane just in front of the camera. The result of looking through the camera is on the right. In this image I enabled Gourard Shading Lines) so that you could see the polygons on the Pryamids.

Editor view of scene	Looking through the camera

Step 3 - Apply 2D Image using Projection Man. Yay, Projection Man at last. Load Projection Man from the Window menu. To apply the original 2D image to the 3 pyramids and the ground plane select them all and drag and drop under the "orginal2D.jpg" image. The image shows the pop up boxes that you get when you do this. I didn´t want alpha channels so I disabled this option and I wanted to use the same Layerset. This process is the same as manually creating a material, loading in an image in the Luminance channel, applying to the object, changing the texture tag projection mode to camera and pressing the "Calculate" button except it´s a lot quicker using Projection Man.

Apply 2D image using Projection Man

After doing this, the Front and Back image plane objects can be hidden as we don´t need to use them again.

Step 4 - Patching each Object. In this example, each object needs its own texture. Alpha channels in this instance aren´t really required but I´ll create them anyway since it takes virtually no longer to create them. I have Photoshop set as my default 2D editing program in the preferences (it could be something else) and for each object all I have to do is select it, then right click and select the To Photoshop command which in this case will perform a Coverage Render and opens the saved render in Photoshop. I´ll rename my pyramids so that I don´t get confused and this can be done in either Projection Man or the Object Manager. The image below shows Projection Man with the To Photoshop command added as an icon. After pressing this the options on the right pop up. I have enabled the alpha channel option although as mentioned I probably don´t need alpha channels since this is a simple scene based on simple shapes. I disabled Constant Shading since the textures are loaded in the luminance channel. Lights in the scene have no effect on objects with these materials applied. If I had loaded textures into the colour channel then scene lighting does impact upon the objects and their materials. The Constant Shading option as far as I can tell generated a render in the colour channel that is the same as if the texture map had been loaded into the luminance channel. Using lights with the colour channel and sending to Photoshop (or any other 2D editor) is just like baking textures where you bake lights and shadows on to a texture applied to an object.

Creating a texture map for each object

This needs to be done for all 3 pyramids and the ground plane.

Step 5 - Patching in Photoshop. The 4 texture maps generated in Step 4 need patching where areas of them that were previously hidden will become visible once we start moving around with an animated camera. The front pyramid doesn´t actually need anything done to it but the other two pyramids and ground plane do. Patching is done like in a 2D paint program mostly using the clone brush . Below are the ground plane, middle and back pyramid objects before and after patching in Photoshop. Projection Man automatically fired up Photoshop and loaded the image. You can see the back pyramid needed a lot of work with the clone brush and also with rotating the blocks to look right. Better results could obviously be achieved by spending more time on this. I didn´t need to edit the alpha channels that CINEMA 4D / Projection Man generated.

Ground Plane before Patching in Photoshop	Ground Plane after Patching in Photoshop

Middle pyramid before Patching in Photoshop	Middle pyramid after Patching in Photoshop

Back pyramid before Patching in Photoshop	Back pyramid after Patching in Photoshop

Step 6 - Load Patched Textures. All that´s left to do now is to load the patched textures for each object. First a camera for the projections is required. I created this by selecting the Plate Camera in the Projection Man window, right clicked and selected "Duplicate Camera" to make a copy of it. I renamed this Camera "Projection Camera" and then from the right click menu applied a protection tag. Then in turn, I dragged each object to be child of this camera. As you do this a dialog window pops up and you load in the patched texture. Following this a second dialog pops up where you can set up the layers / layersets and whether you want to use the alpha channel if the texture had an alpha channel.

I should mention here that in more complex scenes you would likely have multiple objects sharing the same patched texture. If that´s the case then you would select multiple objects and drag them under the projection camera. Alternatively you can drag and drop an object or objects underneath a texture where multiple objects share the same texture. Quite a bit quicker than manually doing it. After doing this I found I had to delete the original plate camera as I had 2 materials applied to each object. I also ended up with faint line around some of the edges. The cure was to make the pyramids slightly smaller by a couple of units. The result of all this is shown below.

Textures Reloaded using Projection Man	Pyramids made slightly smaller

You can use Projection Man to duplicate the original Plate camera and use this for your animation camera or you can copy it in the Object Manager in the normal fashion.

Step 7 - Add Sky and Animate. All that´s left to do now is add in a Sky and animate the camera. I just used a Sky preset from the Content Browser. Below is the final animation which uses an animated sky and helps make the animation feel more real. The result is below.

Final Animation - Projection Man Method. Click image to play (QT 3.1mb h.264)

I liked the yellow colour on the horizon line which is actually part of the sky. I felt this simulated sand far off in the distance. I´ve also for the purpose of this animation increased the speed of the cloud animation.

There´s a bit more to Projection Man than what I have covered here but hopefully I have given you a feel for it.

If you have a Cineversity subscription there is a very good Projection Man tutorial available here. Note that the first 2 tutorials in this series are available free and the first one includes a free Projection Man layout that can be downloaded.

Also available from Cineversity if you have a subscription is Grip Tools, a set of utilities / scripts as shown in the image below. There are all sorts of really useful things like a command Grip:: Copy to Editor Camera to make the viewport editor view / camera the same as a selected camera. Something you´ll sometimes need to do when camera mapping. Also available if you have a Cineversity subscription is a camera rig that I used in the example above. Very handy when working with camera mapping and Projection Man.

Conclusion

I don´t do much camera mapping so I initially found using Projection Man tough going. I understand how camera mapping works but I think if there had been a decent tutorial included with Release 11 I would have made much quicker progress than I did. No doubt CINEMA 4D users who do a lot of camera mapping may well consider Projection Man the best thing since sliced bread. What helped was the fact that I have a Cineversity subscription and was able to download some handy utilities to work with it but in my opinion those utilities should be part of CINEMA 4D and you shouldn´t have to pay extra to make Projection Man more useable. R11.1 maybe?

Projection Man is handy as it speeds up the work flow. The documentation is typical of CINEMA 4D´s documenatation in that tools, options etc are explained but the process of actually doing anything isn´t covered particulary well or even at all. I found the "Workflow Example" mini tutorial in the documentation impossible to follow. What would have been very useful is if Release 11 had come with a tutorial DVD like Release 9 and Release 10 had where many of the new tools and enhancements were demonstrated. Sadly Release 11 does not include a tutorial DVD apart from the R10.5 advanced rigging DVD as mentioned in the introduction.