Projection Mapping on 3D Surfaces

What we do and who we are?

For the past years we have been working with different projects that usually involve

• Graphic design
• Programming
• Projection mapping
• Electronics

.. as well as creating art installations and

• VJ-ing
• Curating
• Organizing workshops
• And audiovisual events

Irina Spicaka

Irina holds BA and MA in Visual Communication and Audiovisual Arts, has studied at the Art Academy of Latvia and Brera Academy in Milan, Italy. In her MA thesis, she was focusing on projection mapping, generative art and audiovisual composition.

Irina works as a graphic designer, curator as well as journalist in fields related to art, business and technology.

Krisjanis Rijnieks

For the past few years, Krisjanis has been exploring different media, programming languages and uses of technology and finally has arrived to a point where he can see how to finally complete his MA studies.

He is a student at the New Media MA programme of the Aalto University of Art, Design and Architecture in Helsinki and also has a background in painting.

CodeBark and CC4AV

We run a small digital media studio CodeBark and are also developing the CC4AV (Creative Coding for Live Audio and Visuals) platform that works as a placeholder for workshops and other activities that we have done in Helsinki (Pixelache pfatform), Riga (Skaņu mežs festival) and Helsingor (Click festival) so far.

We are also collaborating with

• School of Machines
• Fab Lab Berlin
• Betahaus Berlin

Projection Mapping

Multiple Meaning of Projection Mapping

The term mapping highlights the role of space in the context of visual art as this term includes a function – real time projections adjusted to spatial objects.

Projection mapping or real time projections adjusted to spatial objects is a recently created set of electronic audio visual media methods which allows to surpass standard-size projections multiplying the picture, as well as to adjust the picture for different surfaces.

A brief history

While projection mapping has recently exploded into the consciousness of artists and advertisers everywhere, the history of projection mapping dates back longer than you may imagine.

If you try Googling for “Projection Mapping” you won’t find anything older than 3 years. That is because projection mapping’s older, academic name is “Spatial Augmented Reality”. The field is also known as “video mapping”, but projection mapping seems to be winning out in the United States.

For the purposes of this history, I’m only including work that considered projection onto an arbitrarily complex surfaces. Projection onto flat and cylindrical/spherical surfaces has a much older history and goes back to the invention of cinema.

Inventions chronologically

Why It Has Been Invented?

The reason why the projection mapping technique (as we know it today) was invented and developed further was a problem when there was no place for a screen at an event venue. The projector could also be put in a place somewhere in the room where the beam of light breaks partially on the ceiling, partially on one wall and plus on some other surface, thus being absolutely useless. There was no appropriate software at that time yet and specialists of the field worked out – if gamers had various advanced possibilities for using a room in 3D environment then VJs could use it dividing the room by walls and thus providing a more in-depth overview – getting beyond the classic rectangular screen.

How can it be categorized

1. Fixed Surface Projection Mapping
   Projection mapping on buildings or objects that are unlikely to change or, so to say, carved in stone. This allows much more predictable workflow for content creation.
2. Variable Surface Projection Mapping
   Projection mapping on previously unknown surfaces. This limits the content creation to elements of subjective choice. One might prepare videos for different types of surfaces (quads, triangles, circles) or generative visuals with adjustable parameters to fit many scenarios.
3. Custom Surface Projection Mapping
   The type of projection mapping that we will do in this workshop. It allows the most flexibility as you are the architect of your own surface and the author of your own visuals.

Back to the first type it should be noted that when making projections for certain surfaces, the work is first done in 3D and animation software — classical animation methods are applied, such as lightning and special effects, narrative. The most classic example is projection mapping on architectural facades used at light festivals. The buildings are modeled first (or ready-made, modeled prototypes of them are used) and the story is created taking into consideration the design of the particular building. The process is connected with very accurate work where each mistake (let it be no more than a millimeter) both, in animation process or in the projection on the particular surface can be seen as a bias which looks like a defect. Therefore this process cannot be associated with real time performance.

Where projection mapping can be used?

• Stage Design involved in Vj-ing (inluding Light Mapping )
• Audiovisual Performance
• Art Installation
• Scenography and Motion Art
• Large Scale Projections on Architecture
• Window Display and Store Design

Tools

Decent software for creating media content and projection mapping.

Sketching tools

3D Modeling tools

Projection Mapping

Visual programming languages

Text based programming languages

Game Engines

Frame-based animation

Protocols

Software that can bridge visual output from one software to another in real-time.

Tips and Tricks

Best practices from experience.

1 Test it first

Before becomming too artistic, test if the technology works as intended.

2 Backup working versions

If your solution works, save it somewhere safe so when you continue to add changes, you can always switch back to the version that actually did work.

3 Know your surface

The more precise you will know the projection surface, the better. Especially when mapping outdoors on buildings.

4 Sketching helps

Make sketches and storyboards before doing it for real. Going through the process that way initially will help you overcome difficulties before you encounter them in the production process.

5 Always have your own adaptors

And cables. It might save you as well as others. Always have an extra cable and adaptor. It can make you the hero of the day. Or night.

6 No screensavers

Switch off screensavers and sleep for both, power and battery modes, on your laptop.

7 Cold corners

Disable hot corners or Exposé as it ruins the magic when a spectator notices what operating system are you using or that you have the same mess on the desktop as you have.

8 No instant messaging

Disable Skype or any other kind of messenger or service that tends to irritate you when least necessary.

Introduction to Blender

Creating a 3D surface for projection mapping in Blender.

Requirements

• A computer
• A 3-button mouse
• Keyboard with numeric keypad (optional)

Tasks

• Learn Blender basics
• Create your own 3D object
• Export it for assembly in real life

Install Blender

1. Download Blender from blender.org/downloads
2. Install it on your computer
3. Open it

Briefly about Blender

• Blender is a free and open source 3D modelling suite. It supports modeling, rigging, animation, simulation, rendering, compositing and motion tracking, even video editing and game creation. Advanced users can code in Python to customize the application.
• The first version of Blender has been released in the summer of 2000
• Blender is cross-platform and runs equally well on Linux, Windows and Mac computers

3D Computer Graphics

• Three-dimensional representation of coordinate values in 3D space
• Based on Cartesian coordinate system usually
• Uses a process called rendering to represent 3D objects on a 2D screen

3D Objects in 3D Software

• Consists of vertices
• Vertices form faces
• Faces form meshes
• Meshes form objects

Vertices

• Vertices are Cartesian coordinates of a point in space
• Each vertex has a X, Y and Z coordinate
• (1.2, -4, 2.5) - this is a vertex as the computer sees it

Faces

• Are supposed to be flat surfaces most of the time
• A face is formed by no less than 3 vertices
• The simplest form of a face is a triangle

Meshes

• A mesh consists of one or more faces
• It can be a open or closed 3D object
• One or more meshes form objects in Blender

Normals

• Each face has a normal
• A vector that is usually perpendicular to the face and pointing outwards the object
• Is used by the rendering engine to calculate lightning

Blender Shortcut and Cheatsheet

A couple of images that have been made to make the beginning of your Blender modeling life easier.

More about Blender

A couple of links to websites that can help you get confortable with Blender

• blender.org
  Blender main site tutorials. All the basics are there.
• blendtuts.com
  All kinds of Blender tutorials from the Blender user community.
• YouTube
  A lot of questions about Blender can be answered by searching in YouTube.

Exporting for assembly

To make a 3D object out of a 3D model we have to export it as a 2D surface

Install Export Paper Model plugin

1. Download Export Paper Model plugin for Blender:
   github.com/addam/Export-Paper-Model-from-Blender
2. Open Blender User Preferences by hitting Ctrl + Alt + U while in Blender
3. Go to Addons tab
4. Click on Install from File button and select the downloaded io_export_paper_model.py file
5. Enable the plugin by checking the box next to the plugin name in the Addons section

Prepare the Model for Exporting

1. Make sure that you use the correct units (metric) in Scene properties
2. Make sure that the object has scale 1 along all axes
3. If not, go to Object › Apply › Scale to make the current size of the object to be the default scale of it
4. Triangulate all faces by entering edit mode (hit Tab), selecting all faces (hit A) and going to Mesh › Faces › Triangulate Faces or hitting Ctrl + T

Export Paper Model

1. Select the object
2. Go to File › Export › Paper Model (.svg)
3. Adjust properties (detailed explanation in next slide)
4. Chose destination folder and file name
5. Hit the Export Paper Model button

Export Paper Model Properties

• Scale
  For precise output use scale 1:1 by entering 1 in the field. If you will attempt to export a cube that is 20cm wide, high and deep, also the exported faces will be 20cm wide and high.
• Page Size
  Page size that will be used for laying out the unwrapped faces of the model. It should be bigger than the final drawing or the output will be scaled down to fit the page.
• Unfolder DPI
  Dots Per Inch or resolution that will be used by the plugin to generate the drawing. Set it to 72 as it is the default screen resolution.
• Other properties
  You can skip other properties as they depend on subjective taste. Experiment with them and adjust if needed.

Projection Mapping with PiMapper
(and MadMapper)

We have come so far

Summary

We are going to use our computers and available projectors to projection map a surface that has been prepared for this workshop.

A surface?

The surface for projection mapping has it's story on it's own. It has been created in Blender and then made real with the help of a lasercutter.

How is this going to work?

1. Split into groups.
2. Download and install software.
3. Create image or animation source.
4. Have fun with projection mapping!

Software

We are going to use PiMapper during this workshop. And if you like we can go through MadMapper as well (only Mac OS X users). Demo version is free, but you won't be able to save your projects with it.

PiMapper

How to use it and why.

Summary

PiMapper is being developed by Krisjanis Rijnieks as the openFrameworks projection mapping addon mainly to use in projects with the Raspberry Pi devices. It is cheaper way to create standalone mapping installations or portable instruments for audiovisual performances.

Imagine yourself making 10 generative mapping pieces for an exhibiton which is supposed to stay at the gallery for 8 weeks. Would you use 10 Mac computers? Don't do it, it is too expensive!

It's easy to use it!

PiMapper has 4 modes:

"1" (on keyboard) - Presentation mode
"2" - Texture mapping mode
"3" - Surface editing mode
"4" - Source assignment mode

Presentation mode:
This mode is activated once the application starts up. It does not show anything else except the final projection mapping as it was saved previously.

Texture mapping mode:
In this mode you can adjust the texture coordinates of the surface you have selected in the surface editing mode.

Surface editing mode:
Here you can select, move and distort the surfaces you have created.

Souce assignment mode:
After you select a surface in surface editing mode, activate this mode to be able to choose a source for the surface. Afterwards you might want to go to the texture mapping mode to adjust texture coordinates.

Other shortcuts:

"i" (key) - Show info (function)
"n" - Add triangle surface
"q" - Add quad surface
"r" - Add random triangle surface
"f" - Toggle fullscreen
"s" - Save composition
"BACKSPACE" - Delete surface

MadMapper Cheatsheet

A couple of images that have been made to make the beginning of your mad mapping life easier.