Building a Multi-Touch Table: Converting the Webcam

Posted on April 5, 2010 by Comments Off

The style of multi-touch table that we are building relies on the “seeing” of infrared light to pick up IR reflection and subsequently use that information to convert those reflections to “touch points” (for details, see  Multi-Touch Table Kickoff). To accomplish this we converted a webcam for this specific use.

The PS3Eye webcam in it's many parts.

Digital image sensors can inherently “see” IR light, but you’ll find that most cameras and camcorders will contain an IR blocking filter that blocks the IR light before it reaches the sensor, which minimizes IR distortion in the final image. This is a good thing for a majority of the intended audience, though sometimes, whether for creative reasons or geeky, tech reasons (in this case), we want to be able to see that IR light. To do this we need to remove the IR filter from our selected camera, the PS3 Eye. Once that is done, IR light is allowed to pass through the lens and onto the sensor. However, we want to fully isolate what the sensor sees so that it is only IR light. This is done by then filtering out the visible light coming through the lens. We can either do this by adding a more expensive bandpass filter to the camera, which only allows a certain wavelength or “band” of light to pass through it, or go the less expensive route of using a piece of floppy disk to act as our bandpass filter. While not as clean at filtering visible light, the floppy disk does a more than adequate job for our purposes. Plus, we now have a use for that electronic copy of my old undergrad thesis, which is, thanks to the era in which I got my degree, backed up on a floppy disk. Here’s a sped-up version of the conversion process:

A close up of the IR filter in the lens

In short, we replaced the camera’s IR filter with a bandpass filter to allow only IR light to pass through it. At this stage we were able to test how well this conversion worked by connecting the camera to the computer and positioning it facing the table’s acrylic top while up on its side. We turned on our 850nm IR lights, bouncing light into the acrylic, and once that acrylic was touched it created a reflection point for the IR light, which was immediately visible on the computer monitor. Not too shabby. 

 The IR camera in action reading touch points

If you would like to see the detailed video of how this is done I’ll direct you to the wonderfully informative PeauProductions tutorial.

Building a Multi-Touch Table: The Cabinet

Posted on January 25, 2010 by 1 Comment

In my last post, I introduced an internal project aimed at creating a multi-touch computer from the ground up. I introduced the project and talked a little bit about the method we chose to work with (FTIR).

In an FTIR setup, IR light is cast from the edges into an acrylic sheet where it gets trapped, bouncing around within the pane. A “compliant” surface (typically a vellum sheet with silicone on one side) is placed on top of the acrylic. The compliant surface acts as a projection screen for a projector placed inside the multi-touch cabinet. In addition, when touched, the silicone squishes against the glass, scattering the light out the bottom of the acrylic sheet, where it is picked up by an IR camera. The IR camera is attached to a computer, and the touch points are converted to “blobs” via an open source API. Depending upon the API and methods being used, the blobs have size, direction, velocity, etc.

All of that sounds really cool, but we’re getting a little ahead of ourselves. We need something to house all this good stuff. Given a limited budget and the fact that we’ve never done this before, we’re not going to spend a huge amount of time or budget on the cabinet. We agreed we wanted a stand-up table rather than a sit-down type, so with all of this in mind, I used Google SketchUp to create a design for the cabinet, shown here.

Multi-touch Cabinet

The desks we have in our office have a crank that allows us to raise and lower the height. We tested a few heights, starting with a normal kitchen counter — around 36 inches. This seemed a little too low for tall people, so we bumped it up to 38 inches, which seemed to work well in informal testing for a variety of people from 5 feet 1 inch to 6 feet 3 inches. Our touch surface will be 24×30.

Whether or not we go with pressboard is still up for debate, but I’m in favor of painting the cabinet black with a flame job… have other ideas? Know somebody that wants to make us free furniture in exchange for a signed copy of my book? Let me know in the comments or by e-mail! Just remember, we’re on a limited budget…

Building a Multi-Touch Table: Kickoff

Posted on January 25, 2010 by 2 Comments

Waggener Edstrom gives employees the opportunity to create something we call an Experience Influence Project, or EIP. This project is a self-paced, self-designed learning opportunity, and can be done as an individual or group effort. The real goal is to bring some learning to the agency in a way that expands our understanding of influence.

We had an idea floated several months ago that I decided to follow up on for my EIP — building a multi-touch table from scratch. Touch computing has really taken off and doesn’t seem like it will be going away anytime soon; most cell phones support single- or multi-touch, Windows 7 supports multi-touch on tablet PCs, Apple has a tablet PC coming soon according to long-standing rumors, and there are high-end multi-touch systems like the Microsoft Surface table. The goal of my EIP project is to build a multi-touch table along the lines of the Surface computer on a limited budget (the cost of a Surface is north of $10,000).

I started out by gathering a small group of co-workers to help me complete the main build of the system. Once the project is completed, we’ll go into a more broad, open development pattern that involves more people with an interest in the project or environment. The current group consists of me (Lead Experience Developer), Scott McAuliffe (Digital Experience Designer) and Allen Newton (Experience Developer).

In building the table, the first thing we did was a ton of research. There are several methods that can be employed to create a multi-touch environment:

  1. Frustrated Total Internal Reflection (FTIR). For this method infrared (IR) light is cast into an acrylic pane. The IR light becomes trapped inside the pane by internal reflection. When the surface of the acrylic is touched, the light is “frustrated” and scattered downwards out of the acrylic. By placing an inexpensive IR camera beneath the acrylic, the touch points can be detected and processed. To improve the touch sensitivity and feel, a “compliant surface” (usually a vellum sheet coated on one side with silicone) is placed on top of the acrylic pane.
  2. Rear Diffused Illumination. Using this method, IR light is shined at a glass, plexiglass or acrylic screen from below at a diffuser that is placed on top of or below the touch surface. Objects touching the surface reflect more light than the diffuser, and an IR camera is once again used to detect the extra light. A “Front Diffused Illumination” setup casts IR light at the screen from above the touch surface rather than behind.
  3. Laser Light Plane (LLP). To implement this method, a plane of laser light is cast just above the touch surface (acrylic, glass, plexiglass, etc.). The laser light plane is about 1mm thick  (about 10 times the thickness of an average human hair). When a finger intersects the laser light plane, the scattered light is cast through the touch surface and registers as an IR blob to a camera beneath the touch surface.
  4. Diffused Surface Illumination (DSI). This technique is similar to FTIR described above, but uses a special acrylic that contains particles that reflect and spread the IR light cast into the acrylic to the surface.

We’ve elected to go with the FTIR setup as it seems to be one of the easier and faster methods to employ, and is not terribly expensive. There are some downsides, which we will explore later as development advances, but that’s part of the learning experience.

For now, I’d only ask that you sit back and enjoy the knowledge we are able to share. I will be posting updates on this blog as we make progress. Also, if you’re at Waggener Edstrom and would like to check in on our progress or see it when it gets to an interesting point, feel free to swing by one of our cubes (we’re all on the fourth floor of the Portland office).

Thanks to the efforts of the Natural User Interface group at NUI Group for the wealth of information they share on touch computing (and where the descriptions of the above types of multi-touch systems came from).

← Back to WaggenerEdstrom.com