Building a Multi-Touch Table: Converting the Webcam

Posted on April 5, 2010 by Comments Off

The style of multi-touch table that we are building relies on the “seeing” of infrared light to pick up IR reflection and subsequently use that information to convert those reflections to “touch points” (for details, see  Multi-Touch Table Kickoff). To accomplish this we converted a webcam for this specific use.

The PS3Eye webcam in it's many parts.

Digital image sensors can inherently “see” IR light, but you’ll find that most cameras and camcorders will contain an IR blocking filter that blocks the IR light before it reaches the sensor, which minimizes IR distortion in the final image. This is a good thing for a majority of the intended audience, though sometimes, whether for creative reasons or geeky, tech reasons (in this case), we want to be able to see that IR light. To do this we need to remove the IR filter from our selected camera, the PS3 Eye. Once that is done, IR light is allowed to pass through the lens and onto the sensor. However, we want to fully isolate what the sensor sees so that it is only IR light. This is done by then filtering out the visible light coming through the lens. We can either do this by adding a more expensive bandpass filter to the camera, which only allows a certain wavelength or “band” of light to pass through it, or go the less expensive route of using a piece of floppy disk to act as our bandpass filter. While not as clean at filtering visible light, the floppy disk does a more than adequate job for our purposes. Plus, we now have a use for that electronic copy of my old undergrad thesis, which is, thanks to the era in which I got my degree, backed up on a floppy disk. Here’s a sped-up version of the conversion process:

A close up of the IR filter in the lens

In short, we replaced the camera’s IR filter with a bandpass filter to allow only IR light to pass through it. At this stage we were able to test how well this conversion worked by connecting the camera to the computer and positioning it facing the table’s acrylic top while up on its side. We turned on our 850nm IR lights, bouncing light into the acrylic, and once that acrylic was touched it created a reflection point for the IR light, which was immediately visible on the computer monitor. Not too shabby. 

 The IR camera in action reading touch points

If you would like to see the detailed video of how this is done I’ll direct you to the wonderfully informative PeauProductions tutorial.

Building a Multitouch Table: Shopping for Computer Parts

Posted on February 24, 2010 by Comments Off

Our multitouch table requires the use of a fairly powerful computer. Video processing is of course something that we want it to handle really well. There are a few open source touch API (application programming interface) solutions on the Internet, and each requires processing video and data at blistering speeds. You can use either an Apple or a PC.  Because we want to take advantage of the Windows 7 Touch capabilities, we decided to go with a PC on this round.

Usually the least expensive way to get such a PC is to build it yourself. We are lucky in the Silicon Forest to have more than a few small businesses in the area that sell computer parts. There are other avenues on the Internet like NewEgg.com. Like anything over a few hundred dollars, deals are just waiting to be found. Buying parts for a computer generally means you know what to buy. Luckily there is a lot of good information on the Internet for that.

Tom’s Hardware is a great resource for that kind of information. Here is a link to a step-by-step set of articles on the subject: Building a PC. This post isn’t about the step-by-step process but rather is a summarization of our shopping experience. If you would like to try building one yourself, please do read that series.

The list of parts is fairly short. CPU (central processing unit), mainboard (or “mother-board”), RAM (random access memory), video card, hard drive, networking card, DVD/CD-ROM drive, case, case fans, power supply, CPU fans, keyboard, mouse, and monitor. In years past, you would be looking for a 3 1/4-inch floppy drive as well. We have it so good now that those days are behind us!

The parts are listed here in the order they should be considered. Usually the CPU, mainboard and RAM are tightly integrated components. Meaning that once you decide on the CPU, the rest of the parts are specific to the CPU. The CPU drives almost everything else down the list.

Intel and AMD are still the two big CPU makers, but lately we’ve seen performance advantages using Intel processors (CPUs). It is always a good idea to start with the latest and most expensive model. Then work backwards in terms of performance to get to a price you feel good about. We found information about the CPU on the Internet. There is a good comparison grid on Wikipedia here: Comparison of Intel processors.

We decided that the i5 was a good enough processor for the needs of this project, and the price was not too bad. With the CPU choice complete, we were able to shop for the rest of the parts that fit and we came up with a spreadsheet. Here is the parts list:

table

Putting together a desktop is a rewarding and fun experience. Good thing we decided to avoid all of that and buy one preassembled from a vendor! It turned out that even after all this due diligence we weren’t able to beat the price that Jeff Paries found at the beginning of our search. If nothing else, the comparison shopping proved that the deal he found was the best.

We did have to wait a few weeks for the PC to arrive, but for a couple hundred dollars savings it was worth the wait.  It’s here now, and we started installing the Windows 7 operating system and all the associated multitouch software.  In the next couple of weeks the team hopes to have a working prototype to begin writing some demo applications for.  Exciting stuff!

Building a Multi-Touch Table: The Cabinet

Posted on January 25, 2010 by 1 Comment

In my last post, I introduced an internal project aimed at creating a multi-touch computer from the ground up. I introduced the project and talked a little bit about the method we chose to work with (FTIR).

In an FTIR setup, IR light is cast from the edges into an acrylic sheet where it gets trapped, bouncing around within the pane. A “compliant” surface (typically a vellum sheet with silicone on one side) is placed on top of the acrylic. The compliant surface acts as a projection screen for a projector placed inside the multi-touch cabinet. In addition, when touched, the silicone squishes against the glass, scattering the light out the bottom of the acrylic sheet, where it is picked up by an IR camera. The IR camera is attached to a computer, and the touch points are converted to “blobs” via an open source API. Depending upon the API and methods being used, the blobs have size, direction, velocity, etc.

All of that sounds really cool, but we’re getting a little ahead of ourselves. We need something to house all this good stuff. Given a limited budget and the fact that we’ve never done this before, we’re not going to spend a huge amount of time or budget on the cabinet. We agreed we wanted a stand-up table rather than a sit-down type, so with all of this in mind, I used Google SketchUp to create a design for the cabinet, shown here.

Multi-touch Cabinet

The desks we have in our office have a crank that allows us to raise and lower the height. We tested a few heights, starting with a normal kitchen counter — around 36 inches. This seemed a little too low for tall people, so we bumped it up to 38 inches, which seemed to work well in informal testing for a variety of people from 5 feet 1 inch to 6 feet 3 inches. Our touch surface will be 24×30.

Whether or not we go with pressboard is still up for debate, but I’m in favor of painting the cabinet black with a flame job… have other ideas? Know somebody that wants to make us free furniture in exchange for a signed copy of my book? Let me know in the comments or by e-mail! Just remember, we’re on a limited budget…

Building a Multi-Touch Table: Kickoff

Posted on January 25, 2010 by 2 Comments

Waggener Edstrom gives employees the opportunity to create something we call an Experience Influence Project, or EIP. This project is a self-paced, self-designed learning opportunity, and can be done as an individual or group effort. The real goal is to bring some learning to the agency in a way that expands our understanding of influence.

We had an idea floated several months ago that I decided to follow up on for my EIP — building a multi-touch table from scratch. Touch computing has really taken off and doesn’t seem like it will be going away anytime soon; most cell phones support single- or multi-touch, Windows 7 supports multi-touch on tablet PCs, Apple has a tablet PC coming soon according to long-standing rumors, and there are high-end multi-touch systems like the Microsoft Surface table. The goal of my EIP project is to build a multi-touch table along the lines of the Surface computer on a limited budget (the cost of a Surface is north of $10,000).

I started out by gathering a small group of co-workers to help me complete the main build of the system. Once the project is completed, we’ll go into a more broad, open development pattern that involves more people with an interest in the project or environment. The current group consists of me (Lead Experience Developer), Scott McAuliffe (Digital Experience Designer) and Allen Newton (Experience Developer).

In building the table, the first thing we did was a ton of research. There are several methods that can be employed to create a multi-touch environment:

  1. Frustrated Total Internal Reflection (FTIR). For this method infrared (IR) light is cast into an acrylic pane. The IR light becomes trapped inside the pane by internal reflection. When the surface of the acrylic is touched, the light is “frustrated” and scattered downwards out of the acrylic. By placing an inexpensive IR camera beneath the acrylic, the touch points can be detected and processed. To improve the touch sensitivity and feel, a “compliant surface” (usually a vellum sheet coated on one side with silicone) is placed on top of the acrylic pane.
  2. Rear Diffused Illumination. Using this method, IR light is shined at a glass, plexiglass or acrylic screen from below at a diffuser that is placed on top of or below the touch surface. Objects touching the surface reflect more light than the diffuser, and an IR camera is once again used to detect the extra light. A “Front Diffused Illumination” setup casts IR light at the screen from above the touch surface rather than behind.
  3. Laser Light Plane (LLP). To implement this method, a plane of laser light is cast just above the touch surface (acrylic, glass, plexiglass, etc.). The laser light plane is about 1mm thick  (about 10 times the thickness of an average human hair). When a finger intersects the laser light plane, the scattered light is cast through the touch surface and registers as an IR blob to a camera beneath the touch surface.
  4. Diffused Surface Illumination (DSI). This technique is similar to FTIR described above, but uses a special acrylic that contains particles that reflect and spread the IR light cast into the acrylic to the surface.

We’ve elected to go with the FTIR setup as it seems to be one of the easier and faster methods to employ, and is not terribly expensive. There are some downsides, which we will explore later as development advances, but that’s part of the learning experience.

For now, I’d only ask that you sit back and enjoy the knowledge we are able to share. I will be posting updates on this blog as we make progress. Also, if you’re at Waggener Edstrom and would like to check in on our progress or see it when it gets to an interesting point, feel free to swing by one of our cubes (we’re all on the fourth floor of the Portland office).

Thanks to the efforts of the Natural User Interface group at NUI Group for the wealth of information they share on touch computing (and where the descriptions of the above types of multi-touch systems came from).

← Back to WaggenerEdstrom.com