Multi-touch 29 posts

  • 26 February 2012

    Posted by Frank Garofalo | Topic: RIAs, User Experience, Miscellaneous, Multi-touch

    SXSW Interactive 2012

    I am excited to have the opportunity to attend SXSW Interactive 2012. Follow me on Twitter: @fgarofalo

    Last year was my first time attending where I was able to learn best-practices from others, network with others in the tech industry, and make some friends.

    Add a Comment

  • 6 February 2012

    Posted by Frank Garofalo | Topic: Multi-touch

    Corning & Multi-Touch

    This video was posted on Mashable (mashable.com/2012/02/03/day-of-glass/). Corning has a very interesting view of the future. I love the group collaboration with the multi-touch table - very similar to my grad research.

    Add a Comment

  • 20 July 2010

    Posted by Frank Garofalo | Topic: Multi-touch

    Adobe AIR and Multi-touch for Multi-user Collaboration

    Below is an article I wrote for the Adobe Developer Connection (Education Category). The full article, published July 2010, can be found at: http://www.adobe.com/devnet/edu/articles/frank_garofalo.html

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

    "Multi-touch" technology has generated a large amount of buzz in the past several months. Some of the buzz is about mobile/handheld devices, and some has come from broadcast media's adoption of touch devices for their programs (such as CNN's "Magic Wall," NBC's Saturday Night Live, and ESPN). This has resulted in the term "multi-touch" being used to describe a broad range of interaction types, interfaces, and devices. Most of the uses for touch interfaces that I've seen have involved a single user interface allowing only one or two touch points at a time through tapping the screen or through gestures. However, the true capabilities of a "multi-touch" interface can accommodate multiple users interacting through any number of taps or gestures on the screen at the same time.

    Multi-user Multi-touch

    For the purpose of this article I do not refer to "multi-touch" to describe a smart-phone or hand-held device that can recognize only one or two points of touch simultaneously, such as a Motorola DROID; I refer to larger devices like the Microsoft Surface product that can accommodate multiple users and recognize points of touch from several users simultaneously (at least three simultaneous points of touch).

    The research conducted as part of my graduate studies in the Department of Computer Graphics Technology at Purdue University (www.tech.purdue.edu/cgt/) has been a qualitative examination of multi-user collaboration with multi-touch devices. The study explored the experience of users performing a common task in a shared environment (in this case the shared environment is the single display screen of the multi-touch device). For more details about this research study visit www.multiusermultitouch.com.

    Digitally Replicating a Six Sigma Brainstorming Exercise

    For my research, I designed and built my own multi-touch table (which is currently pending a U.S. Patent) that could showcase the interface I developed with the Adobe Flash Platform.

    For the hardware, I used an open-source software package called Community Core Vision (CCV), created by the NUI Group, to serve as a liaison between the hardware components and the Adobe Flash Player. I developed an Adobe AIR application to recreate a brainstorming exercise used within Six Sigma, known as both "Affinity Diagrams" and "KJ Analysis." Typically in Affinity Diagram exercises, participants anonymously record and submit individual ideas or concepts on separate sticky notes. Then the participants sort through the sticky notes, grouping and categorizing them by moving them physically. Through this process, unforeseen relationships between the ideas can emerge. Since this activity requires participants to interact in a shared work environment in the physical world, I chose to replicate it in a multi-touch environment to explore the implications for a multi-user interface.

    Exploring the Multi-user Interface

    A moderator with Six Sigma Green-Belt certification and experience working with both corporations and universities served as a facilitator for the exercises performed during user testing. I selected Adobe AIR as the client since the application could run natively on a laptop computer connected to the multi-touch table, and chose the Adobe Flash Platform to capture the experience of multiple users simultaneously interacting with multi-touch interface and hardware. In the AIR application, users were able to move virtual sticky notes on the multi-touch table, and perform the same grouping and categorizing exercises that people do with physical sticky notes in an Affinity Diagram exercise.

    While both the hardware and software of the multi-touch device created a few challenges for the participants on various levels, including occasional software failures, they were able to complete the exercises. That benefited not only this study, aimed at collecting qualitative data from the participants, but also the participants themselves, who were able to take the results of the Affinity Diagram exercises back to their respective organizations on the Purdue campus.

    Affirming Research Assumptions

    My goal was to determine whether a multi-touch interface and device can be a practical shared environment for a team of individuals to complete a common task. The participants indicated that for completing an Affinity Diagram exercise, the multi-touch device proved to be a useful tool and believe that it improved collaboration. The moderator for the Affinity Diagram exercise was surprised at the speed with which the participants in both rounds completed the tasks, despite a learning curve for them to become familiar with the multi-touch device. She remarked that she has "not done one on paper that quickly." Furthermore, participants in the study expressed appreciation for the usefulness the multi-touch device provided. It allowed them to see all the data in front of them at once, and to see most of the actions of their fellow participants.

    About the author

    Frank Garofalo is a creative developer/technical designer for online media. His professional experience includes operating his own web & interactive multimedia firm named Cyber View (www.cyberviewsites.com) since 1999. In 2002 he began designing and developing with Adobe Flash. He graduated from Purdue University with a degree in Computer Graphics Technology in 2008. Additional professional experience includes employment with the Walt Disney Company and Bank of America. To date his graduate studies at Purdue University have focused on multi-touch device interfaces built on the Adobe Flash Platform and he has examined multi-user collaboration facilitated by multi-touch devices. His blog and portfolio can be found at www.frankgarofalo.com.


    Add a Comment

  • 13 July 2010

    Posted by Frank Garofalo | Topic: Multi-touch

    Semi-Finalist in Adobe Design Achievement Awards

    2010 Adobe Design 

Achievement AwardsThe submission of the Adobe AIR application I developed as the user-interface for my graduate research project has been selected as a semi-finalist in the Adobe Design Achievement Awards. Below are my submission details:

    • Submission Category: Non-Browser-Based Design
    • Submission #: 484
    • Entry title: Multi-user Collaboration
    • Number of Pieces: 1

    For more details about the Adobe Design Achievement Awards, visit: https://adaa.adobe.com/us/content/adaaLive

    Add a Comment

  • 12 July 2010

    Posted by Frank Garofalo | Topic: Multi-touch

    Success! Grad Research Complete

    Today I gave my defense presentation to my graduate research committee and the professors signed-off on my research report and passed me! I get to graduate in August with a Master of Science in Computer Graphics Technology.

    Add a Comment

  • 11 July 2010

    Posted by Frank Garofalo | Topic: Multi-touch

    Defense Presentation

    My Defense Presentation is tomorrow and I'm ready! The Defense will take place at 1pm EST in room Knoy 373. The presentation slides are done and I've reviewed them several times already.

    Grad school has been an interesting and challenging journey. I'll openly admit that there were some points along the way I wasn't sure if I was going to see the end. I'm glad it's finally coming to a conclusion. I have learned a significant amount not only about my research topic and methods for conducting qualitative research studies, but also I've learned more about myself... the cliche of "when the going gets tough, the tough get going" comes to mind.

    I have several "thank yous" to give out for my sincere appreciation:
    • My parents and brother
    • Robert Brophy
    • Dr. James Mohler
    • Prof. Rodney Vandeveer
    • Prof. Terry Burton
    • Prof. Gail Farnsley
    • Dr. Mary Sadowski
    • Julie Talz
    • My Resident Assistant Staff Members at Cary Quad from the past few years
    • ... and all my friends who have stood by me (and let me vent/rant at times!), you know who you are
    Thanks!

    Add a Comment

  • 16 June 2010

    Posted by Frank Garofalo | Topic: Multi-touch

    Data Analysis

    I've completed transcribing the audio recordings of the 8 1-on-1 interviews and 1 of the 2 focus group sessions. Now it's time to start the data analysis...

    From all 8 interviews there were 5,846 words transcribed, but to get a visual of the most frequent spoken words from my testing participants (plus filtering out 'common English words') below is the result - thanks to Wordle:

    Add a Comment

  • 24 May 2010

    Posted by Frank Garofalo | Topic: Multi-touch

    User Testing - Round 2 Part 1

    Today marked the beginning of Round 2 of user testing with Part 1 - the Affinity Diagram exercise. From Round 1 (User Testing - Part 1) the voluntary participant number wasn't high enough so a second round of user testing was required. The testing today went very well overall. I'm glad to say that the system didn't crash, as it did during Round 1 - Part 1 (so the modification I made to the open source code seemed to resolve the issue that was occurring).

    The largest challenge with the user testing today seemed to be when the digital sticky notes would either move to the default top left corner (the 0,0 coordinate) or it would move to the location of the last touch point (which was usually another testing participants touch point). This seems to be an issue with Community Core Vision (CCV), the open source code, and the ActionScript 3.0 class which communication with CCV.

    Add a Comment

  • 1 May 2010

    Posted by Frank Garofalo | Topic: Multi-touch

    A Multi-user Collaboration Multi-touch Device

    The digital post-it notes being sorted included various popular social networking sites, such as Facebook, YouTube, Flickr, Twitter, etc. Special thanks for volunteering to be video taped - Sean S., Brian E., Ben T., and Mike M.

    Add a Comment

  • 29 April 2010

    Posted by Frank Garofalo | Topic: Multi-touch

    Provisional Patent

    On April 23rd 2010 a provisional patent application was filed with the United States Patent and Trademark Office (USPTO). The provisional number is 61/327,354 with Frank J. Garofalo and James L. Mohler listed as the inventors, and the Purdue Research Foundation (http://www.prf.org/otc/) as the assignee.

    Add a Comment

  • 20 April 2010

    Posted by Frank Garofalo | Topic: Multi-touch

    User Testing - Part 1

    Yesterday (April 19th 2010), I started part 1 of the user testing for my research project regarding the multi-touch device and software application I've been developing. The software application is based on the Six Sigma Affinity Diagram (aka KJ Analysis) exercise. This is typically used to sort through qualitative data (such as ideas from a brainstorming session) to find unforeseen relationships between the the data items / ideas by grouping them into categories.

    I was granted permission by Dr. Mary Sadowski, Associate Dean for the College of Technology at Purdue University, to ask faculty members from a committee within the College of Technology to use my multi-touch device and software application to aide them in sorting through some ideas/outcomes regarding a project they are working on.


    Image 1 - Doing a run-through prior to user testing during set-up.


    Image 2 - User Testing.

    During the user testing the software application unfortunately crashed twice. Each time the users had to restart the Affinity Diagram from the beginning. After the second crash, I went into the code and quickly made some alterations. The problem was occurring with an open-source class written in ActionScript 3.0 which processes data received from Community Core Vision (the open-source software which interprets the images received from the camera component of the device's hardware). I added some "try / catch" statements. This seemed to resolve the issue since the software application didn't crash after the modifications.

    Overall, the user testing proved to be successful. Excellent data was collected both regarding the interaction of user-to-user and user-to-interface. Also, notes for improvements to the software application.

    Add a Comment

  • 7 April 2010

    Posted by Frank Garofalo | Topic: Multi-touch

    Finally, creating multi-colored post-it notes

    I originally hoped to use the ColorPicker component in Flash but I was having too many problems trying to modify the code behind the component to get it to recognize the Touch Events from CCV and TUIO. For the sake of time and meeting my approaching deadline, I decided to ditch that idea and just make a set number of colors that will automatically be selected as groups/categories are created.

    Below is an example of what I've come up with so far.



    Still more to come, next is the ability to vote on the priority of categories/groups.

    Add a Comment

  • 23 March 2010

    Posted by Frank Garofalo | Topic: Multi-touch

    Making Slow Progress... but progress

    I've made progress on my multi-touch software. I can now have multiple assets displayed on the device being touched (selected) at the same time and have events respond respectively to each individual asset selected. Getting this to work was a major accomplishment. Thank goodness for error catching.



    Now I've run into another problem trying to save over an existing XML file with updated data. Spent about 3 hours this evening trying to figure it out without much success. I've searched Google to see what other people have tried but I've yet to find a working solution... slightly frustrating. If I find one, I'll post about it.

    Add a Comment

  • 16 March 2010

    Posted by Frank Garofalo | Topic: Multi-touch

    Adobe AIR & Multi-touch?

    Adobe Flash 

PlatformIt is possible to create an Adobe AIR app for a multi-touch device. Using TUIO and Community Core Vision (CCV), I've been able to create an AIR app to run on the multi-touch device I've built.

    In my case, the AIR app has to be developed to specifically recognize TouchEvents from TUIO and CCV. More details coming soon!

    Add a Comment

  • 15 March 2010

    Posted by Frank Garofalo | Topic: Multi-touch

    Device & Software Update

    So now that the hardware of the multi-touch device is essentially done (there are some things I could probably do to further improve it), I've moved on to the software development side. I've been working in Adobe Flash CS4 with ActionScript 3.0. It's been a lot of trial and error testing code as I go. However, I'm making progress - just slow progress.

    One of the major hurtles was a problem I ran into with embedFont = true; making the dynamic textfields disappear. But I found a solution and blogged about it - link.

    Add a Comment

  • 2 February 2010

    Posted by Frank Garofalo | Topic: Multi-touch

    I've created a multi-touch monster

    The multi-touch device I've been building since June is finally working now. There can be some tweaking and calibrating to improve its performance, but at least now its working.



    Above pictures are using demo apps from the NUI Group.

    Add a Comment

  • 16 January 2010

    Posted by Frank Garofalo | Topic: Multi-touch

    Multi-touch Stand

    Received the materials to build my multi-touch stand this past Thursday. There were a few minor issues with the materials and design, so now I need to wait for the supplier to me some fabricated pieces. It's a little frustrating right now since it's so close. I can't wait to get the hardware components to a completed stage (...aka satisficing) so I can focus solely on the software ui with Adobe Flash.

    Add a Comment

  • 22 October 2009

    Posted by Frank Garofalo | Topic: Multi-touch

    Planning to build a frame

    Now I'm trying to determine the best materials to use to build a frame. Right now I rigged up a bookcase that I bought at Target to support the acrylic. Someone recommended using aluminum frame from Bosch Group Rexroth (http://www13.boschrexroth-us.com/framing_shop/Default.aspx ). I like this idea because it seems like you can make adjustments to the structure compared to building a wooden frame. I got the idea to use Bosch Group Rexroth from a multi-touch project by Dominik Schmidt (http://eis.comp.lancs.ac.uk/~dominik/cms/).

    Add a Comment

  • 22 October 2009

    Posted by Frank Garofalo | Topic: Multi-touch

    DSI Multi-touch with Enlightened Acrylic

    My sincerest thanks to Mark Miller from Miller & Associates. I met Mark at the Adobe MAX 2009 conference in Los Angeles, California. His company had teamed up with Adobe to build a multi-touch device for the conference to showcase using Adobe Flash for multi-touch devices. He was generous enough to provide me with a sheet of enlightened acrylic outfitted with an aluminum track and infrared LEDs. At Adobe MAX you truely can "connect, discover, and inspire." Here are some pictures of the enlightened acrylic with CCV running:




    Thanks again Mark for supporting a graduate student.

    Add a Comment

  • 16 October 2009

    Posted by Frank Garofalo | Topic: Multi-touch

    Flash & AS 3 for Multi-touch

    Today I spent time working on AS3 code for my multi-touch device. I found the following links very helpful:

    A few days ago I also changed the configuration of my mutli-touch device from a tabletop (horizontal) orientation to a wall (vertical) orientation. I mainly did this after the PS3 Eye camera arrived to see if the PS3 Eye would detect the hot spot created on the acrylic from the projector's light beam. I was pleased to see that the camera didn't detect the hot spot.

    Add a Comment

  • 12 October 2009

    Posted by Frank Garofalo | Topic: Multi-touch

    Build Supplies List

    Here is my build supplies list:
    • Projector: Sharp Front-Projector XR30-S (Short Throw)
    • IR LEDs: Narrow Dimmable LED Ribbon Flex by the foot (High density infrared 850nm) from EnvironmentalLight.com
    • LED Power Supply: 60 Watt 12 VDC Power Supply from EnvironmentalLight.com
    • IR LED Connector: Ribbon Flex connector - 2 conductor ribbon to cable from EnvironmentalLight.com
    • Power Supply Plug: Male Mini Power Supply Plug and 20" Cord from EnvironmentalLight.com
    • Frame: Aluminum Deep U-Channel 98in (0.5in thick) long brushed Nickel from EnvironmentalLight.com
    • Frame End Caps: 4 x End Cap for Deep U-Channel brushed Nickel from EnvironmentalLight.com
    • LED Setting Blocks: 4 x 4" Neoprene Setting Blocks from EnvironmentalLight.com
    • Projection Surface: Rosco Grey from RoseBrand
    • LED Power Supply: 60 Watt 12 VDC Power Supply from EnvironmentalLight.com
    • Mirror: (need to get a front-surface mirror)
    • Compliant Surface: Transparent Silicone Rubber .020"x24"x36" from Rubber Sheet Roll
    • Plexiglass: 18" x 24" Clear Acrylic Plexiglass - 1/2" think with polished edges from Meyer Plastics, Inc
    • IR Camera: Camera + m12 Mount + IR BP Filter (custom) filter wavelength 850nm from Peau Productions

    Add a Comment

  • 12 October 2009

    Posted by Frank Garofalo | Topic: Multi-touch

    FTIR Multitouch with PS3 Eye Camera

    My new PS3 Eye camera arrived to replace the other infrared camera I had been using. The PS3 Eye was modified to capture infrared light. After swapping out the camera, my multi-touch device was responding much better.


    I have also posted this video on the NUI Group Forums: http://nuigroup.com/forums/viewthread/7282/
    The camera was purchased from Peau Productions: http://www.peauproductions.com/blog/



    Add a Comment

  • 1 October 2009

    Posted by Frank Garofalo | Topic: Multi-touch

    Multi-touch Device Progress

    Some progress was made with the multi-touch device I've been building. Using the CCV software and some demo Flash apps provided on the CCV web site, here are some pictures and a video:










    Thanks to Zach, Ben, and Caleb.

    Add a Comment

  • 27 September 2009

    Posted by Frank Garofalo | Topic: Multi-touch

    Resource

    I've found, in addition to the NUI Group Forum (www.nuigroup.com), the Touch Factors Blog to be a very helpful resource: http://www.touchfactors.com/?p=103

    Add a Comment

  • 24 September 2009

    Posted by Frank Garofalo | Topic: Multi-touch

    New Silicone sheet on Acrylic - Attempt 2

    Ok, so attempt 1 was a learning process. Now here we go with attempt 2.


    Add a Comment

  • 24 September 2009

    Posted by Frank Garofalo | Topic: Multi-touch

    Removing Silicone

    I decided to try to remove the silicone layer on the acrylic. I purchased a plastic scrapper, similar to that used for removing decals from vehicles. The instructions for the scrapper stated to use water or a solvent. I tried both and only was able to get up small fragments. After talking to the maintenance staff at my work location, they said that the silicone has most likely set on the acrylic and its not going to come off easily without damaging the acrylic (which the acrylic would need to remain unscratched and transparent).



    I gave up the idea of removing the silicone and decided to purchase a new piece of acrylic.

    Add a Comment

  • 16 September 2009

    Posted by Frank Garofalo | Topic: Multi-touch

    Possible issue with silicone layer

    I think there is a possible issue with the silicone layer of my multi-touch device. I had tried to do the method described by Tickerman on the NUI Group Forum (http://nuigroup.com/forums/viewthread/2197/P0/; YouTube Video: http://www.youtube.com/watch?v=IEgoVBfF68Y&feature=player_embedde d). Tickerman calls it the "siliconed vellum" method.

    I was unable to get the silicone completely smooth and it seems that the infrared camera I have is having difficulty distinguishing the blobs created by fingers in contact with the mt surface and the impurities/imperfections of the silicone.


    Screen capture of CCV without contact to the MT surface


    Screen capture of CCV with five fingers in contact on the MT surface

    Add a Comment

  • 16 September 2009

    Posted by Frank Garofalo | Topic: Multi-touch

    Building a Multi-touch Device

    I've started building my own multi-touch device after reading several posts of blogs and forums through the NUI Group web site (www.nuigroup.com). I complied a list of commonly mentioned components and ordered them on-line. If anyone wants the list I can post that on here.




    Yesterday I tried to run my multi-touch device with the Community Core Vision (formerly tBeta) software from the NUI Group. However the software wasn't detecting the blobs created by the fingers in contact with the surface of the device. It seemed that it was picking up too much heat reflecting off the backside of the acrylic/Plexiglas. I also tried to use a mirror to bounce the projector's image to the backside of the multi-touch device with the infrared camera next to the mirror to see if this would help with the situation. However, there wasn't a noticeable difference. Back to the grindstone to figure out what's wrong and why it isn't working correctly.





    Add a Comment

  • 16 April 2009

    Posted by Frank Garofalo | Topic: Multi-touch

    MS Surface

    Today I had an opportunity to meet with professors in Purdue's Engineering Education department who have a MS Surface. Spent about 3 hours meeting with them. It was great to play with a multi-touch device that was professionally manufactured.

    I was surprised that the multi-touch capabilities are only available when in user mode which seemed to run a Silverlight app that serves as a core menu to launch other apps. Already having previous knowledge, from researching, about gestures for multi-touch devices I still found the gestures very natural and intuitive. When I was using the device by myself, I found the responsiveness of the device and applications to be acceptably accurate. There were a few apps, such as the MSNBC News Sphere to be difficult to turn the sphere. During this time, we at most had three individuals interacting with the Surface and the responsiveness was good.

    Our goal is to test using Adobe Flash and Adobe AIR on the device utilizing the t-Beta protocol (found on the NUI Group web site: www.nuigroup.com). Since the device is not ours, we are waiting for permission to install the t-Beta program on the device. I'm excited to see how the device will respond to a non-Microsoft platform. There will be more blog posts in the future once we move forward with testing Flash on the MS Surface.

    Add a Comment