This presentation shared at the Vancouver AR Dev Camp 2010 covers the work being done by Maribeth Back's team at FXPAL for the TCHO Chocolate Factory.

1. The Virtual Factory
Maribeth Back, Tony Dunnigan, Jonathan Foote, Don Kimber, Bee
Liew, Jun Shingu, Jim Vaughan, Dave Arendash

2. Intro: FX Palo Alto Laboratory
FXPAL is an advanced R&D laboratory located in Silicon
Valley
– Roughly 50 people (25 PhD researchers)
– Research expertise
• multimedia systems & applications, user interface design & evaluation,
distributed networks and systems, and information systems
– Wholly owned by Fuji Xerox (FX)
• which is owned by Fuji Film (75%) and Xerox (25%)
– FXPAL is not PARC (they are a sister lab, owned by Xerox)
• But we have similar research missions: create patents and IP,
publications, products, prototypes…

3. Virtual Factory overview
Vision
Apply FXPAL collaboration technologies in a real-world factory setting.
Innovate in real/virtual collaborations, monitoring, and control
technologies based on real users and real-world problems.

4. TCHO Chocolate Factory
• Located on Pier 17, SF; factory opened Fall 2008
• Positioned as high-tech chocolate: accuracy in recipe invention
and replication
• Award-winning chocolate makers (already!)
• One of only a few bean-to-bar chocolate factories in US
• Management is tech-savvy and willing to experiment: CEO is
Louis Rossetto (founder of Wired)

5. Virtual Factory / Industrial Collaboration Environments
Create mixed reality spaces for simulation, visualization,
and collaboration
– Integrated real-time 3D, 2D, and mobile layers of information about
factory processes
– Sensor and controller data, multi-camera surveillance, image
processing, machine histories
– First instances: a 3D virtual factory floor in a multi-user virtual world,
and an iPhone/VR laboratory control application
Six types of users, with subsets of each:
– Managers (CEO, operations manager need different info)
– Operators (technical, mechanical)
– Tour guides (real-life, VR)
– B2B clients (technically knowledgeable)
– Suppliers and sources (cacao farmers and producers)
– Consumer customers (real-life, Web, VR)
Collaborative applications for:
– Remote factory observation and control
– Process/workflow monitoring and analysis
– Planning and design analysis
– Quality management
– Virtual inspections
– Tour guides (real-life, VR) for B2B or general public
– Education and training of employees
– Inventory tracking

6. Virtual Factory: real-time data integration
Multi-user virtual “mirror world” of factory
Data Spots, embedded HD video and machine animations appear upon avatar approach
Factory control software Wonderware into mixed reality architecture
– Wonderware InTouch: same software as used in some FX factories
– Detailed, spatially distributed DataSpots, Data History graphs appear when clicked
– Archiving and playback of factory data

7. DataSpot detailed sensor displays

8. Mixed Reality Lab Controller
Remote monitor/control/collaboration system for chocolate making
machines in the TCHO Development Lab. Monitor and control power and
temperature for melangers, with live PTZ video of lab to confirm command
execution. All driven from same database and python script set that
controls the Virtual Factory. The system is now in daily use.

9. Mobile remote control on iPhone
Monitor/control/collaboration system for keeping
track of machines in chocolate development
laboratory
iPhone, Web and Virtual World instantiations
– All driven from same database and python script set
– Atomic: most recent command wins, from any
interface
Use parameters (from interviews and observation):
– Monitor and control power and temperature for
melangers (mixer/grinders)
– Monitor and control temperature for roasting ovens
– Capture temperature data over time to database
– Video camera in lab to confirm control
Now in daily use at TCHO Chocolate
Laboratory

10. Hardware setup in Flavor Lab
Comtrol ethernet/serial Serial isolated I/O
10/100 ethernet 8-relay module
iPhone web app
Omega temp sense/set (serial)
for melangers and roasters AC power switches
(x 8) on melangers and
lab light (x 6 + 1)

11. FXPAL/TCHO project - recent press
Apple.com Discover Online
Wired UK
KQED TV
FXPAL CTC Report (April
CNET Slide 11
2009)

12. Thanks!