The Artiﬁcial Technology Center AiDigital Library ™
Proprietary and conﬁdential July 2000 1Artificial, Inc.
Proprietary and conﬁdential July 2000 1Artificial, Inc.
Proprietary and conﬁdential July 2000 1Artificial, Inc.
Author: Daniel Delgado Aug. 31, 1999 Revised: Gordon H Kraft July 5, 2000 Oct. 9, 2001 (c) COPYRIGHT 1999 - 2001 Artificial, Inc.Proprietary and conﬁdential July 2000 1Artificial, Inc.
SECTION I: The Artificial Technology Center™ with an AiHome™What is the Artificial Technology Center?The Artificial Technology Center (ATC) is to be a new and unique research anddevelopment laboratory. Its purpose is to advance and promote the technology of high-speed (Broadband) Internet applications. The center will advance the technology with aninnovative research and development series of projects. It will promote the technology bydemonstrating and educating the public as to the new possibilities offered by a broadbandInternet (Bi). The ATC will use a constant flow of the latest PC and Audio/Videoequipment and systems. The ATC will be built inside of a normal Executive OfficeBuilding with the construction of a partial house (approximately 1200sqft.) with easy tovisualize Smart House wiring and the ability to use the ATC house called a AiHome™ asa retail store, a live development lab, and a classroom.Why is it needed?Proprietary and conﬁdential July 2000 1Artificial, Inc.
The Center is needed for one primary reason - to develop profitable systems for theInternet of tomorrow. All businessmen are aware of the difficulties in developing newproducts. Developing products for the Internet is even more challenging because no oneknows how the Internet will grow and what it will grow into. We do know that itsamazing technological advances have changed the rules of commerce, yet we do not yetknow what those rules have changed into. Because of this, developing creative andprofitable products is much more difficult on the Internet than in the past.The Center will be an environment designed to quickly produce new products andtechnologies. It takes advantage of the fact that advances in broadband technology willsoon turn the Internet into a high bandwidth network to the home. These changes willopen the door for a host of previously impossible applications. The Center solves theproblem of how to create these different applications and it also determines if they will beprofitable. It does this by building an environment where an Internet user can use the newapplications in a meaningful and productive way, and where at the same time, we canassess their usefulness and profitability. The ATC’s environment is available online viathe Bi and is called the AiDigital Library™.Proprietary and conﬁdential July 2000 1Artificial, Inc.
Gateway ComputersProprietary and conﬁdential July 2000 1Artificial, Inc.
SECTION II: The AiDigital Library™What is the AiDigital Library?The AiDigital Library is conceived as integration, both physically and in the abstract, ofa library with the World Wide Web as it’s bookshelf. The goal is have an environmentconsisting of library and web, which fulfills the functions of both in the most useful andhuman centered manner possible.We are building a library because a library, seen as a physical container of knowledge,has the same function as the Internet - today’s most successful and powerful container ofKnowledge. The Internet has its enormity and timeliness of information, hyperlinks, plusthe well-honed excellence of the various search engines The library, on the other hand,has the advantage of human comfort such as windows, chairs, desks, and space. Bothhowever have the same function, which is to serve us humans in our ability to acquire,manipulate, and create information.As the Center’s main tool for product development, the AiDigital Library will servemultiple roles as think tank, R&D facility, and technology showcase. By building it wehope not only to demonstrate the advanced capabilities offered by a high bandwidthInternet, but also to act as a bootstrap and catalytic agent for the development ofcommercially successful products.The AiDigital Library will require innovative hardware and software I/O devices andsolutions. The library must make one’s physical environment a powerful tool foraccessing and manipulating the multitude of information streams present on the Internet:such as HDTV video, surround sound, hypertext search engines, and 3D visual data sets.Proprietary and conﬁdential July 2000 1Artificial, Inc.
Conversely, it must also allow one to control the physical aspects of the library in such away as to facilitate one’s control of the Internet and it’s multimedia. Solving theseproblems, and in general building the library, will result in the profitable Internetapplication’s needed for commercial success.SECTION III: The Components OverviewThe Artificial, Inc. “AiDigital Library” is a ongoing research and development project aimed at providingthe user a multimedia digital library of all of their documents, photos, and home/corporate videos, as wellas the using the internet as our Global Infobase. How it’s doneThe computer selected is a SONY VAIO computer system with a Pentium IV, DVD, DV Firewire interface,Macintosh G4 DVDr system, VideoConference Camera, Color printer, Sony LCD flat panel screen, and aDigital Camera. The human interface is a local web site using our AI 3D-library QTVR look and feel; theuser can operate everything in the room. From the rack of the Home Theater equipment in the AI 3Dlibrary, the user can access the any of the equipment using a connection to the AV control system, as well ascruise the internet, or retrieve and show documents, movies, and images. The Control PC is connected tothe home Cat 5 network or wireless network and to the Internet via a Broadband T1/DSL or faster. Theperformance of the resulting AiHome must have the ability to transmit and receive full motion video in aminimum of MPEG4 format to a maximum of HDTV format.Proprietary and conﬁdential July 2000 1Artificial, Inc.
The use of various high capacity storage or Mass Storage Devices will be required to store and retrieve themassive amounts of digital information.Proprietary and conﬁdential July 2000 1Artificial, Inc.
AiLibrary, Inc. • A Sony Clie PDA is used as the Remote Control for the AiLibrary.Proprietary and conﬁdential July 2000 1Artificial, Inc.
AiLibrary, Inc. • A Sony Clie PDA is used as the Remote Control for the AiLibrary. • You click via finger or stylus the visual item area of interest.Proprietary and conﬁdential July 2000 1Artificial, Inc.
Sony AiDigital Library™Proprietary and conﬁdential July 2000 1Artificial, Inc.
Proprietary and conﬁdential July 2000 1Artificial, Inc.
Proprietary and conﬁdential July 2000 1Artificial, Inc.
PowerFile™Proprietary and conﬁdential July 2000 1Artificial, Inc.
Proprietary and conﬁdential July 2000 1Artificial, Inc.
Proprietary and conﬁdential July 2000 1Artificial, Inc.
Proprietary and conﬁdential July 2000 1Artificial, Inc.
PowerFile™ Mass Storage Facility capable of 1 Terabyte per unit(can be daisychained) connected via Firewire™.Proprietary and conﬁdential July 2000 1Artificial, Inc.
Proprietary and conﬁdential July 2000 1Artificial, Inc.
Macintosh AiDigital Library™ From imagination to realityProprietary and conﬁdential July 2000 1Artificial, Inc.
Digital Library Hardware/Software Configuration DiagramsThese diagrams show the information flow through the hardware and software of the Digital Library. Asoftware box is always inside a hardware box, so, for example, the Library Server is a large gray boxcontaining the boxes naming the server software, the AIQuery Engine, the Crestron web software, and theweb browser.An arrow means a physical connection with the direction being either unidirectional such as with the audio/video signal going to the Wide Screen Monitor (red), or bi-directional such as with the Library PersonalComputer Ethernet connection to the server (green).One thing to notice is that in both diagrams the software AIServer box extends across the hardware box.This is done to signify that the server is indeed the server, such that all interactions between the Internet andany software that needs input or output must pass through it.In addition, both the Portable Digital Library and the Digital Library are shown with independent hardwarefor the speech recognition. The portable one uses an interface card (which is why its box is inside theserver box), while the Digital Library uses an external serial device (blue). This is done in order to reducecomputational load on the server computer. Speech recognition is a real-time process so if the load is toogreat then recognition performance gets too poor. However we may find a software-only package thatavoids this. Using an external device also simplifies programming since the Digital Server then receives itsspeech as a serial ASCII stream.The ST-COM, ST-I/O, ST-VC hardware boxes are modules need to do serial communication, bit I/O, anddevice control (e.g. receiver volume) respectively.The Digital Library book is a display device with a touch screen used to manipulate web pages, and mustbe very simple and light. It does not by itself have any user input apart from that used in the touch screen.The main user control will be the touch screen with the display mirroring the server browser, and also theAI AudioResponse System (AiARS).The position sensors used on both libraries are for are for virtual windowing. One cannot use the RealityFusion camera method for this because it would be too slow. Several infrared (non-invasive) systems areavailable. Another benefit is that if for some reason the Reality Fusion method is not workable then we canalso use it to obtain the users position.What will the library consist of?The AiDigital Library will have a variety of software and hardware components. Theyare named the following: The Physical Library The Library Web Site The Library Query Engine The Library Server Advanced Concept ToolsThe Physical LibraryThe Physical Library is an actual library. The present one is located in Florida. Thelibrary is comfortable, well lit, spacious, with computer access and a private collection ofProprietary and conﬁdential July 2000 1Artificial, Inc.
books, records, tapes etc. It also contains media players such as a multimedia capablecomputer, a HDTV capable large screen monitor, and a stereo system. Regardless as tohow we construct the AiDigital Library, we will make sure it remains comfortable,spacious and well lit.The AiDigital Library is not meant to be a replacement for a traditional library, nor forthe creation of a ‘super’ high tech system with lots of bells and whistles. These may belaudable goals but they miss the point of the AiDigital Library. The goal is to produce amelding of a library with its Internet counterpart. To do so successfully, we must maintainthose qualities of a traditional library that makes it enjoyable and easy to use. Theseinclude in no small part those that make it comfortable, spacious, and well lit.Proprietary and conﬁdential July 2000 1Artificial, Inc.
The Library Web SiteAs with the Physical Library, the Library Web Site (version alpha) is active (http://www.ghklibrary.com/aidl). The Web Site will serve as the main conduit for the interactionsbetween the user, the Internet, and the Physical Library. Its most important page is a well-done QuickTime VR rendition of the Physical Library. This page will have controls andgateways to other control pages, which will allow the manipulation of the AiDigitalLibrary. For example, if one double clicks on the image of the stereo system, then a newpage comes up with information about the stereo system as well as controls to manipulateit. Likewise, if one double clicks on the image of the Wide Screen TV, information aboutyour video database and controls become available.Using a web site to control the library give certain advantages. We envision the AiDigitalLibrary as containing a variety of different displays placed at various locations in it.Using the web in this fashion allows the user to control any number of library Internetdevices with the same interface and on the different displays. As such, the web sitebecomes a universal controller for the user.The current Physical Library uses software from Crestron Home Control Systems (http://www.crestron.com/website/index.html) for controlling the home. This software will alsobe used for the control of the library from the web site. Its use will save us developmenttime and assure compatibility with the house. Most importantly, the system is ideal forweb hardware interfacing since it has been specifically designed for Internet control. Inaddition, it is a Java based system, which makes programming for the library mucheasier.The Library Web Site user interface, although doable in Java, will require the use ofShockwave and Director. Shockwave was used for the construction of the current website’s interface, and continuing do so allows for the use of previous work. Furthermore,Shockwave works with QuickTime VR graphics. Currently, QuickTime VR type graphicsare an important part of the web site since the main control page is a QuickTime VRgraphic. Many similar pages will be needed for the future development. Fortunately bothApple and Macromedia (http://www.macromedia.com/shockwave/) have publicly availableSoftware Development Kits (SDK). Apple released theirs years ago and Macromedia hasjust released theirs.Proprietary and conﬁdential July 2000 1Artificial, Inc.
The Library Query EngineIn a regular library a user is able to arrange his books, music etc. in whateverarrangement he desires (provided that it’s not the public library!). In addition, he can andprobably will continuously add new material to his library. He could store these in anyfashion he desires, perhaps even randomly. What he most likely will do is to arrange themin categories and store them in various locations such as shelves to help him find them inthe future. Essentially, he wants space to store his materials and a method to find themwhenever he wants to.The Library Query Engine is the AiDigital Library’s method for selecting and storingInternet information that the user wants to select and store. In effect, it allows a user to dothe same as he would in a regular library, however with all the types of information onthe Internet.As one can imagine, developing a query engine is not trivial because of the diversity ofthe media on the Internet and the complexity of categorizing the information.A good example is the situation with photo and movie formats. There are many differentforms of media and variations of them, such as the picture formats JPG GIF, TIFF, andthe various movies formats MOV, AVI, MPG etc. A query engine must be sophisticatedenough to decode all these formats and also learn about any new ones.A much more difficult problem is how the query engine should categorize information ina manner pertinent to the user. Consider the example of a GIF image of a painting. Onecould categorize it by the colors, history, and even meaning. One analysis provides sevenways of making a suitable search engine (http://www.gils.net/showcase/). Some methods arevery sophisticated and require the use of intelligent agents and/or content analysis.Fortunately, there has been a large amount of government and industrial cooperation inthe making of a universally accepted search engine (e.g., http://www.dlib.org/projects.html).The result has been a concerted international effort to develop a world standard forsearching for information. The standard is known as ISO 10163 in all places except theUS, where it is more commonly know as ANSI Z39.50 does. This standard handles textinformation and also large complex collections such as found in libraries, universities,and museums (http://www.gils.net/webz3950.html).The US government has established the Global Information Locator Service (GILS) toassist in the development and dissemination of Z39.50 to the World Wide Web (http://www.gils.net/webz3950.html). The result has been the easy transfer of the standard to thecommercial sector. Among other efforts, GILS makes available code samples andprogramming utilities for companies to implement search engines using Z39.50. Onesuch company, Blue Angel Technology uses GILS to provide a large amount of thefunctionality required by our engine (http://www.blueangeltech.com/). The AiDigital Library‘s query engine could be made in conjunction with this or other such developers so as tosave time and not duplicate efforts.Proprietary and conﬁdential July 2000 1Artificial, Inc.
Proprietary and conﬁdential July 2000 1Artificial, Inc.
The Library ServerIn a regular library, the user has shelves in which to store his information. The LibraryServer will allow the user to do the same. However not just with text information asfound in books, but with all different sorts of media. The Library Server will work inconjunction with the Query Engine to store multimedia and other information in thelibrary. Furthermore, just as in a regular library where one never has enough shelves, theLibrary Server should have the most possible amount of disk space possible to hold allthe information the user will store.The Library Server must, just like the Query Engine, categorize and select information inan intelligent fashion. Think of what would happen otherwise. All of us who surf the webinvariably save some information. Consequently, all of us who surf the web will mostlikely run out of disk space. What prevents this from occurring all the time is that we saveonly those bits of information meaningful to us? The Library Server needs to do the samething.In addition to selecting what to save, the Library Server must deal with the format inwhich to save it in. The PDF format from Adobe appears to be the best to use becausePDF will be the most common interface output for the display panels. We might haveused Z39.50 since it makes use of a common internal data format. However on the WorldProprietary and conﬁdential July 2000 1Artificial, Inc.
Wide Web, the .PDF format is increasingly becoming the most favored method of storingdocuments that need to be accessed from a web site. In addition, there are a large numberof skilled programmers available with expertise in the .PDF format.Proprietary and conﬁdential July 2000 1Artificial, Inc.
SECTION IV:Advanced Concept ToolsAdvanced concept tools are possible candidates for commercialization: Each tool will becarefully examined as to its potential and feasibility. If acceptable, the concept will thenbe turned into a working prototype for use in the AiDigital Library. The library will beshown to the public and their reaction to the tool will be assessed. Based on this feedbackProprietary and conﬁdential July 2000 1Artificial, Inc.
and other more rigorous assessments (such as is done in usability testing and humanfactors), the tool will either be rejected, or added permanently to the AiDigital Libraryand developed for the marketplace.Building a tool is the most central part of the Library because it directly addresses theissue of profits. Building a tool uses all aspects of the Library. When ideas are proposedand critiqued, the library functions as a think tank. When a tool is built, the libraryfunctions as a rapid prototype and development lab. Finally, when the tool isdemonstrated in the AiDigital Library, it acts as a marketing agency and a usability-testing laboratory.A potential tool must be assessed very carefully and meet certain criteria to be successful.As previously mentioned, the AiDigital Library can be thought of as the integration ofthe Internet to one’s personal environment. It has as least two functions. First, it allowsyou to use one’s physical actions to work with the Internet. Secondly, it allows you to usethe Internet to control one’s physical environment. The library should be comprised oftools that accomplish these functions. This is a loose criteria, but essential formaintaining the focus on the AiDigital Library.Currently, most Internet users explore the Web with a browser on a PC. With theAiDigital Library, we want this capability and much more. The reason we want muchmore is because now, instead of accessing the Internet via one PC, we have an entireroom that we can transform into tools for interacting with the Internet. Some of thesetools will use the room features as control devices for manipulating the Internet. Otherswill serve to use the room as a large-scale addition to the web browser.The following is a list with descriptions of tools that have been proposed as of 9/19/99:Tool: The Internet Book.Description: A flat LCD panel display that allows control of the AiDigital Library HomePage anywhere in the Library. The device would be very flat, lightweight and rugged(encased in a cushion or leather padding). At the top left and right hand corners therewould be large arrow buttons. These are used with the Forward Backward icons in theweb browser.The user goes to where he wants to in the library. Picks up the book and right away whathe sees is the AiDigital Library page. As he manipulates the web site icons, appropriateresponses occur in the various items in the real library. The interface is the web browserso that there is a little learning curve for the user.Technically this is easy to implement. The quickest would be to use an industrial LCDpanel with a long cable for connection to the network. If a cable is unwieldy we mightwant to go with a PC portable with an IR link to the net. Of course the IR link speedwould not allow for the display to be used for video media, however that would not be itsmain use. The user would turn his head up and view the movie on the wide screenProprietary and conﬁdential July 2000 1Artificial, Inc.
monitor. The book would be mostly as a room/Internet controller and use only the localpages.Tool: Visible Mouse FocusDescription: Mouse Focus refers to the object that is currently selected by a mouse. Inthe real world we can also think about having a mouse focus that mimics the position ofthe mouse in the web site. This can be implemented in several ways. One is that as theuser moves the cursor to a hot spot (e.g. the stereo), a spotlight focused on the real objectlights up. Another possibility is by a servo-controlled light. This light would change itsposition as the mouse moves on the screen. This may be nicer because the moving light isa closer simile to a moving mouse than a spotlight turning on. There is also anotherfunction that this tool serves. We can also use it to let the browser select a book from yourlibrary. If the browser knew your books and their location, it could point to a particularvolume when the browser search engine hits include information found in your librarybooks.Tool: Reality FusionDescription: Without a doubt this can be a tremendous enhancement to the library. Usingany common computer video camera, it allows you to use the subject’s position andposture to control things in the library. The company has a web site with a demo versionthat one can download.http://www.realityfusion.comIt is very low cost, has a SDK, and being a local company, can perhaps be a valuable resource.Reality Fusion allows a tool called “User Focus”.Tool: User FocusDescription: This is an extension on the Visible Mouse Focus concept. We can think ofthe user as having a ‘mouse focus’ that we want the Internet browser to be aware of. Forexample, suppose that the user is choosing and adjusting his music system. Most likely,he will be standing next to the audio system so as to adjust the controls. The position nextto the music system is a hot site just as the Web version location is a hot zone for doubleclicking. Knowing the hot zone can be very useful for modifying the equipment inpreparation for its use. This is where Reality Fusion’s product is so helpful. We can use itto easily identify where the user is.Tool: Voice RecognitionDescription: This is an obvious one and must be included. This tool is an example of aclass of tools characterized by the fact that they are not marketable, yet still are neededfor the Library. It is also a very important one because a poor implementation would beworse then none at all. People get enormously frustrated at voice recognition systems thatProprietary and conﬁdential July 2000 1Artificial, Inc.
do not work. A careful choice of the many available must be made to avoid publicdissatisfaction with the AiDigital Library as a wholeTool: Web Integrated Video Editor for Wide Screen TVDescription: The wide screen TV offers lots of possibilities that will be looked at inanother paper. Here, The tool gives is a non-linear editing capability for movies. Itdoesn’t have to be a complete set of controls, just sufficient to allow a user to stop,review, clip and save. A user could make small pieces of multimedia that serve as notes,study guides’ etc. For example, suppose the viewer just saw a news report aboutsomething pertinent to his job. The user should be able to freeze the video, selectsegments, and create a small video clip. The browser will serve to let the user manipulatethe clips. For example, when the user wants to select clips to review, he selects their iconsand drags them onto the Screen iconTool: Virtual HeadphonesDescription: As the user moves about, the library notes his location and moves the audio“sweet spot” so that it follows the user. We can use Reality Fusion technology to obtainthe user’s position. The movement of the sweet spot is possible with PC sound cards thathave 3D imaging capability, such as the SoundBlaster and Gravis cards.Tool: Video ConferencingDescription: Again an obvious one, not marketable, yet needed. To save developmenttime, we will use an upper-end turnkey system offered to large corporations. Themagazine AV Video has reviews of these. For public relations, we might want to get awell-known appropriate celebrity and have him be at the other end of the conversationduring important demonstrationsTool: Keyed informationDescription: Keeping in mind the library metaphor, which connotes an ability to doresearch, we should offer some method of keying various parts of the multimedia stream.For example during video playback, the user should be able to mark the sequence (aspreviously described), but he should also be able to attach a text string used as a key foraccessing information. The most natural way of doing this would use voice recognition.Envision a user looking at a news clip. He sees something of interest. Speaks “Stop”,“Rewind”, “Play for 10 seconds”, “Save Segment as Conference Example”. Resulting ina video clip saved to the server.Proprietary and conﬁdential July 2000 1Artificial, Inc.
SECTION V: The Portable AiDigital LibraryThe Portable AiDigital Library is an Advanced Concept Tool. It was not mentioned in theprevious section because it merits a detailed description by itself. It will be the first toolto be built and the most complex to date.The Portable AiDigital Library will be designed so as to allow a person to have as muchfunctionality of the real AiDigital Library as possible. This tool is needed for a variety ofreasons. In order to market successfully any product arising from the AiDigital Library,we have to take it to potential customers and tradeshows. The Portable AiDigital Libraryis designed for this purpose. In addition, because it is substantially different than theAiDigital Library (because of its size), it will most likely result in additional productconcepts than the original.The Portable AiDigital Library will have three separate LCD panels (not counting the PCdisplay). Each one will be thin, lightweight and mounted on a swiveling armature, similarto that found in some light-stands. The display panels are mounted so that one can easilypick one up and reposition it by hand.The displays have several features. They have touch screens. They have the Dimension-Xoverlay (http://www.dti3d.com) that allows 3D imaging. They have sensors that give theposition of the screen with respect to the user.The display panels form the basics of visualization dome’ that can be rearrangeddepending of the type of multimedia to be presented. Here is how we can use them:Proprietary and conﬁdential July 2000 1Artificial, Inc.
Functions of the Portable AiDigital Library -Panoramic images and movies:The user can put them side to side to form one large display. This can be used to showmovies and images in the wide screen format.Movie editing:One can place a movie clip on each display and use the third as a touch screen control forthe editor. The editor could be a video-editing page on the library web site.QuickTime VR viewing and control:If the user arranges them in a half-circle facing him, he would have an excellent viewerfor QuickTime VR images. The displays arranged like this could give up to about a 120-degree wraparound screen. Viewing a QuickTime VR image, e.g. of the AiDigitalLibrary web site would be impressive. The touch screens would allow the user to sweephis hand across the display and literally pull the graphics to where he wants.GIS analysis:If you place a screen looking up and below you, it can act as 3D-terrain viewer forGeographic Information Systems (GIS). Its application could be in petrochemical 3Dmapping and satellite analysis.Flight, 3D, and game simulations:If you place the displays in a front, left, and right formation, they can be used for car andflight simulators. Used with a standard VRML plug-in, a browser (Cosmo is the typicalone) can act as ‘spaceship’ that one can use to fly through the VRML space. This is doneusing the Dimension-X screen coating and the position sensors on the displays. TheDimension-X allows stereoscopic 3D displays without the encumbrance of glasses, andthe position sensors allow virtual windowing.Book and sheet music viewer:One could place 3 pages of a sheet of music or consecutive pages from a book. As a sheetmusic viewer, we can use Reality Fusion technology so that the user can turn the pageusing physical motions.Videoconferencing:If we add a video camera to each display, we would have a videophone capable of aconference with multiple views of the participants (maybe useful) or of anything that theyput in front of the cameras (very useful).Proprietary and conﬁdential July 2000 1Artificial, Inc.
Distance Learning:Capturing live videos of lectures to a server will allow students in remote classrooms,distant cities and other far parts of the world the ability to attend classes at secondaryschools, colleges and universities at their convenience. The server allows not onlydistance learning but also time learning, in the sense that they can view the lessons at atime of their choosing. This allows company employees working on undergraduate orgraduate degrees, which must travel on company business or have to work during criticaltimes, the ability to learn their lessons even while accomplishing their regular companybusiness. Another market is in corporate training, military training or any otherapplication where important information needs to be retrieved when the viewer hasavailable time.Tool: 3D-scanner station:If we add a video camera to each display, you could use it to produce real-time avatars ofany object placed in front of the displays. Several companies make software that takesmultiple view of an object, such as with pictures, and recreates it as a 3D volumetricmodel. What has not been produced is a real-time 3D volumetric model! This is notnecessarily due to it being technically unfeasible, but rather because without some sort ofmultiple camera arraignment, there is no use for such software. The Portable AiDigitalLibrary will have multiple cameras and so can make use of such software. One companymakes a product that wraps a picture of a person in a smart manner so that it looksrealistic. However, it does this on a generic head model. A combination of this technologycombined with real-time 3D volumetric rendering has great potential, and at the leastwould be impressive.Networking:There is no reason apart from cost not to have the displays be addressable from theInternet. This would allow the useful feature of automatic configuration. A user couldjust put one of these things close to another, and they would sense each others presenceand configure the visual displays accordingly.Section VI: Potential Business AssociatesThis project will require a variety of technologies, and thus will also required businessrelationships with the companies that make those technologies. This is viewed as anotherProprietary and conﬁdential July 2000 1Artificial, Inc.
benefit derived from the building of the AiDigital Library. Here we identify thosebusinesses.BroadBand Networks Corporationhttp://www.bbnc.comBroadBand Networks Corporation or BBNC is a company specializing in a variety ofhardware for making broadband transmission possible, primarily for the cable industry.BBNC is a prime example of the company that the Center was designed to help. As such,we provide new product concepts and marketing by using their devices in the AiDigitalLibrary.Macromediahttp://www.macromedia.com/shockwave/Macromedia makes Shockwave, which allows multimedia like effects with on the web. Itis well suited for use with QuickTime VR. They freely give out their SDK, which we willuse.Reality Fusionhttp://www.realityfusion.com/Reality Fusion makes a computer video camera application that allows one to use thebody space of the user for control of the computer. They supply a SDK and also a demoversion of the software. This software is highly suited for the library because it makes useof the space around the user for control.Cardiff Softwarehttp://www.cardiff.com/Cardiff Software makes a product that allows one to create forms using the .PDF format.The library uses .PDF extensively, both for the display devices and also as the storagetype for the library server. This product would be ideal for use on the library web site forthis purpose. There seems to be no licensing difficulties except that they require a one-week training session.Proprietary and conﬁdential July 2000 1Artificial, Inc.
Blue Angel Technologyhttp://www.blueangeltech.com/Blue Angel Technology makes software that implements the ISO standard for a universalquery engine. Although we will be using .PDF as the libraries native storage format, wemay still take advantage of any existing code that we can modify for our purposes.SFS Softwarehttp://www.siteforum.com/SFSS software has several products useful for the library. SiteEater, useful for the server,allows one to have local copies of web sites. InternalWeb creates a full text search indexfor any downloaded sites (via SiteEater). We most likely will need a developers licensebecause the products, as is, have their own interfaces that may be incompatible for ourpurposes.D-Lib Magazinehttp://www.dlib.org/projects.htmlLibrary science research on digital libraries is extensive and diverse. D-Lib Magazine is agood conduit for possible cooperation with the many organizations such as governmentagencies and universities that are studying Digital Libraries.Dimension Technologies Inc., (DTI)http://www.dti3d.comDTI makes a lenticular coating for LCD monitors that allow a user to view 3D withoutthe use of glasses. This is important for the library because, although we want the user toview and use 3D content on the Internet, we do not want to constrict him in any way. Hemust be free to use the entire space available to him in the library.Section VII: TasksHow do we begin? Building a sophisticated system such as the Digital and PortableAiDigital Library will require several major tasks. This section will look at the problemof construction and specify the various tasks needed to build it. Some of these will be tobuild the hardware network backbone that will connect to the Internet and otherbroadband information channels to the library. Others will be for creating the softwarerequired both to connect the network to the AiDigital Library, and also to build theAdvance Concept Tools. A large task will be to devise or integrate a query engine into theProprietary and conﬁdential July 2000 1Artificial, Inc.
Library. Another will be to develop the Library Server software. The server software willneed to selectively select and store the information that will stream across the network.To start off with the first task will be the creating of the Library Server software. The Server Software:Description:Task Description:Programmer Requirements:Section VIII: Other IssuesConcerning the Oxygen ProjectThis is a large (est. $10 million) project from the AI group at MIT whose purpose is tocome up with the most integrated use of transmission technology - resulting in a varietyof household objects that a person could query in a natural language context. In manyways this can be viewed as a generalized AiDigital Library. However in this case theenvironment is expanded to include every possible space a human may interact in. Thisproject is devised with technology projected to be available in about ten years. Obviouslyit is not pertinent to our effort in making commercially successful Internet products now.One thing that perhaps may be useful to us is their work on command language parsing.This might be available under a technology transfer program.ProfitabilityAs designed, the AiDigital Library in itself has a limited market because of its cost.Some such markets are the government, petrochemical companies, and medical researchgroups. What it was designed for was to provide features that could be marketedindependently of the AiDigital Library. What this means is that while we hope that theAiDigital Library by itself will be marketable; we expect that some Advance ConceptTools will be marketable.Proprietary and conﬁdential July 2000 1Artificial, Inc.
Appendix A: White Paper CommentsThis document is the combined effort of a variety of people. The writer (Daniel Delgado)served as an editor of all the great ideas, which from all the people who contributed insome way or another. The creator of the concept of an AiDigital Library, its majordesigner and supporter, is Gordon Kraft. Chris King, also provided major input. As wellas Ralph P. Manfredo of BroadBand Networks Corporation.This document is our blueprint for the AiDigital Library, and also the conclusion of theserious discussions and careful thought processes that went into designing it. As such,this document is also continuously in the process of being revised and expanded as newinput and more decisions are made. This appendix lists various recommendations aboutthe white paper for discussion. It is in an appendix because it is not intended to be seenexcept for those involved it its creation.Section on Business Relationships:Proprietary and conﬁdential July 2000 1Artificial, Inc.
Our connections to the various individual and companies are very important. It would help us to clarifywho and why we need any such relationship. This has been implemented as Section VIII: PotentialBusiness Associates.Digital Presentation of the white paper:This document is intended to be as complete and detailed as possible so as to serve as the blueprint for theconstruction of the Center and the AiDigital Library. Its detail makes it unsuitable as a marketing tool. Wecan resolve this by having a high-level digital presentation. It would be high-level so as to get the essentialconcepts across to our audience, yet not loose them in the detail. It would be digital because this wouldallow us the most possible audience. The presentation could be done as a PowerPoint demonstration, and/oras a well-done multimedia CD-ROM.Revision and editing by a professional technical writer:Without question, this document must be as clear and as well written as possible. This document will beshown to potential business partners and investors. Its presentation must also be excellent for the samereason. It is important enough so as to let a professional technical writer edit and revise it.Library research expert (digital):Considering the difficulties mentioned in building a universal library query engine, we should identify apartner or affiliation with that expertise. In addition to the benefits derived from a technology transfer, wecould also use this for marketing and other support. Current candidates for such partners are thosecompanies that build search engine type software (e.g. Blue Angel Technology: http://www.blueangletech.com/). The other choice is an affiliation with a research group such as found in auniversity or government agency.Internet use of Library for surveys and usability testing:Would it be feasible to give access to the basic library (free) in order to gain a large user email list - inexchange for surveys and usability testing? If yes, then levels should be created for whatever functionalityis offered. There is where the profitability begins - at what level, at what point and at what cost? Thelibrary could represent the initial acceptance of the user and notify us that the library has become useful andimportant.Top ten site access is needed:There are lists available that give you the top ten web sites visited on the Internet. These should be includedin the basic library so se can be assured of user, market share. Each of the tops ten should be available inperhaps a richer and more useful environment.Proprietary and conﬁdential July 2000 1Artificial, Inc.
Gordon H Kraft is has been a frequent attendee and panel member of ETRE and ATRE conferences. Mr.Kraft currently owns Artificial, Inc. a Internet Research and Development Company, and is a privateinvestor in several high tech startups, on their respective Board of Directors. He is the founder of DiagSoft,Inc. which was sold to Sykes, Inc. in Aug. 96. Mr. Kraft has over 35 years of experience in the computerindustry. Mr. Kraft owned Big Kahuna Productions a multimedia production company located in Maui, Hi.Mr. Kraft has served as Vice President of Sales and Marketing for Award Software and Condor Computer.A recognized expert in mainframe, minicomputer, and microcomputer technology. Mr. Kraft has heldexecutive positions at Handle Technologies, Artelonics (Shell Oil), Microform Data Systems, Arthur Young& CO., Caelus Memories, Fairchild Semiconductor, and GKE Software, which he founded.Hobbies: Mr. Kraft currently owns and races two APBA/SBI Offshore Racing Powerboats and is amember of the new APBA Offshore2000, Inc. Board of Directors. (soon to be the Nascar of OffshorePowerboat Racing)The CompanyArtificial is a private Internet Research and Development company that Mr. Kraft owns. The company isfocused on commercializing our previous R&D efforts. Our Phase II - AiDigital Library is currentlybeing prototype.Technology & Market InformationArtificial developed the following web sites:www.ghklibrary.com 3D QTVR AiDigital Librarywww.artificialinc.com Corporate Sitewww.bigkahuna.net Major Hawaiian Portalwww.talkstory.com TalkStory Island CHAT rooms and TalkStory Offshore Racingwww.avstreams.com prototype Real Video Streaming sitewww.cyberforms.com prototype Avatar site for TalkStory Island CHAT roomswww.aiengines.com initial Ai research sitewww.moses-tm.com GHK investment - USB Home LANwww.bbnc.com GHK investment - MPEG2 Video for ATM and Internetwww.star-power.com GHK investment - Home Theater retail chainwww.ghklibrary.com/lazzara/beta GHK investment - Lazzara Yachtswww.apba-offshore.com GHK investment - APBA Offshore Racing!Mr. Kraft utilizes Artificial, Inc. to assist the GHK investments and provide a private “skunk works” formodeling our product visions. Artificial, Inc. licenses it’s designs, copyrights, and trademarks to OEM’s ona “sticker” bundling basis. This is very similar to Mr. Kraft’s previous successes at Award Software, whereMr. Kraft created the “sticker” BIOS business, and at DiagSoft for bundling QAPlus Diagnostics tovirtually all of the worlds PC manufacturers.Artificial’s current product focus is on the “Convergence of Hollywood and Microsoft”. Also know as“Digital Living Rooms.”Proprietary and conﬁdential July 2000 1Artificial, Inc.
Financial InformationRevenue InformationGHK Investment Fund $5MMr. and Mrs. Kraft own the GHK Investment Fund.Company Objectives for attending the ConferencesTo meet with my ETRE friends. I am seeking Corporate Sponsors for the APBA Offshore2000, andseeking additional investment capital and potential partners.North America OfficesArtificial, Inc.889 S. Heathwood Dr.Marco Island, Fl. email@example.com 1.Artificial Intelligence ResearchMAST, Inc.It was a most profound experience for me intellectually; to have worked with a ArtificialIntelligence company named MAST, Inc.Proprietary and conﬁdential July 2000 1Artificial, Inc.
Maui Analysis Synthesis & Technologies (M.A.S.T.) was a “Neural Network” software company, locatedin Kihei, Maui. MAST was founded by Mr. S. Warrick a naval architect, and very brilliant individual. Mr.Warrick, from the northeastern U.S., had moved to Maui partially because of the awesome wind surfingconditions present there. Soon thereafter, Mr. Warrick developed the sail design software that Mr. Kraftbelieves is still the hot ticket in wind surfing sail design. MAST had another mainland guy named Mr. P.Billings, a mathematician from New Mexico. And a gifted computer programmer, part-time concertpianist, and a top Wind Surfer named Mr. J. Crim. They had developed a Neural Network program calledNetCaster that attempted to “predict the S&P 500.” DiagSoft, Inc.Mr. Kraft was the Founder, Chairman/CEO, and ChiefArchitect of DiagSoft’s PC Hardware QA Diagnosticsbusiness. At the time of the sale of DiagSoft we hadbundled over 30,000,000 copies of QAPlus, and had over85% of the worlds PC manufacturers testing their PCSystems with DiagSoft’s QAPlus Factory QA System.Mr. Kraft opened DiagSoft’s Pacific Rim HQ just down the hall from MAST. And soon thereafter, Mr. C.King and Mr. Kraft founded a Multimedia company called “Big Kahuna Productions” in the Maui Research& Technology (MRTC) building.Mr. Kraft purchased MAST thinking that the addition ofArtificial Intelligence to DiagSoft’s QA products, and hisnew Electronic Technical Support Center (ETSC) project,and a multitude of other non DiagSoft computerimplementations. On August 30, 1996, Mr. Kraft soldDiagSoft, Inc. to Sykes Enterprises in Tampa, Florida.Unfortunately MAST went with the sale. MAST was notsupposed to be on the books of DiagSoft. Mr. Kraft hadintended to move on with MAST after the sale ofDiagSoft, and venture into the general ArtificialIntelligence arena.So without MAST to build on, Mr. Kraft founded Artificial, Inc. a Florida corporation. www.artificialinc.com http://www.ghklibrary.com/aiprojects/ http://www.aiengines.com/Proprietary and conﬁdential July 2000 1Artificial, Inc.
Proprietary and conﬁdential July 2000 1Artificial, Inc.
Artificial, Inc. Research & Development – 1997(2). AI Digital LibraryThe Artificial, Inc. “AI Digital Library” is an Internet based multimedia virtual library that has the abilityto interface with a Crestron Home Theater control system. Mr. Kraft began this project based on myprevious expertise in Document Storage and Retrieval systems integration at Microform Data Systems inthe 70’s, and my expertise in Office Automation Workstations and “Paperless Offices”. www.crestron.com(3). GHK Digital LibraryMr. Kraft is an investor in the Dallas based, StarPower Home Entertainment, Inc. retail chain at therequest of my friend, Nathan Morton, StarPower’s Chairman.The initial implementation of the AI Digital Library was to create a multimedia library replicating a actualold English cherrywood library that Mr. Kraft have added to our home. This version is called the “GHKLibrary”, and can be found at: www.ghklibrary.com 2.StarPower Home TheaterThe Home Theater Surround Sound experience is an area that Mr. Kraft has personally spent substantialtime and money on. Mr. Kraft has always been one of the first customers of each new Audio-Videotechnology introduction. http://www.star-power.com/The AI Digital Library uses an Oracle Video SQL Server for the Storage and Retrieval of the userspersonal and/or corporate videos. The AI Digital Library stores all photos, artwork, and all documentsin .DOC and .PDF format. And since it is Internet based, media and documents can be added from anyInternet workstation, and retrieved as well, anyplace in the world... Real Video Streaming and PointMPEG-2 transmission to Multipoint locations is supported.The user(s) therefore have an “Artificial library with Intelligence”. Hence, a ArtificialIntelligence Digital Library.(5). Digital Living RoomsAI Digital Library is also a Internet based “user command interface” to a “Digital Living Room”. http://www.digitallivingroom.com/ http://www.software.ibm.com/is/dig-lib/ http://www.ita.sel.sony.com/vaioworld/home.htmlProprietary and conﬁdential July 2000 1Artificial, Inc.
http://www.zdnet.com/zdnn/stories/zdnn_display/0,3440,2114009,00.html http://www.cedia.org/ 3.Merging Microsoft and Hollywood, the basis of the “StarPower Project”The continued growth path of the PC industry has finally reached the initial beachhead of intrusion into theMovie and Television marketplace. Today, Microsoft owns a major interest in NBC Television, as anoutgrowth of MS Network (MSNBC), which Mr. Kraft expect, will ultimately consume NBC Televisionand others unless the DOJ changes the current course of Microsoft.So now Microsoft has a “Edutainment Channel”.Proprietary and conﬁdential July 2000 1Artificial, Inc.
9.SummaryThe demand among individual users, home office users, and small business users for networking andcommunication solutions is increasing exponentially. The growth in Internet usage and especially withDSL & Cable high speed digital access is fueling this demand, and the further increase in the number ofmultiple PC homes, and the limited number of networked small offices is adding to demand. Concurrently,there are numerous cable and USB product manufacturers as well as network hardware marketers offeringpartial solutions But none of these companies today are offering networking solutions that provide the userwith auto installer capabilities of Moses NetAgent even when connecting a single PC such as the networkconnection of DSL. The installation of DSL is spotty and needs a much better approach in connecting thePC to the Internet, and installing the Ethernet adapter and 10BaseT networking hardware. With DSL,Moses has a way of integrating all the Ethernet system pieces by connecting through Moses DSL-Exchange. Moses software then installs and establishes communications for all devices. In doing thehardware and software, and the combination of both, Moses is the only solution provider offering the endusers a complete solution to their networking and communications needs. With this advantage in themarketplace and Moses ability to successfully debut these products, and continue the introduction of newproducts, Moses will capture a significant portion of the growing high-speed access market.Proprietary and conﬁdential July 2000 1Artificial, Inc.
Artificial, Inc. Plan 99 Internet Research Services Strategic Business Solutions CBT Training Solutions Internet Product R & D Business Analysis ServicesProprietary and conﬁdential July 2000 1Artificial, Inc.
Artificial, Inc. completes 97/98 PlanGordon H. Kraft created Artificial, Inc., as an Internet based, Research and Development "virtualcorporation". Artificial uses the full capability of the Internet to provide its customers with themost advanced Internet resources.After an exhausting research effort into various Artificial Intelligence product areas(1), we concluded thatthe "Internet" offered the best long-term variant of Artificial Intelligence. (AI) from a profit generationbasis, will provide the best-case scenario for data mining the Internet. We then began our initialdevelopment to create an "artificial world". As such, we have employed several high-end tools in thedevelopment of our sites with the intention to use these sites as the gateway for building the sites of thefuture. Tools like Shockwave, Real Video and Audio plus original animations and graphics mixed with liveand taped media will propel artificial sites beyond the ordinary. Since the inception of Artificial Inc., wehave pushed the content of our sites and our client sites to the maximum the Internet can provide. Thefollowing sites reflect our 97/98 R&D efforts:Proprietary and conﬁdential July 2000 1Artificial, Inc.
www.artificialinc.comThe Artificial Inc corporate site links users to virtually all aspects of our Internet site development utilizingleading Internet technologies. Included is our own 3D TalkStory Island Palace Chat site connecting islandpeople around the world (nokaoi). This site utilizes Macromedia Flash and integrates all of our sites in anoriginal graphical interface. We utilize Real Video Audio/Visual Video Streaming, and products likeNetPodium for wide area business presentations, meetings, and CBT training. Artificial, Inc. electedJeffrey O. Kraft as our new President in June 98. Jeff is responsible for the commercialization of our R&Defforts.www.ghklibrary.comThe GHKlibrary is a state of the art, Apple QuickTime (QTVR/with hotspots) replication of a real SoHolibrary. Its purpose is for building future custom internet based Home Digital Librarys (HDL)(2), This siteis our initial effort to develop a next generation "what you see is what you get, even if your not at home".And provides its user unparalleled information search and retrieval from the Internets vast InfoBase.Unlike desktop routines, which are merely a navigation aid, the GHKlibrary is a web based resource centerfrom which the user can manipulate both desktop applications and the Internet for a truly integrateddesktop management tool. While this version of the Library may never see the light of day as a commercialproduct, it does show how technology can be combined into one format. Prior to releasing the Library as acommercial product, several key security and product issues must be resolved.The Library was created for the sole purpose of uniting the Internet with the desktop. As anyone knowswho has accumulated file upon file of information, it has become increasingly impossible to catalog thisdata for future reference. The Library, by use of third party applications, will bring this under control.The real GHKlibrary has a complete Computer Research Lab for creating and testing of integrated leadingInternet technologies(3).Think of it as a huge desk drawer that the user dumps everything that they come into contact with. Thisincludes BetaSP/SVHS/Hi8/DVcam Home Video, CD Library, DVD Library, Star Power Home Theater(4),and includes a complete multimedia Video Editing Suite, as well as Textual and Graphical info in amultitude of data formats including:MS Word, MS Excel, MS PowerPoint, MS Outlook, Adobe PDF library, MPEG/JPEG/GIF/PhotoShoplibraryA IBM Infinity 5500 with dual Pentium II 400 NT Video Server for the home, with over 100GB of harddisk, and DAT backup running an automated home over 100T cat 5 LAN.Microsoft and Intel are also doing R&D on "Digital Living Rooms(5)" which is very similar to ourStarPower Project(6) of "integrating Microsoft and Hollywood".Proprietary and conﬁdential July 2000 1Artificial, Inc.
www.talkstory.comWe believe that our Hawaiian theme of Aloha, embedded in our Internet Chat site will continue to grow invaluation. Based on the successful joint interest and involvement of our friends throughout Hawaii, this sitewill grow. TSI was created to allow those Hawaiians currently living on the Island and off can have a placeto chat and share in a comfortable and friendly environment. TSI also employs Quick Time VR, StreamingAudio and Video plus links to our Offshore Racing Team; “Team Lazzara Offshore Racing”.www.bigkahuna.netWe will create a TalkStory Island Mall providing shop space to our friends companies throughout theislands of Hawaii and other Island communities throughout the World. This Mall will be a virtual Mall andtherefore will not restrict itself to just one store, but to a multitude of independent stores all thriving lessthan one roof.Oh you thought that TalkStory was limited to the Hawaiian Islands, no bra, dye be more man.A r t i f i c i a l, I n c.889 S. Heathwood Dr.Marco Island, Fl. 34145www.avstreams.comAVStreams is a fully video / audio conversant site. What this means is that AVStreams is a demo sitedesigned to show how video can be played over the Internet for a variety of uses. The main target of thissite is to show video can be successfully played at common 28.8k modem speeds. The benefit of payingvideo over the Internet is very simple. Video is the best method of providing interactive training andinformation to anyone anywhere and at any time. The user benefits form having the ability to viewinstructional data at their leisure and from virtually anywhere. The actual application of playing video oraudio would occur on the customer site and therefore, the customer has total security of who has access andwhat materials are displayed. AVStreams is the host and technical link in providing this service.www.lazzarayachts.comLazzara Yachts, one of the premier Mega yacht builders in the world, contracted Artificial, Inc., to developa completely new and unique web site. This site was created using all of the features Artificial haspromoted and as a result has created one of the most striking and interesting sites on the web today. Thissite incorporates Palace Chat, Real Video, Streaming Audio, Quick Time animations, privacy areas,document retrieval, customer support and e-commerce. The primary purpose of this site is to provide anew level of customer support to the existing and prospective owners of a Lazzara Yacht. It also hasprovided a direct method of communicating to the owners regardless of where they may be located.AV Streams.com A NEWSLETTERWelcome and Aloha! From Artificial, Inc. Developers of Virtual Worlds.AVStreams tales full advantage of cutting edge technologies. With full 3D web sites, high speed modems,streamed audio and video plus services to back up we offer, AVStreams is set to take the Internet into the21st Century. Take full advantage of high-end web development and services, AVStreams offers ISDN,SOHO Networking, WEB Publishing, AV Streaming, Encoding and WEB Broadcasting.Proprietary and conﬁdential July 2000 1Artificial, Inc.
THE SITEThis site is the first of several new sites being developed by Artificial, Inc., which will full advantage ofHigh-Speed (HS) web services. As an indication of our commitment to newer and faster technologies, weoffer ISDN adapters for 128k Internet access. While viewing multimedia content, this HS access willcontinue to prove critical over the coming years.Streaming technology can be accomplished in many ways. We have chosen to utilize Real Networks as ourplatform to present video. The videos shown are all original copyrighted materials for which we retainexclusive rights. To view these videos, the user must download the viewer from the Real Network site. Wehave supplied a button to accomplish this small task. Video, which is displayed, can be created in virtuallyany format and then imported to the Real System. Artificial Inc. our parent company, is licensed to provideReal Server services to play video through our server seamlessly to any web site. What AVStreams offersis not only this service but complete editing, and encoding services too.Proprietary and conﬁdential July 2000 1Artificial, Inc.
Encoding and Editing ServicesIn addition to hosting video, AVStreams is capable of full editing and encoding of existing video. Encodingis a critical piece in presentation of video over the web. Encoding provides for smooth video viewing evenat slower modem speeds. When used on high speed networks, video and be viewed at near perfect speeds.Our encoding service will convert your video for play over the Internet. Encoding also helps incompressing video so less server file space is utilized.BEACHCAMAVStreams is fully multimedia enabled. Select any of the buttons to view a multimedia experience. Wehave added a BeachCAM for your enjoyment. From our location on the beach in Capitola Village,California, you can view current real beach conditions. And yes, it really is from our window. Use of webcameras will continue to grow and become common amongst many popular destinations. Our examplehere is to show you how a WEBCAM would appear in your WEB Site. Typical webcam applications couldincluded Safety and Health Training videos, promotional information, special sales information, securitycameras and even video conferencing within the site.STREAMED VIDEOWhen you selected the AVStreams logo, it immediately brought you to a “STREAMED” video. This videois a simple example of the ability Artificial, Inc. can bring to your web site. By incorporating audio withstreamed video or a slide show, we can significantly increase viewer enjoyment. Streamed Audio andVideo can be viewed from this site by selecting any of the icons from the top of the screen. Streamed videocan be very simple to quite extensive. The frame size shown on our site is set at 640 x 480. We have setthe resolution at this size to take full advantage of the video and graphics for most users’ capabilities.However, we can design a site to any specification. Streaming Video is extremely useful when conductingin house educational programs or merely as informational video.Proprietary and conﬁdential July 2000 1Artificial, Inc.
Proprietary and conﬁdential July 2000 1Artificial, Inc.
ShowcaseThe example below is a brief demonstration of how a viewer could look. We have chosen a drive in theaterlook. We can design graphics, which could represent your company image or logo. Take a minute topreview the other streamed examples to see how we have created custom showcases for the differentvideos.ISDN and Multi-Point ISDNAnother example of our efforts to provide high-speed high quality is our offering of ISDN modems. ISDNcan increase web access by as much as 50 times over what you’re currently experiencing. We offer twotypes of ISDN products. The first product is for single user application. The second is for multi-userapplications. Whether you have a small office or home office, the multi-point set up can hook up to eightusers sharing the ISDN modem. This starter kit provides all the hardware and software necessary to gettwo PCs up and running on the net.Frequently Asked Questions (FAQ)When we created this site, we tried to anticipate all types of questions. The screen below is typical of theinformation we provide.Proprietary and conﬁdential July 2000 1Artificial, Inc.
While there may be questions be haven’t answered, we’ve tried to answer those which are asked mostcommonly. If you have a question we have not answered, please respond by e-mail firstname.lastname@example.org .More News and Press ReleasesWatch this page for more news and upcoming Press Releases. Artificial, Inc. promises to not leave anyonewanting for information or the latest in WEB design. Over the next few months we will be announcingmajor new web sites, services and promotions. Until then Aloha. And why aloha? It is because our nextweb site will feature the islands of Hawaii, its people and its culture. It is our hope to bring some of thatculture to the mainland to be shared.Our next issue will feature video editing and how our services can be implemented into any organization’semployee development program. Created and published monthly by Artificial, Inc. Developers of Virtual WorldsProprietary and conﬁdential July 2000 1Artificial, Inc.
Portable Library SpecificationsThese specifications are for the Portable Digital Library, which is being built as the mainshowcase for the Artificial Technical Center. Both the Center and the Library areexplained in detail in the white paper "The Artificial Technology Center and the DigitalLibrary", available on the Artificial Technology web site:http://www.ghklibrary.com/aidl/This document is for the purpose of describing how to build the Portable Library. It iscomprised of three sections. The first section, Methods, describes the desired featuresand then procedures for making them. The second section, Materials, is a list of items tobe purchased. The third section, Notes, contains information pertinent to the library as awhole. In addition, a schematic overview of the librarys hardware and softwarecomponents is shown in Figure 1.MethodsFeature: Moveable Display ScreensDescription: The ability of a user to place the displays in almost any position ororientation desired. This is a fundamental component of the Portable Library. ThePortable Library has three main LCD panels for its displays. By positioning the displaysin various configurations, the user of the Portable Library can use it as either a widescreen system, or a wraparound QTVR viewer, or an out-the-window views such as isused for flight and car simulators (these configurations are described separately asindividual features). In addition it also allows the user to use the entire space around himfor the Windows Desktop. With this a user can, for example, place file icons for moviesclips above his head, and sound clips next to his left ear.Procedure: Modifying a "Magnifier Reading" lamp provides the mechanical arm thatsupports the Librarys displays:http://www.hammacher.com/DefaultPage/default.asp?ContentPage=/publish/67468.htmThese lamps have a large magnifying lens and are used by people who do precisionminiature work as well as by the sight impaired. Because of their needs the mechanicalsupport linkage has been optimized for maintaining the light and lens at a position of theviewers choice. This is the capability the library needs for its displays.Three lamps are needed. Each is modified so as to remove the head/light assembly andsubstitute it with a LCD display. Depending on the weight of the lamp, we may alsoreplace various springs for ones of higher tension to compensate for the greater weight ofthe display.Proprietary and conﬁdential July 2000 1Artificial, Inc.
To obtain the displays location and orientation in real-time, we use image-processingsoftware provided by Reality Fusion:http://www.realityfusion.com/described in the Materials section. Each display monitors the position of the others so asto provide a continuous feedback as to its position and orientation.Written with Reality Fusions SDK, a program on the Portable Library calculates thedisplay position with respect to the user, and then provides the information to other partsof the library. The library can then access the information in one of two ways; either byusing Window event messages or by using a shared memory location. The conventionalmethod is to use Window event messages. However, it this is not feasible we will resortto a shared memory location. In either case, any program that requires positionalinformation, such as a Java Applet activated from the browser, must make use of thetrackers SDK if it needs to access this information.For the out the window simulators, the program will change the display mode to oneviewport per display. The different views can then be produce by either configuring thesoftware for this arraignment or by running multiple instances of the executable, but withdifferent viewing positions (e.g. left, right, and front views).For the QTVR and Panoramic viewer, the program will change the display mode to onelarge viewport with each display mapping into a different portion of the view. Thisconfiguration will also be used to enlarge the Desktop to surround the user.Feature: QTVR ViewerDescription: The QTVR Viewer is a configuration of the library that is optimized for thewraparound images of QuickTime VR. The LCD panels are positioned side-bye-side in asemicircle. Such an arrangement can increase the field of view (FOV) to as much as 120degrees. The aim is to give an experience similar to that of a Circle-rama theater such asis found in Disney Land. Since QTVR images are used extensively by the library, thiswill be an important part of the overall effect of the Portable Library.Procedure: A program will be written that monitors the position of the three displays.When the user places them side by side in a semi-circle, the program will send aWindows event message to all active programs, and also change the display mode to thatof one large viewport with each display mapping into a different portion of the view. Inaddition, the size and aspect ratio of each viewport will be changed to match the FOV ofthe combined monitors and a system message will be added that reminds the user toreposition the displays when viewing a QTVR image.Feature: 3D DisplayProprietary and conﬁdential July 2000 1Artificial, Inc.
Description: The use of the Dimension Technologies lenticular coating allows a user tosee volumetric 3D data without any encumbrances such as goggles or LCD shutterglasses:http://www.dti3d.com/With it the user will be able to explore 3D data sets on the Internet using a VRMLbrowser. He will also use it with car and flight type simulations and for watching 3DmoviesProcedure: The displays must be very lightweight LCD panels. To use the DimensionTechnologies 3D coating, we must use a LCD panel display because the effect dependson the characteristic of liquid crystals to polarize light. To use the display for thePortable Library, we must also make the displays as lightweight as possible. Weight is animportant factor because the Librarys support arms are optimized for the weight of itslamp and lens assembly. Our displays must be the same weight or lighter.We will require three displays. They will be bought off-the-shelf, or adapted from laptopdisplays, or built by us. The most convenient way would be to purchase them, howeverit is unlikely that we will find off-the-shelf displays that meets our needs. The next easiestsolution is to buy a portable computer and remove its display, power supply and videocircuit board. This may be made more convenient by finding a manufacturer that will sellus just the video components from its laptop. Regardless, we then substitute long cableassemblies and put only the LCD panel on the arm, all else will be placed at the base ofthe arm.If obtaining a manufactured display is not possible, we will buy a LCD panel, a videointerface board, power supply, and build it. This is not very difficult but is a moreinvolved process than the other methods. On the other hand this way does give us theadvantage of allowing us to select the best and most appropriate parts.After obtaining the displays, we then send them to Dimension Technologies for them toapply the lenticular coating need for 3D. Since this may take some time (not under ourcontrol) we will send the displays to the manufacturer as soon as possible. Developmentwill then continue using a standard monitor display. To speed development, an effectiveyet costly way would be to obtain an off-the-shelf 3D monitor from DimensionTechnologies in addition to the coatings for the librarys three displays. The 3D monitorcan then be used for development until the librarys displays are modified.Since VRML 3D models are the most common types of 3D objects found on the web, wewill have a VRML browser with 3D capability. The Windows web browser Explorer callsits own VRML browser, but others are easily substituted. Hopefully, the DimensionTechnologies people know or sell a web browser with built-in 3D support. Otherwise, wewill use a well-known VRML browser such as Cosmo Player and adapt it for use in theLibrary.Proprietary and conﬁdential July 2000 1Artificial, Inc.
Many simulator games now support 3D viewing. The Portable Library will be able towork with all such games and programs. These are meant to be use with LCD Flickerglasses. The lens go dark then light, only allowing one eye to see an image at any onetime, usually at a rate of the display monitor synch (e.g. 60 hertz). By presenting to eacheye one image of a stereoscopic pair, a fairly effective 3D effect is produced. ThePortable Librarys 3D coating is compatible with this method. We should verify this withMS Flight Simulator and a popular network flight simulator, and some other popularprograms.Feature: Panoramic ViewerDescription: The displays are placed side by side for viewing videos with wide aspectratios such as Panoramic movies. Unlike the QTVR displays, the images are not arrangedin a semi-circle but as one large flat display.Procedure: Although this is a simple concept, achieving a workable view will bedifficult. When watching movies and looking at images, one should not have a borderalong each portion of the image mapped by a display. Yet just about all displays have aborder along all edges of the display, somewhat like a picture frame (in the case of a LCDpanel, this is probably the display circuit board). The ideal solution is to find a displaywith no borders at least along one dimension. Without this, we may be able to achievethe same effect with real-time optical correction and some type of mirror arrangement.Please see the Addendum: Concerning making a flat display with no gaps (Gordon, Ithink we might have some patents here).Regardless of how it is achieved, speech, hand motion, and the touch screen will controlthe Panoramic Viewer. As with the QTVR viewer, it will make use of the IR positionsensors on the librarys three display support arms. A program will be written thatchanges the display mode to that of one large viewport with each display mapping into adifferent portion of the view. Again as with the QTVR viewer, the size and aspect ratioof each viewport will be changed to match the FOV of the combined monitors, and asystem message will be added that reminds the user to reposition the displays when he isviewing a QTVR image.A user will be able to use the Panoramic Viewer for editing and interacting with video(such as is done with enhanced DVDs). While watching a news clip the user could say"stop" followed by "save as example one", which would result in a of a clip of videosaved onto the hard drive. He could also use hand motions or the touch screen to changethe speed of the clip. The user will also be able to manipulate static images. He will beable save an image to disk by using his finger to trace around a frozen video segment(such as with a mouse with MS Photo Editor).Proprietary and conﬁdential July 2000 1Artificial, Inc.
Feature: A Friendly LibrarianDescription: The user can easily search both the Librarys material and the Internetwithout the user needing to know browser commands. It then presents the material in amanner appropriate to the media, so for example video is sent to the Wide Screen TV andaudio is channeled to the music system.Procedure: This is a very challenging feature. It must make use of the software routinesof every SDK used in the library. Software must be written to allow control with voice,hand gestures, touch screen and keyboard entry. With voice the user will be able toaddress the librarian, ask a question, and have packaged "power-point" type presentationof the information. The initial version will try to simplify the problems by limiting theinformation and media to that obtained from the Internet via the web site of MargaretAnderson:http://www.servtech.com/~mvail/home.html.This allows the program that is parsing the information to make use of a common dataformat. A Java/C++ programmer will be used.Even using Margarets web site as a test site, various programs must be written andintegrated together. First a control program that monitors all I/O, meaning keyboard,voice, gesture, and desktop commands. Upon activation by the command Librarian,which e.g. is done by voice by calling out "Librarian", the program must parse thesubjects question, and then output one or more browser commands to obtain theinformation. Having received it, the program must then select the appropriate mediumand present the information. The difficulty lies of course in programming thepresentation.Feature: Hand Motion ControlDescription: The user can control various aspects of the Library hand by using handgestures. These are the following: QTVR viewer Video Photographic images MusicHopefully this will be one of the more natural ways for controlling the library. With thisa person could sweep his hands across a QuickTime VR image and have it rotate aroundhim. He could point to a display icon and launch it by literally making a throwing motionor other such gesture. With photos or other graphics, a user can use his hands to pan animage larger then the viewport. For video, the user can use gestures for the editingcommands such as stop, fast forward, and reverse. For music, the user can interact withany of the numerous programs available for creating, conducting and playing music. ForProprietary and conﬁdential July 2000 1Artificial, Inc.
example, a user could do such things as change tempo, switch or add instruments, changethe rhythm or turn a page while one plays.Procedure: Software from Reality Fusion will be used for recognizing gestures. Thissoftware does a real-time analysis of the video input so as to respond to specific hand andbody gestures by the user. To use it a small lightweight video camera facing the user willbe attached to each display panel. Using the Software Developers Kit (SDK) that hasbeen provided by Reality Fusion, we will write software to interface it with the Library.The software will send the appropriate user I/O (most likely emulating ASCII from thekeyboard) when it senses a command gesture. Reality Fusion will be used for thefollowing: QuickTime VR Viewer Melody Assistant (shareware) Microsoft Photo Editor (GIF, JPG and others) QuickTime Viewer (MOV) Microsoft Active Player (MPG)Feature: Speech ControlDescription: The user controls various functions of the Library by voice. Voicecommands will be used for the following: Windows Commands Web browser Library web pages Display settings (brightness, aspect ratio, contrast etc) Video (editing, stop, start, rewind etc.) Music (loudness, bass, treble etc)Procedure: The software ViaVoice will be used for speech control. Like most voicerecognition systems, it comes with a command set for controlling Windows (such as withfile commands) and also procedures for setting up command sets for programs. With thiswe can then give the user control over the display settings since they are adjustable by theoperating system. Command sets will be written for the other functions.The first command set will be written for the browser since it will be the primary userinterface. By doing so we add voice commands to the Librarys web pages. To add voicecapability for video, music and the display mode, additional command sets will be set upfor the most commonly used video and music editor programs, as well as for changingthe display. The programs that will have command sets written are the following: Microsoft Photo Editor QuickTime Viewer QuickTime VR Viewer Microsoft Active PlayerAdditional command sets will also be written for the audio player and other multimediaprograms that were provided with to the computer when purchased.Proprietary and conﬁdential July 2000 1Artificial, Inc.
For show demonstrations we need to make sure we can use voice independent commandsfor at least a subset of those needed. If ViaVoice does not supply this than we obtain anadditional software package that does to be used in conjunction with ViaVoice.Feature: Touch Screen ControlDescription: The user can control the Library using touch screens on its three displays.Using the software typically supplied with the screens, the user will be able to selecticons, move scroll bars, etc. just as if done by a mouse.Procedure: The touch screen will use an active optical matrix keypad and interface withthe Library computer through the serial port. Tentatively, we will use a display fromIntersense Inc.http://www.isense.comThis works by placing optical infrared emitter/transmitters around the perimeter of thedisplay. Unlike the pressure sensitive kind, they do not obscure the image. We will alsohave fewer problems with vibration because they do not require an actual press on thescreen. This particular touch screen needs to be tested to see if it does not interfere withthe IR from the head tracker. We may have to use the pressure sensitive kind if theothers infrared system interferes with the tracking system that is used for display andother positional information.Touch screens typically come with software so that a user can control the screen as if itwere a mouse. We will use this for most of the functions in the library that will make useof the touch screen (those that are mouse like). If the library is using the displays in agroup (such as with QTVR), the touch screen will be active across all three screens andwith one shared viewport.Feature: Three-Camera Viewport.Description: Each of the three LCD displays has a lightweight video camera attached toit. Used with a good user interface this feature could be one of the more marketableaspects of the library. They provide the benefit of three viewpoints of the participants inconference. In addition, because a user of the Portable Library can reposition thedisplays and cameras, they can move them so as to interact with other participants.We could greatly enhance the effectiveness of this type of videoconferencing bydesigning utilities that help the user use the three-viewport arrangement. These couldinclude such things as a 3D global display of all participants in one picture, or as a tactileresponse when remote displays bump into each other.Another thing we could do stereoscopic video conferencing. This is possible in thePortable Library because all the displays are 3D capable. If the user positions two of theProprietary and conﬁdential July 2000 1Artificial, Inc.
cameras/displays towards a view, the two images would be used to produced stereoscopicpairs that result in a three dimensional view of the visual scene.These enhanced video features are well suited for distance learning and teleoperationssuch as remote surgery.Procedure: A Logitech PC QuickCam will be attached to each display so as to face theviewer:http://www.realityfusion.com/gamecam/about_cam.htmlA Logitech camera is used because it is packaged with the Reality Fusion software:http://www.netsales.net/pk.wcgi/reality/prod/1341273-1Hopefully this means we will have less of a problem interfacing both the camera andsoftware to LCD displays. The cameras will be connected via a USB port so as to have asmall cable footprint. This is necessary to keep the combine weight of the display andcamera as low as possible.Feature: Three-dimensional volumetric reconstruction.Description: The user of the library can use the cameras to reconstruct a 3D volumetricimage of whatever he places in front of the library. If he does this to himself he is alsoprovided with a 3D analogue suitable for use as an Avatar.Procedure: Several commercial software companies make products that construct a 3Dobject from pictures of the object from several viewpoints (Gordon, still checking thisout). In addition, there is one company, Microlead:http://www.ulead.com/ff/runme.htmthat makes software to wrap a picture of a person intelligibly on a generic 3D model of ahead. We obtain these products and hopefully SDKs so as to do this in real-time in thelibrary.Feature: Head TrackingDescription: Using a I/R tracking device, the users head position is monitored in real-time. This is used for creating a Virtual Window display, and also for any additionalfeatures that may need real-time image processing with respect to the users eye gaze.Procedure: An I/R tracker from Origin Instruments (DynaSight) will be used to providethe tracking:http://www.origin.com/We use the tracker primarily to gather real-time data so that we can have a realisticVirtual Window display. This is an impressive feature that we should have. A VirtualProprietary and conﬁdential July 2000 1Artificial, Inc.
Window is a close cousin to my device a Portal Display (and its basis). Virtual Windowswork by monitoring the head position of the user so that the graphics through a window,such as with an airplane, car, and other vehicles, is accurately recreated for the opticalpath between the user and the scene. We create an out-the-window scene (in 3D becauseof the Dimension Technologies coating) such as is needed for a flight or car simulator,where we calculate in real-time the visual scene using the head and display position andorientation provided by the tracker. Using this we have a virtual window. The trackerwill also be used in any features that involve a real-time optical correction of the displayimage.MaterialsHardware Window 95 machine optimized for graphics and as fast as possible (700 MHz) with monitor (TBD) Three PCI graphic cards, OpenGL capable with NTSC out (TBD) Three display support and position systems Three very lightweight LCD displays (TBD) Three IR Touch Screens (TBD) Lenticular stereoscopic coating for LCD displays Three very lightweight video cameras A Multi sensor IR tracking system A high speed Internet connection (TBD)Software: ViaVoice (IBM) Reality Fusions SDK SDK for IR tracking system FaceMaker Melody Assistant SDK for the touch screen Microsoft Photo Editor QuickTime Viewer QuickTime VR Viewer Microsoft Active Player Friendly Librarian control interfacesCompaniesDisplay supports: Hammacher Schlemmerhttp://www.hammacher.com/DefaultPage/default.asp?ContentPage=/publish/67468.htmInfrared 3D tracker: Origin Instrumentshttp://www.origin.com/Gesture and Display Position Software: Reality Fusionhttp://www.dti3d.com/Display video cameras: LogitechProprietary and conﬁdential July 2000 1Artificial, Inc.
http://www.realityfusion.com/gamecam/about_cam.htmlDigital Librarian: Margaret Andersonhttp://www.servtech.com/~mvail/home.html3D-face reconstruction: MicroLeadhttp://www.ulead.com/ff/runme.htm3D Lenticular Coating: Dimension Technologieshttp://www.dti3d.com/Voice Recognition: IBM ViaVoicehttp://www-4.ibm.com/software/speech/howtobuy/index.htmlMusic composing and playback: Myriad Software (AKA Guillion Bros. Software)http://www.myriad-online.com/Home Control Software: Crestron Remote Control Systemshttp://www.m-media.com/crestron_expan.htmlTouch Screen: Intersense Inc.http://www.isense.comComputer Workstation: Microspherehttp://www.microsphere.comCost: Windows PC ~3000.00 Graphics cards 3 @ ~350.00 Monitor arms 3 @ 275.00 LCD coatings 3 @ ~13000.00 Video Cameras 3 * 129.00 IR Touch Screen email@example.com IR tracking system 2890.00 Internet connection TBD Software Librarian TBD FaceMaker 25.00 Reality Fusion 49.00 IBM ViaVoice 149.99Notes:Concerning a flat display with no gaps: As was pointed out in describing the PanoramicViewer, to do so out of multiple displays requires a display with no border. At this point,I am not aware of any available. This is surprising.There must be a great demand for this characteristic in a display. This feature could beused by all the stadiums and large events to make extremely large configurable displaysof any type they need (a display is an analog of a pixel, more pixels more display). Weshould investigate further to see if this may be a profitable product.Proprietary and conﬁdential July 2000 1Artificial, Inc.
Unfortunately, as of right now I do not know of a way to solve this problem for such amarket. In the more limited case of the Portable Library, we have the situation of havingonly three monitors that can be placed in a side by side arrangement. We may be able touse mirrors and a rearrangement of the viewport to accomplish this. I will present a smallarticle if the interest is there.Concerning the serial ports: As specified, almost all the I/O is done via the serial port.We should however explore the use of other faster communication methods, for exampleFirewire. The serial ports are specified to insure a baseline level of communication so asto reduce unknowns and ensure success. A lighter, perhaps wireless, video transmissionof high resolution would be the most desirable.Concerning the motion trackers:As currently envisioned, the Portable Digital Library will have two separate ways ofcapturing the users position as well as his gestures. In one way, we make use of theReality Fusion software to input a users commands as well as to keep track of theposition of the displays. In the other way, we make use of an infrared tracking system,typically by tracking a small IR reflective dot on the persons head in real-time.The IR tracker is an expensive item, and one may think it overkill just to track the usershead position. However other methods, such as mechanical resistance measures ormagnetic field monitors etc. all require tethers, in addition some would require substantialengineering to integrate them into the library.Since we do have another method (Reality Fusion), one may ask why not use it for thehead tracking. The software is sophisticated enough to follow and recognize a personshand gestures. This is done by a real-time analysis of a video image (from a typicalinexpensive video cam) of the user.Unfortunately, since it is a software DSP process, it currently is too slow to use at a 60-Hertz rate. However in the future I think that this method is the one most likely to beuseful in a commercial market.For the library, we use the Reality Fusion video system for other functions, mostprominently as a user interface to the web browser and the video/audio controls of theportable library. The software is being continually revised and with a very well supportedSDK. It used to be available free of charge, but we now need a developers agreement ofsome sort:http://www.realityfusion.com/sdk.htm(Gordon, I will email it when received). An additional use of the Reality Fusion softwareis unique to the Portable Library. The three displays will each have a separate videocamera. This will be used not only to obtain the users gestures, but also of themselves.This will allow the library to change the video display format (size of desktop, etc) to theProprietary and conﬁdential July 2000 1Artificial, Inc.