Pick-and-Drop: A Direct Manipulation Technique for Multiple Computer Environments
Upcoming SlideShare
Loading in...5

Like this? Share it with your network


Pick-and-Drop: A Direct Manipulation Technique for Multiple Computer Environments

Uploaded on


More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 1

http://www.slideshare.net 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Pick-and-Drop: A Direct Manipulation Technique for Multiple Computer Environments Jun Rekimoto Sony Computer Science Laboratory Inc Oct.15.1997
  • 2. From Single Device UI to multi-Device UI Computer PC PDA PDA Kiosk Wall PC PDA PDA Current PC/GUIs are designed for One-user -- One-computer environment Future Computing Environment must be designed for Multi-user -- Multi-computer environments PDA
  • 3. Limitations of Single-Device GUIs
    • One-to-One dogma
      • Mouse and Keyboard are connected to one computer
      • cf. Paper and Pencil
    • No Direct-Manipulation technique that can interrelate two or more computers
      • e.g., Drag-and-drop between two computers
  • 4. Mouse Jungle
  • 5. Data Copying through Human Body Note: These computers are connected to the Network
  • 6. How people transfer information between nearby computers? % % % % % % A survey conducted on 40 engineers in Sony’s Software Labs.
  • 7. Multi-Computer (Device) UIs
    • No cognitive overload as the number of devices increases
    • Virtual / Physical Characteristics
      • Number, size, orientation, and location of devices
      • Effective combination of a variety of devices
      • Boundaries between computers/devices
        • “ Seamful but beautifully designed seams” [M.Weiser, Invited Talk at UIST94]
    • An Example: Pick-and-Drop
  • 8. Drag&Drop Pick-and-Drop Inter Computer Pick-and-Drop
  • 9. Information transition between virtual and physical spaces Computer Space 1 Physical Space Computer Space 2 Display Surface Pen virtually Carries the data Pick Drop
  • 10. Data Shadow
    • Visual feedback for pen carrying data
    Display Surface
  • 11. Applications of Pick-and-Drop Information exchange between PDAs Picking up information from Kiosk (push media) terminal
  • 12. Wall-sized Display + Palmtop Computer
  • 13. Video
  • 14. System Configuration Network Pen ID = 1 Wall Size Display (or Physical Wall) Desktop Display PDA (Nomadic PCs) ID Data 1 xx.gif Pen ID = 2 Pen Manager
  • 15. Pick-and-Drop between Paper and Computer Tablet Printed Objects Page ID Camera Object ID Page Code Pick Position
  • 16. Discussions
    • Traditional Data Transfer
      • Copy data from C:/ My Document of Machine A to /user/local ….
    • Pick-and-Drop
      • Copy data from here to there
    Symbolic Spatial
  • 17. Summary
    • Multi-Computer (Multi-Device) UI
      • as a new paradigm of direct-manipulation techniques
    • Pick-and-Drop:
      • information transfer technique for multi-device environment
  • 18. Future Work
    • Generalized Pick-and-Drop
      • information exchange between electronic appliance (e.g., VCR), ubiquitous computers, and physical objects (paper, wall, etc.)
    • Pick-and-Drop of active objects
      • e.g., JavaBeans
    • Application to distributed environments
      • Pick-and-drop over video conferencing
  • 19. Pick-and-Drop over ClearBoard ENVISIONMENT
  • 20.  
  • 21. State Transition Diagram PEN-DOWN DRAG PEN-UP MOVE Time Out DRAG PEN-UP MOVE DOWN Time Out DRAG MOVE PICK DROP PEN-UP Time Out
  • 22. Pick-Move, Pick-Copy Move Copy
  • 23. Related Work (1/2)
    • DigitalDesk [Wellner 93]
      • Data transfer from paper to the desk
    • LiveBoard + PARC TAB [Want et.al 95]
      • PARC Tab as a remote pointer for the LiveBoard
    • Graspable UIs [Fitzmaurice et.al 95]
      • Using physical handles (bricks) to manipulate computer objects
  • 24. Related Work (2/2)
    • PDA-ITV [Robertson et.al 96]
      • PDA as a remote controller for the interactive TV
    • PaperLink [Arai et.al 97]
      • Picking up information from paper
    • Ultra Magic Key (UMK) [Usuda et.al 97]
      • TV manipulation using a paper booklet
    • Tangible Bits [Ishii et.al 97]
      • Physical artifact for “Bits” (phicons)