Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Computational Fields meet Augmented Reality: Perspectives and Challenges

157 views

Published on

Recently, two different techniques emerged that are tailored to environments pervaded of computational devices. On the one hand, aggregate programming, and especially computational fields-based programming, is a promising abstraction for coordinating the activities of multiple situated devices. On the other hand, augmented reality is emerging as new means of interaction with both software and physical entities. We note that both computational fields and augmented reality are tightly bound to the physical world, and that they both enrich it, with collective computation and augmented information respectively. This work presents an initial analysis of possible future research directions that involve these techniques, discussing some possible ways of integrating them.

Published in: Science
  • Be the first to comment

  • Be the first to like this

Computational Fields meet Augmented Reality: Perspectives and Challenges

  1. 1. Computational Fields meet Augmented Reality: Perspectives and Challenges Danilo Pianini, Angelo Croatti, Mirko Viroli, Alessandro Ricci {danilo.pianini, a.croatti, mirko.viroli, a.ricci}@unibo.it Alma Mater Studiorum—Universit`a di Bologna a Cesena Spatial and COllective PErvasive Computing Systems (SCOPES) September 21, 2015 - Cambridge, USA Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 1 / 25
  2. 2. Outline 1 Introduction Motivation 2 Augmented Reality Basics 3 Computational fields Introduction to aggregate programming and computational fields Languages for aggregate computing 4 Augmented Fields Augmented reality as visual interface for fields Augmented reality-based input for CF program Computational fields as enabling technology for AR applications Example 5 Conclusion Conclusion and future work Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 2 / 25
  3. 3. Introduction Motivation Outline 1 Introduction Motivation 2 Augmented Reality Basics 3 Computational fields Introduction to aggregate programming and computational fields Languages for aggregate computing 4 Augmented Fields Augmented reality as visual interface for fields Augmented reality-based input for CF program Computational fields as enabling technology for AR applications Example 5 Conclusion Conclusion and future work Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 3 / 25
  4. 4. Introduction Motivation Why Augmented Reality and Computational Fields? Differences They are definitely different entities CF is a programming abstraction. AR is a mean of interaction. CF focuses on collectivity, AR (mostly) on the single user. Commonalities They share a common context of application Both are devoted at environments pervaded with computational devices Both are bound to the physical world Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 3 / 25
  5. 5. Augmented Reality Basics Outline 1 Introduction Motivation 2 Augmented Reality Basics 3 Computational fields Introduction to aggregate programming and computational fields Languages for aggregate computing 4 Augmented Fields Augmented reality as visual interface for fields Augmented reality-based input for CF program Computational fields as enabling technology for AR applications Example 5 Conclusion Conclusion and future work Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 4 / 25
  6. 6. Augmented Reality Basics Definition Augmented Reality is a technology that allows the user to see the real world, with virtual objects superimposed upon or mixed with the real world [Azu97]. The ultimate goal is a seamless integration of reality and virtuality Mostly via see-through devices Images from Microsoft Hololens presentation video Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 4 / 25
  7. 7. Augmented Reality Basics Degrees of augmentation GHOST AGENT GHOST BODY STREET LAMP PLAYER BODY PLAYER ASSISTANT AGENT MIRROR WORLD PHYSICAL WORLD 8 Augmentation can happen at different levels: 1 Unrelated to the elements in the user’s FOV E.g. displaying a map on a visual overlay 2 Dynamically associated to elements in the user’s FOV E.g. displaying historical information next to a monument in FOV 3 Entirely virtual and interactive elements on real world E.g. the two images of the previous slide, the ghost game [RPTC15] Image on left: DashWare. Image on right from [RPTC15] Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 5 / 25
  8. 8. Computational fields Introduction to aggregate programming and computational fields Outline 1 Introduction Motivation 2 Augmented Reality Basics 3 Computational fields Introduction to aggregate programming and computational fields Languages for aggregate computing 4 Augmented Fields Augmented reality as visual interface for fields Augmented reality-based input for CF program Computational fields as enabling technology for AR applications Example 5 Conclusion Conclusion and future work Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 6 / 25
  9. 9. Computational fields Introduction to aggregate programming and computational fields Manifesto of aggregate computing Main observation In a world with ever increasing number of deployed devices, one conveniently views a dense aggregation of interacting devices as a discrete approximation of the space of computational environment through which they are distributed. neighborhood device Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 6 / 25
  10. 10. Computational fields Introduction to aggregate programming and computational fields Manifesto of aggregate computing Main observation 1 the reference “machine” for pervasive computing processes is abstracted to be the continuum of computational devices; 2 the reference “elaboration process” is the manipulation of a physically distributed data structure; 3 how computation is carried on by the cooperation of devices in that region is hidden “under-the-hood” of the model/platform neighborhood device Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 7 / 25
  11. 11. Computational fields Introduction to aggregate programming and computational fields Pervasive continuum and computational fields Aggregate computing From “what value the single device computes”... ...to “what distributed data structure the pervasive fabric computes” The notion of computational field arises [MZ09, BB06] ⇒ a map from the space to (structured) values typically evolving over time, possibly self-stabilising better “understood” on continuous domains neighborhood device Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 8 / 25
  12. 12. Computational fields Languages for aggregate computing Outline 1 Introduction Motivation 2 Augmented Reality Basics 3 Computational fields Introduction to aggregate programming and computational fields Languages for aggregate computing 4 Augmented Fields Augmented reality as visual interface for fields Augmented reality-based input for CF program Computational fields as enabling technology for AR applications Example 5 Conclusion Conclusion and future work Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 9 / 25
  13. 13. Computational fields Languages for aggregate computing Existing languages MIT Proto [BB06] is the most known and successful Developed at MIT and maintained at BBN Technologies Functional language, LISP-like syntax (I know you hate it too) All devices run the same program Computation happens in rounds: Every device sleeps for some time Processes the messages received from the neighbours Executes its program Sends all the neighbours its result Complex operational semantics Difficult to maintain and extend Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 9 / 25
  14. 14. Computational fields Languages for aggregate computing Field Calculus A “distillate” of Proto Provides a lightweight operational semantics [VDB13] ((((LISP-like syntax)))) Still a functional language Simple enough to formally prove properties, powerful enough to be universal (proved!) Theoretical object, no runtime nor simulation tool provided Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 10 / 25
  15. 15. Computational fields Languages for aggregate computing Protelis [PVB15] Ordinary language features Functional language Same operational semantics of the field calculus C-family syntax with infix operators Java interoperability: static methods imports and calls, method invocation with dynamic binding Higher order functions (functions as arguments, lambdas) Dynamic code Suggested read [BPV15] Jacob Beal, Danilo Pianini, and Mirko Viroli. Aggregate programming for the internet of things. IEEE Computer, 48(9), 2015 Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 11 / 25
  16. 16. Computational fields Languages for aggregate computing Code example def count() { rep(x<-0) { x + 1 } } def maxh(field) { maxHood(nbr(field)) } def distanceTo(source) { rep (d <- Infinity) { mux (source) { 0 } else { minHood(nbr(d) + nbrRange) } } } def distanceToWithObstacle(source, obstacle) { if (obstacle) { Infinity } else { distanceTo(source) } } Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 12 / 25
  17. 17. Computational fields Languages for aggregate computing Architecture Implementation, distribution Based on Xtext Eclipse plugin Now distributed through Maven Central a Integrated with Alchemist [PMV13] Easy to port to new platforms a artifact: org.protelis:protelis Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 13 / 25
  18. 18. Augmented Fields Augmented reality as visual interface for fields Outline 1 Introduction Motivation 2 Augmented Reality Basics 3 Computational fields Introduction to aggregate programming and computational fields Languages for aggregate computing 4 Augmented Fields Augmented reality as visual interface for fields Augmented reality-based input for CF program Computational fields as enabling technology for AR applications Example 5 Conclusion Conclusion and future work Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 14 / 25
  19. 19. Augmented Fields Augmented reality as visual interface for fields See the fields Inject the ability to use AR to see the ongoing CF computation How Read the program output, display it properly Requires basically no intervention on the CF program Why The mapped field may be directly mapped to the user (e.g. a crowd warning system) Useful for seeing the shape of a field while developing Limitations Some fields cannot be trivially mapped to a graphical output How would you map a field of anonymous functions? No interaction Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 14 / 25
  20. 20. Augmented Fields Augmented reality as visual interface for fields User view Crowd detection scenario: change the perceived hue in such a way that dangerously dense areas are properly advertised. Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 15 / 25
  21. 21. Augmented Fields Augmented reality as visual interface for fields Developer view Crowd detection program development: see the field. Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 16 / 25
  22. 22. Augmented Fields Augmented reality-based input for CF program Outline 1 Introduction Motivation 2 Augmented Reality Basics 3 Computational fields Introduction to aggregate programming and computational fields Languages for aggregate computing 4 Augmented Fields Augmented reality as visual interface for fields Augmented reality-based input for CF program Computational fields as enabling technology for AR applications Example 5 Conclusion Conclusion and future work Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 17 / 25
  23. 23. Augmented Fields Augmented reality-based input for CF program Weak interaction Provide input to a CF program through AR interaction How Connect the (processed) AR input to the “sensors” of a CF program Requires basically no intervention on the CF program New information must be added to an existing device participating the aggregate Why Some system settings could be very naturally set by means of a gesture Limitations The new information is constrained in a device (a single point in space) Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 17 / 25
  24. 24. Augmented Fields Augmented reality-based input for CF program Strong interaction GHOST AGENT GHOST BODY STREET LAMP PLAYER BODY PLAYER ASSISTANT AGENT MIRROR WORLD PHYSICAL WORLD 8 Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 18 / 25
  25. 25. Augmented Fields Augmented reality-based input for CF program Strong interaction Inject new information that lives in a “Mirror World” How Modify the CF program to “sustain” the existence of entities in areas where no device is located e.g. with a field of mappings between virtual entities and their position Why Support for highest degree of augmented reality Mixed real-virtual collective applications Limitations The program must be conceived with this kind of interaction in mind Harder to realize Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 19 / 25
  26. 26. Augmented Fields Computational fields as enabling technology for AR applications Outline 1 Introduction Motivation 2 Augmented Reality Basics 3 Computational fields Introduction to aggregate programming and computational fields Languages for aggregate computing 4 Augmented Fields Augmented reality as visual interface for fields Augmented reality-based input for CF program Computational fields as enabling technology for AR applications Example 5 Conclusion Conclusion and future work Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 20 / 25
  27. 27. Augmented Fields Computational fields as enabling technology for AR applications A look the other way around We have discussed increasing degrees of interaction between augmented reality and computational fields, up to a “strong” interaction, but... ...are computational fields a valid abstraction for creating such programs? Progression “Weak” interaction is almost a free lunch “Strong” interaction seems powerful, but requires CF programs to be designed with AR in mind AR is no longer confined to be an I/O component of the system, it impacts the “business logic” of the application instead Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 20 / 25
  28. 28. Augmented Fields Computational fields as enabling technology for AR applications Mirror Worlds on Augmented Fields Basic strategy: Switch from a simple field of mirrored entities to a field of entities with location Exploit CF programming to keep the entities alive and their status aligned on multiple devices If entities have their own behavior, mobile code support is required [DVPB15] Reusable building blocks can be exploited in order to devise a “standard” solution [BPV15] Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 21 / 25
  29. 29. Augmented Fields Example Outline 1 Introduction Motivation 2 Augmented Reality Basics 3 Computational fields Introduction to aggregate programming and computational fields Languages for aggregate computing 4 Augmented Fields Augmented reality as visual interface for fields Augmented reality-based input for CF program Computational fields as enabling technology for AR applications Example 5 Conclusion Conclusion and future work Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 22 / 25
  30. 30. Augmented Fields Example Aggregate triage Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 22 / 25
  31. 31. Conclusion Conclusion and future work Outline 1 Introduction Motivation 2 Augmented Reality Basics 3 Computational fields Introduction to aggregate programming and computational fields Languages for aggregate computing 4 Augmented Fields Augmented reality as visual interface for fields Augmented reality-based input for CF program Computational fields as enabling technology for AR applications Example 5 Conclusion Conclusion and future work Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 23 / 25
  32. 32. Conclusion Conclusion and future work Conclusion We noted that both computational fields and augmented reality work best in environments densely populated with computational devices We have explored increasing degrees of integration AR as UI ⇒ not hard, no change in aggregate programs Local input from AR devices ⇒ still not hard, still no change Injection of (possibly proactive) augmented entities, ⇒ requires CF programs to be properly designed Modern aggregate programming languages support code mobility and reusability We believe that CF are a promising programming abstraction for supporting AR applications, and languages and technologies are maturing Future work Build a working demo is the most effective way of experimentally confirm our hypothesis. Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 23 / 25
  33. 33. References References I Ronald Azuma. A survey of augmented reality. 6(4):355–385, 1997. Jacob Beal and Jonathan Bachrach. Infrastructure for engineered emergence on sensor/actuator networks. IEEE Intelligent Systems, 21(2), 2006. Jacob Beal, Danilo Pianini, and Mirko Viroli. Aggregate programming for the internet of things. IEEE Computer, 48(9), 2015. Ferruccio Damiani, Mirko Viroli, Danilo Pianini, and Jacob Beal. Code mobility meets self-organisation: A higher-order calculus of computational fields. In Susanne Graf and Mahesh Viswanathan, editors, Formal Techniques for Distributed Objects, Components, and Systems, volume 9039 of Lecture Notes in Computer Science, pages 113–128. Springer International Publishing, 2015. Marco Mamei and Franco Zambonelli. Programming pervasive and mobile computing applications: The tota approach. ACM Transactions on Software Engineering and Methodologies, 18(4), 2009. Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 24 / 25
  34. 34. References References II Danilo Pianini, Sara Montagna, and Mirko Viroli. Chemical-oriented simulation of computational systems with Alchemist. Journal of Simulation, 2013. Danilo Pianini, Mirko Viroli, and Jacob Beal. Protelis: Practical aggregate programming. In Roger L. Wainwright, Juan Manuel Corchado, Alessio Bechini, and Jiman Hong, editors, Proceedings of the 30th Annual ACM Symposium on Applied Computing, Salamanca, Spain, April 13-17, 2015, pages 1846–1853, Salamanca, Spain, 2015. ACM. Alessandro Ricci, Michele Piunti, Luca Tummolini, and Cristiano Castelfranchi. The mirror world: Preparing for mixed-reality living. IEEE Pervasive Computing, 14(2):60–63, 2015. Mirko Viroli, Ferruccio Damiani, and Jacob Beal. A calculus of computational fields. In Carlos Canal and Massimo Villari, editors, Advances in Service-Oriented and Cloud Computing, volume 393 of Communications in Computer and Information Sci., pages 114–128. Springer Berlin Heidelberg, 2013. Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 25 / 25

×