From 2D Map to Mobile 3D MirrorWorldA Live Virtual Advertising Use CaseYu You, David MurphyMixed Reality SolutionsNRC, Tampere, Finland
Mobile 3D Mirror World • Mirror world (MR) is “informationally-enhanced virtual models or ‘reflections’ of the physical world” . • Provides increased precision of an added dimension with more lively and immersive UX • A nice complement to Augmented Reality (AR), which is commonly used in real-time and on-line situation • Mobile MR can take advantage of mobile AR where mobile sensors and other positioning and tracking technologies are usedNokia Research Center
Motivation • study the mobile advertising scenarios from a narrow subject (street-level mirror world) and try to answer an immediate question: how and where to publish the advertisement content? • This paper focuses on: • Accurate but simple content automatic placement • Mobile content creation and visualizationNokia Research Center
Experiment setup• Data set • 360 degree panoramic street-level imagery • 3D building models with global geo. coordinates • Ground-terrain 3D mesh data• Technology • OpenGL-based client renders the building models and ground around the chosen panoramic image onto which the panoramic image is projectedA mobile client• Video 1: Nokia CitySceneNokia Research Center
The system in a nutshell • Web-centric end-to-end setup with RESTful Web APIs for ad. content management and mobile 3D mirror world client • WGS84 geo. boundary or proximity queries • KML/KMZ is the main data representation format • 3D objects are modeled separated in COLLADA filesNokia Research Center
Scalability consideration• Unlike ad. content data, which is rather dynamic, any other static data are stored in different servers, i.e. provided by Content Delivery Network (CDN) • Typically CDNs does not offer geo. query functionality • Each 3D data has a geo. coordinate and then its geo-hash is calculated (e.g. we use quad-key method) at a fixed resolution (zooming) level (i.e. 17, appr. 200m2) • Therefore an index is created for all data in that geo. boundary and stored in the file system using a directory naming convention, which is correspondent to its geo-hash value. So does the real data. • Client needs to fetch the metadata (index) files at one given geo. coordinate and then all 3D data.Nokia Research Center
Mobile 2D map-based content planning 1• Two types of ad. content representation 1. Static object like billboards standing on one location 2. Mobile 3D object along the vehicle driving road • 2D coordinates are needed only; the altitude is calculated from the ground terrain mesh• Problems 2 • Hard to position content accurately to the buildings, not to mention the façade • That reflects again the common problem in other systems where geo-tagged data (e.g. point of interest) are misplaced in the middle of streets; or even worse - on the wrong side of the streetNokia Research Center
3D content manual placement• High-value campaigns requires pixel-level accuracy• Developed a on-device authoring tool for non- expert users with following few steps • Choose the appropriate building façade • Drawing the size of the bounding box as the content placement or drawing canvas • Saving the drawing or inserted content with metadata (e.g. the normal to the façade and bounding box data in the 3D)• The content placement layer can be used to render any media data properly in 3D scene• However, manual placement is time-consuming and not scalableNokia Research Center
3D content heuristic/auto placement• Auto alignment from only 2D coordinates • Select the closest building model (the center of mass of the surrounding models) • Determine the intersected façade by casting a light-ray to the model from a predefined view point • External media like banner images are converted to SVG format to scaling of the scene (e.g. zooming in/out)Nokia Research Center
Video 2 Moving box for parking infoNokia Research Center
Findings and open issues• Alternative ad. placement (e.g. façade vs. ground) • Placing banners altered the appearance of the building facade, and thus deteriorated the representational value of the mirror world• Lighting, visual improvement • Illumination enhancement e.g. artificial lighting and content blending with proper shedder, but lighting blobs on facades and tree shadows are difficult to be segmented• Proper ad. content selection for moving artifacts • Fixed geo. location as the context wont always work for mobile artifacts• Study more metrics concerning mobile/3D-specific user actions for analytical processing • Navigation, messaging etc. • Zooming and panning• Various pricing strategies vs. moving speed and frequencies• Mobile Web app. vs. native application • One solution for all OSNokia Research Center
Summary• Bringing ad. content to 3D Mirror World allows innovative types of advertising, coming along with new challenges• We’ve proposed two basic types of ad. approaches but expect more realistic representations to appearNokia Research Center
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.