Lessons learned in inserting AR into IKEA's native mobile app
•Download as PPTX, PDF•
3 likes•3,285 views
This document discusses lessons learned from inserting augmented reality (AR) into IKEA's native mobile app using Unity as a library. It covers perspectives on building an iOS project with Unity as a framework, communicating between native and Unity code, and combining the two. Common issues and sample workflows for continuous integration/continuous delivery (CI/CD) with Unity as a library are presented. The benefits for IKEA of reusing AR technology by distributing it to other franchises are discussed, as well as new opportunities for encapsulating Unity3D in native applications that require 3D views.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Lessons learned in inserting AR into IKEA's native mobile app
Similar to Lessons learned in inserting AR into IKEA's native mobile app (20)
More from Unity Technologies
More from Unity Technologies (20)
Recently uploaded
Recently uploaded (20)
Lessons learned in inserting AR into IKEA's native mobile app
- 2. Lessons learned in inserting AR into IKEA's native mobile app Jefferson Scomacao – PTC AR Field Lead Manager
- 3. IKEA Place and Unity as a Library 3
- 4. Unity as a Library: what is that anyway? 4
- 5. Unity as a Library 5 — Create your iOS Project — Communication with Unity to and from native — Built Binary
- 6. Unity Perspective 6 — Build Process — Unity Framework — Interface Code
- 7. Native Perspective 7 — Usual Workspace and XCode Project configuration — Add your native code — Add your Unity built Framework 1 2
- 8. Combined Perspective Your Unity View — UIWindow for your Unity project — It becomes one new object in Native language — Easy to track and maintain 8 Native Views
- 9. New flows new features, new challenges! 9
- 10. New workflow 10
- 11. Versioning of Unity as a Library can be set in different ways. Adapt to what is best for your CI/CD. Common Issues 11 CI/CD with Unity as a Library sample flow. Here you can see a good approach.
- 12. Building Blocks: Unity to Native 12 C# External Calls extern “C”
- 13. Building Blocks: Native to Native 13 extern “C” Protocol
- 14. Building Blocks: Protocol 14 Protocol Host Application
- 15. Building Blocks: Native to Unity 15 Native Application UnityFramework C# Unity
- 16. Building Blocks - Combined 16
- 17. IKEA use of Unity as a Library: benefits how to reuse AR tech 17
- 18. The best re-use of code and projects: 18 IKEA Place Distribute AR to other franchises
- 19. New opportunities: where to go now? 19
- 20. Native applications that requires 3D views can encapsulate Unity3D in new ways including other frameworks and new ways to monetize or license. Heavy use of Unity3D capabilities and using the power of native UI and device specific features without creating extra code for Unity. Reutilization of code base for the same feature in different projects as well as different licensing models. Where Unity as a Library can be used: 20 $
Editor's Notes
- Hi everyone, great to be here, I am Jefferson from Twnkls, a PTC company and -> Move to Slide(1)
- Today you will learn about a new way of building your projects. In this special case for iOS platform and using IkeaPlace as study case. You can call it “Unity as a Library” (move around stage and signal)
- And before going to what is Unity as a Library, I would like to give a bit of context for this presentation. IKEA is a major Furniture retailer and its AR powered application is called IkeaPlace that was developed for the arkit release in 2017. The concept of AR worked so well that IKEA wanted to have AR on all their applications. To do this we would need to code libraries, ways and endless efforts to achieve this. This is where Unity as a Library comes into place to reduce the amount of work and costs to distribute AR features into other IKEA applications. IkeaPlace is known to set the standards for AR retail applications and many times referenced in other conferences, media and other Unite events as the "most performant ar application made with Unity", Berlin 2018. We will see here how it was done and how it can be used in other ways.
- The Unity as a Library was introduced in the Unity3D 2019 cycle and represents a new deployment format for iOS projects, today we are discussing the iOS version based on Ikea case and now it is available for Android too. (Signal, and move) what does it mean? From Unity Editor you can create a package (in this case for iOS it is called Framework) and use inside another XCode project. In simple terms, Unity can output a package and later we can use inside another application (another iOS application)
- This is the highlevel flow for the new Unity as a Library, After you build your Unity application to export to iOS platform, we can integrate as a simple XCode Project. Inside a XCode workspace (that is just a way to have multiple projects into one solution) you will have two distincts parts, one is your Native application you want to integrate Unity as a Library and the other one is the Actual Unity as a Library Project Binary. To make the communication between then work properly we need to have a proper communication layer. It is important to understand that Unity as a Library builds will be part of your Native application, meaning that at the end everything will become one full application.
- From Unity perspective, the build process is almost the same, except a few extra code you need to have in order to output the files properly. Image -> Diagram Flow -> Unity ios Build -> Framework created -> Xcode Project
- From Native side it is transparent, as inside XCode you have a completely distinct extra project (called Unity-iPhone or anything else you want to call it). Image -> Xcode project setup -> Steps 1) Your Workspace 2) Your Native iOS application and Unity Project 3) Setup Complete
- And after your combine both Native and Unity as a Library together you will have a seamless integration of your features. For the IkeaPlace we have all native UI done in the native side and all AR features done using Unity. For the final result, your application will contain the Unity View and you can organize your code flow around it.
- When starting a new project with Unity as a Library in focus, a few new challenges will be waiting for you.
- The basic idea is to use what it is already given to you, in this case, Two XCode projects. The Unity one can be always used by different Other projects as they can communicate. Saying this, it is always important to define the communication you will have between native and unity. This will save you time and make your life easier when reusing the project. Diagram ->
- The flow of work creates two projects, one native and one Unity that needs to be maintained and compiled. Also, the versioning of both projects needs to be accounted for. Use of separate repositories will work perfect. Choose of integration tools are also interesting, Git, CocoaPods and Carthage are proven to work as expected with Two projects (signal) one Native, one Unity. In the end, the Unity side is just a big compiled chunk (or a framework) to be used by your Native app. And here on the right you can see the basic setup for the project after you build, A simple WorkSpace that has your project and the Unity compiled project added as XCode project. (a few details to look at, Data folder needs to have the proper Target)
- Ok, we now know the basic principles, now lets see a bit how the integration between unity as a library and another Xcode project works. First we start as any usual Unity iOS project where you need to communicate with Native API's, we C# unity we can call methods on XCode in the "C" level. Nothing new, any Unity iOS plugin will start here.
- From Unity now we access a method in C inside your XCode project. We should define a Protocol where your classes will subscribe and receive proper method calls.
- And inside your Native Project you need to extend the protocol and receive the calls for it. There are several ways to organize your project around this. This completes the cycle to Send information from Unity to your Native application.
- And to send information back to Unity, we can use the tools already available inside the Unity as a Library code, basically we have the SendMessage command that outputs data to Unity from anywhere UnityFramework is acessible. Remember, UnityFramework is the actual Built project from Unity as a Library you embed into your native part.
- Ok, now we know the integration has two parts, your Native application and your Unity as a Library build. Also we can see here the complete flow to send information from Unity to your native, the use of protocol is one of the possibilities, there are other ways you can try yourself. Sending information back to Unity is simple and ready to use from UnityFramework.
- A little background on Ikea's experiment with AR-- they wanted to test the power of AR, but you had to build a separate app to do it. The experiment worked so well, they wanted to insert AR into their native mobile app and the ability to use Unity as a Library allowed them to do this. Now it is possible to reuse the core AR feature inside virtually all IKEA applications with the effort of implementing a minimal code as the interface between Native and Unity was kept simple and efficient.
- Today, IKEA Place 4.0 and newer versions use the Unity as a Library way. The encapsulated AR feature is made with Unity and redistributed back to other IKEA products. While the UI and Application flow is commanded by the Native Application. The reuse of AR features inside other IKEA products will reduce the costs and make it possible to have a consistent look and feel for the same feature across multiple vendor and platforms. Diagram -> Ikea Place Logo -> Small diagram (app, arc) -> arrow from arc to list of IKEA icons (franchises)
- Using Unity as a Library open new opportunities for use 3D content on new and existing Apple devices. You can implement all Unity3D features and create your 3D solution as you see fit. Create communication for user interaction and create new outstanding features, combining Unity and Native power. Using the best of both worlds.
- Unity as a Library opens up new ways to distribute and work with your Unity projects, The base case you create your package and redistribute to other Native applications. This is the IkeaPlace use and there are more possibilities here as we can also integration into previous existing applications. There are cases the Unity as a Library by itself is a good option when you have a heavy usage of Native UI where Native code will perform better and have more options than the Unity one as the last is meant to be compatible with multiple platforms. In this case the separation of features will make your application even more beautiful and performant. Another possibility that is now possible with both Android and iOS Unity as a Library is to create products out of it and create new monetization ways for your project. The licensing models can be worked in different ways now As Unity as a Library can encapsulate virtually all current Unity features and more giving you the freedom to rethink strategies and product lifecycles. --- So, for today we learned how Ikea is saving costs and improving their AR platform by distributing to all their franchises. Also we saw how it is done and understand that it is a new set of possibilities for developers and companies.
- I would like to give special thanks to the Twnkls team that made all this possible in a record timeframe, and of course, a big thank you for Ikea and Space10 for the trust put into this project. Fell free to reach to understand more using jraulino@ptc.com or via website www.twnkls.com Thank you! And lets open the QA session.