The document outlines the functionalities and applications of computer vision and lidar technology, focusing on Luminar's high-performance lidar sensors used for autonomous vehicles. It describes the integration of Unity for data simulation, level design, and sensor model implementation, emphasizing the need for accurate data capture and scenario building. Additionally, it highlights challenges in object placement, traffic simulation, and performance considerations in a gaming context.

1. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation

2. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
What is computer vision?

3. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
What is lidar?

4. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Who is Luminar?

5. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Who is Luminar?
We design and manufacture high performance lidar sensors for use in
autonomous vehicles.

6. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Who is Luminar?
We measure millions
of points per second,
and put that resolution
where it matters most.
This allows our
sensors to see not just
where objects are, but
what they are — even
at distance.

7. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Who is Luminar?

8. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Who is Luminar?

9. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Why use Unity?

10. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Why use Unity?

11. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Simulated sensor model

12. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
How it works
1. C/C++ pattern generator with a C# wrapper as a DLL
2. Sensor model (prefab + C#) as a DLL
3. Job system compatibility

13. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Data visualizer
A data visualizer is an example of how to use the DLLs

14. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Data simulation
1) Display and
manipulate data
whether it is
recorded or real
time
2) Working at the data
level means the
same code can be
used on recorded
data or data coming
from a live sensor –
the simulation does
not know or care
where the data is
coming from

15. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Unity time!
UNITY PROJECTS

16. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Unity time!
SENSOR MODEL
INTEGRATION

17. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Unity time!
HIGHWAY
SCENARIO

18. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Gathering data requires level design
++
Vehicle Physics PluginSensor model Road + Traffic Plugin

19. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Similar to game development – player camera
Sensor model
1) You still need a
traditional camera
2) There are times where
your users need to see
what is happening with
the data and explore it
3) Decoupling your
camera from your lidar
sensor is essential
4) We suggest setting up
a simple fly camera
you can toggle

20. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Similar to game development – scenario building
Road + Traffic Plugin
Most scenarios center around
two goals: creating data for
training AI and building custom
scenes to test the 1% of edge
cases the industry is still trying
to solve. Regardless of which
problem you are trying to solve
you will be diving into
traditional level design. There
are several unity packages on
the asset store available to get
you started.

21. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Data capture requirements
1) Every object picked up by
our simulated sensor
requires an accurate
collider mesh
2) Not a requirement, but we
recommend making a new
layer to organize objects
in lidar – helps when
juggling between multiple
sensor types
3) In addition to point clouds,
we need to capture the
bounds objects
4) We use a component
attached to all tracked
objects allowing us to
display perfectly accurate
2D and 3D bounds

22. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Animation curves + splines = easily configurable highway scenario
1. We are a small team and tried to make this as simple as possible to build,
maintain, and extend
2. Splines are the foundation for generating our highway scenario
3. Animation curves control the elevation and layout of the road

23. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Placing objects and controlling traffic
1. Objects on the road are procedurally placed using offsets from the main spline
2. Traffic nodes are positioned with the same system and linked together creating
a typical traffic network
3. Plug in data from street/map plugins and simulate real roads

24. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Speed bumps and road blocks
1. Game assets don’t always work when using ray casting e.g. foliage
2. Performance and timing are a juggling act
3. Developing for Linux is a mixed bag

25. CONFIDENTIAL: NOT FOR EXTERNAL DISTRIBUTION
Computer Vision Simulation
Questions?