For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2021/10/10-things-you-must-know-before-designing-your-own-camera-a-presentation-from-panopteo/ Alex Fink, consultant at Panopteo, presents the “10 Things You Must Know Before Designing Your Own Camera” tutorial at the May 2021 Embedded Vision Summit. Computer vision requires vision. This is why companies that use computer vision often decide they need to create a custom camera module (and perhaps other custom sensors) that meets the specific needs of their unique application. This presentation will help you understand how cameras are different from other types of electronic products; what mistakes companies often make when attempting to design their own cameras; and what you can do to end up with cameras that are built on spec, on schedule and on budget.

1. © 2021 Panopteo
10 Things You Must Know
Before Designing
Your Own Camera
Alex Fink
Panopteo

2. Things that AREN’T

3. © 2021 Panopteo
Thing 1 - a camera is not a widget
Very few contract manufacturers know how to handle optics
3

4. © 2021 Panopteo
Thing 1 - a camera is not a widget
Even fewer know how to do geometric calibrations…
4

5. © 2021 Panopteo
Thing 1 - a camera is not a widget
The more specialized skills you require, the fewer CMs will qualify
5

6. © 2021 Panopteo
Thing 1 - a camera is not a widget
If you shop later (after you know your exact requirements)
You’ll shop better
6

7. © 2021 Panopteo
Thing 2 - a camera does not have ONE key component
7
What lens people think
What sensor people think
What SoC people think

8. © 2021 Panopteo
Thing 2 - a camera does not have ONE key component
A univariate approach will not make a great camera
It’s better to ask -- what does the entire pipeline look like?
8
Function 1 Function 2 Function 3 Function .. Function N-1 Function N
Component A Component .. Component K
Device I Device II

9. © 2021 Panopteo
Thing 2 - a camera does not have ONE key component
And what do you mean by “camera”?
9
Module
Webcam
Point-and-shoot
Serializer
Coax cable
Deserializer
Image
Signal
Processor
MIPI-to-USB
USB cable
System
On a
Chip
SD
WiFi
Ethernet
USB
HDMI
…

10. © 2021 Panopteo
Thing 3 - computer vision development is not camera
development
Scrums
Sprints
Frequent iteration
Fail fast!
But
Do these methods work when compilation takes 3-6 months?
10

11. © 2021 Panopteo
Thing 3 - computer vision development is not camera
development
State-of-the-art tech is better than old tech
But
What if the state-of-the-art hasn’t shipped yet?
Do you want to be the beta-tester for your vendors’ experiments?
11

12. © 2021 Panopteo
Thing 3 - computer vision development is not camera
development
The best spec is “as good as possible”
But
Can you select vendors and design partners to meet this spec?
What would you write in the SOWs?
12

13. Things that ARE

14. © 2021 Panopteo
Thing 4 – good HW design starts from the end
(and works backwards)
The camera has users (human or otherwise)
The users have use-cases
The use-cases can succeed or fail
What optical requirements will determine the outcome?
14

15. © 2021 Panopteo
Thing 4 – good HW design starts from the end
(and works backwards)
The product needs to ship on a particular date
The program has a budget
What architecture and components can be used to achieve the requirements --
on schedule and on budget?
15

16. © 2021 Panopteo
Thing 4 – good HW design starts from the end
(and works backwards)
Optics are a subset of physics
Most things are either feasible or not feasible
Doing the math is cheaper than building HW!
16

17. © 2021 Panopteo
Thing 5 – platforms are sticky
Some choices are harder to change than others
Software engineers don’t like it when the platform is swapped under their feet
but
The platform you prototype on might not be shippable
17

18. © 2021 Panopteo
Thing 5 – platforms are sticky
Many of my customers start out on open-source HW
Many of my customers start out on off-the-shelf HW
Few of my customers ship products on either
18

19. © 2021 Panopteo
Thing 5 – platforms are sticky
It’s best to make all prototypes on something that can be a mass-production platform
19

20. Things that OUGHT to be (done)

21. © 2021 Panopteo
Thing 6 – outsource anything that isn't your core
competence
Do you really need optics engineers in-house?
21

22. © 2021 Panopteo
Thing 6 – outsource anything that isn't your core
competence
Do you really need real-time firmware engineers in-house?
22

23. © 2021 Panopteo
Thing 7 – set a clear line between your engineering and
your vendors' engineering
If possible, separate on-camera work
from the rest of the system
23
Optics ME
EE FW
Camera
…
The rest of your system
AI/ML ME
EE FW
SW Cloud
Vendor’s engineers Your engineers
Single interface

24. © 2021 Panopteo
Thing 7 – set a clear line between your engineering and
your vendors' engineering
Who is in charge of integrating every new HW or FW version of the camera?
24
Sensor ISP Compression
Output
interface
Data
Ingestion
…
Camera The rest of your system

25. © 2021 Panopteo
Thing 8 – set clear acceptance criteria for vendors'
deliverables
If the camera were a standalone device, how would you test it?
How do your real-world criteria map to camera criteria?
25

26. © 2021 Panopteo
Thing 8 – set clear acceptance criteria for vendors'
deliverables
Is there a list of functions and test-cases?
Can you reject deliverables?
Can the vendor disagree?
26

27. © 2021 Panopteo
Thing 9 – test your own image quality
(even if the vendor already tested it)
What does "good image quality" mean to you?
27

28. © 2021 Panopteo
Thing 9 – test your own image quality
(even if the vendor already tested it)
In-lab tuning vs real-world scenarios
28

29. © 2021 Panopteo
Thing 10 – plan for V2.0
(before V1.0 is done)
Where do all the great ideas go?
29

30. © 2021 Panopteo
Thing 10 – plan for V2.0
(before V1.0 is done)
Your engineers should trust that shelved features aren't discarded features
30

31. © 2021 Panopteo
Thing 10 – plan for V2.0
(before V1.0 is done)
It helps to launch V2.0 pre-planning
In parallel to V1.0 development
31

32. © 2021 Panopteo
Recap
1. A camera is not a widget
2. A camera does not have ONE
key component
3. Computer vision development is
not camera development
4. Good HW design starts from the
end (and works backwards)
5. Platforms are sticky
32
6. Outsource anything that isn't your core
competence
7. Set a clear line between your engineering
and your vendors' engineering
8. Set clear acceptance criteria for vendors'
deliverables
9. Test your own image quality
(even if the vendor already tested it)
10. Plan for V2.0 (before V1.0 is done)

33. Thank you!

34. © 2021 Panopteo
References
34
Panopteo’s website:
http://panopteo.com/
My email:
alex@panopteo.com
My linkedin:
https://www.linkedin.com/in/temuchin43/