Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Accelerating Software
Development for MIPI
CSI-2℠ Cameras
Licinio Sousa
Synopsys, Inc.
The Challenge
Developing SW for Multiple HW Configs
•  Nowadays HW platforms are always evolving
•  It is hard for SW to k...
The Challenge
Developing SW for Multiple HW Configs
3
Complete	MIPI	Camera	and	Display	Solu6on
The Solution
Leverage Linux Resources to Reduce Design Efforts
•  Systems with embedded Linux can leverage existing
librar...
Agenda
•  Introduce a few Linux concepts and resources
•  Share our experience
•  Conclusion
Linux
# facts
- Linux is a free and open source software
~18 000 000 lines of code
- 5.64 patches accepted in mainline ker...
Linux
Wearables	Smartphones	
Servers	
Mainframes	
Embedded	systems
When is Linux’s Birthday?
8
From:torvalds@klaava.Helsinki.FI	(Linus	Benedict	Torvalds)	
	Newsgroup:	comp.os.minix	
	Subjec...
Inside Linux – Architecture
User Space vs. Kernel Space
•  User Space
•  If SW is running in userspace it’s usually called an Application
•  Memory ar...
User-Kernel Interface
•  User-kernel
interaction is made
via Linux APIs
•  Linux provides APIs in
order to allow
programme...
Video4Linux2 API
API for Video capture
•  One of the Linux APIs is V4L (Video4Linux) a.k.a. V4L2
•  V4L2 is an API for sup...
V4L2 API
Kernel	Space						
ApplicaIon	Space	
V4L2	Linux	Drivers	
V4L2	IOCTLs	
mplayer	 ffmpeg	 vlc	
V4L2	IOCTLs	
Hardware	...
V4L2 Devices and Subdevices
Modular Approach to Model the HW
•  Hardware is often
made of several
integrated circuits that...
Media Controller
API to Define Relation Between Device/Sub-Devices
•  Allows to discover a platform configuration and
conf...
Device/Sub-device and Media Entity
Source & Sink Pads to Represent Data Flow
Media Device - HW Pipeline
V4L2 Device/Sub-Devices Structured Using
MediaController API
Media	
Device	
Camera	
CSI-2	
Host...
Device Driver Interaction
Only High-Level Interaction is Needed
•  The main driver will call the sub-device drivers to
•  ...
Device Tree
Describes the HW Configuration
•  The format is expressive and
able to describe most board
design aspects incl...
Device Tree
Hardware Details Listed
•  The HW specific details will be available in the Device
Tree
•  HW addresses, IRQs ...
V4L2, Media Controller and Device Tree
Putting it All Together
21
V4L2, Media Controller and Device Tree
Putting it All Together
•  Device driver is designed using V4L2 API – API for
video...
V4L2, Media Controller and Device Tree
Putting it All Together
•  When kernel starts, it will look at the device tree and
...
V4L2, Media Controller and Device Tree
Putting it All Together
•  If a change in the platform is required (replace camera
...
OUR EXPERIENCE
CSI-2 SW Development
The Media Device HW Pipeline
26
CSI-2 SW Development
The Device Tree
Camera	
PHY	
Video	DMA	
CSI	
Controller
CSI-2 SW Development
Simple to Change CSI-2 Camera
Camera
MIPI CSI-2 Host IP Prototyping Kit
•  Synopsys offers an integrated HW/SW solution that
takes advantage of all these conce...
Leverage Kernel Tools to Accelerate CSI-2
SW Development
•  Application interacts with driver only using high level
Comman...
Thank You!
Synopsys’	broad	porkolio	of	DesignWare	MIPI	IP	soluIons	
synopsys.com/mipi
Upcoming SlideShare
Loading in …5
×

MIPI DevCon 2016: Accelerating Software Development for MIPI CSI-2 Cameras

1,358 views

Published on

MIPI CSI-2-compliant cameras are popular in mobile and mobile-influenced devices because of the specification’s ability to handle high image resolution over fast links with low-power consumption. SoC designers can accelerate their design process by integrating the software drivers to make initial development easier and directly control boot-up sequences. This presentation by Licinio Sousa of Synopsys describes how to use the existing host-side V4L2 API and V4L2 subdevice interfaces to ease the integration of a CSI-2-compliant camera with an existing system. This approach allows designers to easily change their camera without having to make any changes to the CSI-2 host driver.

Published in: Mobile
  • Be the first to comment

  • Be the first to like this

MIPI DevCon 2016: Accelerating Software Development for MIPI CSI-2 Cameras

  1. 1. Accelerating Software Development for MIPI CSI-2℠ Cameras Licinio Sousa Synopsys, Inc.
  2. 2. The Challenge Developing SW for Multiple HW Configs •  Nowadays HW platforms are always evolving •  It is hard for SW to keep up with the changes •  Using a monolithic approach might be simple to begin, but harder to mantain •  To support a new device, the designer then needs to either split the code or add support for another device in the initial code •  After a while the code will become “spaghetti code” •  Each device has specific configuration requirements and characteristics, which complicates the tasks of providing support and integration, due to limited code reusability •  Steep learning curve 2
  3. 3. The Challenge Developing SW for Multiple HW Configs 3 Complete MIPI Camera and Display Solu6on
  4. 4. The Solution Leverage Linux Resources to Reduce Design Efforts •  Systems with embedded Linux can leverage existing libraries, APIs and driver frameworks to reduce effort and accelerate SW development •  V4L2 API and modular design approach •  Media controller •  Device tree
  5. 5. Agenda •  Introduce a few Linux concepts and resources •  Share our experience •  Conclusion
  6. 6. Linux # facts - Linux is a free and open source software ~18 000 000 lines of code - 5.64 patches accepted in mainline kernel / h - More than 1000 engineers work in each release # who is working in Linux - Only 17.9 % make kernel hacking for free - The other 82.1% is made by top tech players
  7. 7. Linux Wearables Smartphones Servers Mainframes Embedded systems
  8. 8. When is Linux’s Birthday? 8 From:torvalds@klaava.Helsinki.FI (Linus Benedict Torvalds) Newsgroup: comp.os.minix Subject: GCC-1.40 and a posix ques6on Message-ID: 1991Jul13, 100050.9886@klaava.Helsinki.FI Date: 3 Jul 91 10:00:50 GMT Hello netlanders, Due a project I'm working on (in minix), I'm interested in the posix standard defini6on. Could somebody please point me to a (preferably) machine- readable format of the latest posix rules? Ftp-sites would be nice. Linus Torvalds torvalds@kruuna.helsinki.fi From:torvalds@klaava.Helsinki.FI (Linus Benedict Torvalds) Newsgroup: comp.os.minix Subject: What would you like to see most in minix? Summary: small poll for my new operaIng system Message-ID: 1991Aug25, 20578.9541@klaava.Helsinki.FI Date: 25 Aug 91 20:57:08 GMT OrganizaIon: University of Helsinki. Hello everybody out there using minix- I'm doing a (free) operaIng system (just a hobby, won't be big and professional like gnu) for 386(486) AT clones. (…) Any suggesIons are welcome, but I won't promise I'll implement them :-) Linus Torvalds torvalds@kruuna.helsinki.fi From: torvalds@klaava.Helsinki.FI (Linus Benedict Torvalds) Newsgroups: comp.os.minix Subject: Free minix-like kernel sources for 386-AT Message-ID: <1991Oct5.054106.4647@klaava.Helsinki.FI> Date: 5 Oct 91 05:41:06 GMT Organiza6on: University of Helsinki Do you pine for the nice days of minix-1.1, when men were men and wrote their own device drivers? Are you without a nice project and just dying to cut your teeth on a OS you can try to modify for your needs? Are you finding it frustra6ng when everything works on minix? No more all- nighters to get a ni`y program working? Then this post might be just for you :-) As I menIoned a month(?) ago, I'm working on a free version of a minix-lookalike for AT-386 computers. It has finally reached the stage where it's even usable (though may not be depending on what you want), and I am willing to put out the sources for wider distribuIon. It is just version 0.02 (+1 (very small) patch already), but I've successfully run bash/gcc/gnu-make/gnu- sed/compress etc under it. There are actually three "birthdays" one can idenIfy for Linux!!
  9. 9. Inside Linux – Architecture
  10. 10. User Space vs. Kernel Space •  User Space •  If SW is running in userspace it’s usually called an Application •  Memory area where application software will run •  Kernel Space •  Strictly reserved for running privileged OS kernel •  Runs kernel extensions and device drivers •  A driver has direct access to the HW
  11. 11. User-Kernel Interface •  User-kernel interaction is made via Linux APIs •  Linux provides APIs in order to allow programmers to write applications consistent with the operating environment •  Some APIs can be divided in 2 parts, which run in two different areas •  Kernel space •  User space
  12. 12. Video4Linux2 API API for Video capture •  One of the Linux APIs is V4L (Video4Linux) a.k.a. V4L2 •  V4L2 is an API for supporting real-time video capture in Linux •  V4L2 is used to build drivers for many USB webcams, TV tuners, smartphone cameras, and related devices •  With V4L2, applications can easily add video support in a standard way
  13. 13. V4L2 API Kernel Space ApplicaIon Space V4L2 Linux Drivers V4L2 IOCTLs mplayer ffmpeg vlc V4L2 IOCTLs Hardware Hardware
  14. 14. V4L2 Devices and Subdevices Modular Approach to Model the HW •  Hardware is often made of several integrated circuits that need to interact with each other •  This can lead to complex drivers. •  The drivers must reflect the hardware model in software •  Modelling the different hardware components as software blocks called sub-devices. V4L2 device V4L2 sub-device V4L2 sub-device V4L2 sub-device V4L2 sub-device
  15. 15. Media Controller API to Define Relation Between Device/Sub-Devices •  Allows to discover a platform configuration and configure it •  Devices are described as building blocks •  Connected through pads •  Pads can be either sinks or sources •  No limit for pads and sinks on each entity
  16. 16. Device/Sub-device and Media Entity Source & Sink Pads to Represent Data Flow
  17. 17. Media Device - HW Pipeline V4L2 Device/Sub-Devices Structured Using MediaController API Media Device Camera CSI-2 Host Colorspace converter Video Device /dev/video
  18. 18. Device Driver Interaction Only High-Level Interaction is Needed •  The main driver will call the sub-device drivers to •  power on, •  start streaming, •  change video mode, •  and many more. •  All this without needing to know anything about the HW it is talking to Video Device /dev/video mplayer Kernel Space ApplicaIon Space
  19. 19. Device Tree Describes the HW Configuration •  The format is expressive and able to describe most board design aspects including •  the number and type of CPUs •  base addresses and size of RAM •  busses and bridges •  peripheral device connections •  interrupt controllers and IRQ line connections •  and others
  20. 20. Device Tree Hardware Details Listed •  The HW specific details will be available in the Device Tree •  HW addresses, IRQs and more are always available in the Device Tree •  In the case of V4L2 cameras, information like number of lanes, frequency and others is also available
  21. 21. V4L2, Media Controller and Device Tree Putting it All Together 21
  22. 22. V4L2, Media Controller and Device Tree Putting it All Together •  Device driver is designed using V4L2 API – API for video capture •  It is modular and captures the HW elements – Sub- devices •  Media controller API is used to define the relationship between the device and sub-devices, forming a HW pipeline that represents the actual HW platform •  Device tree describes the HW platform and individual block specific details (address, interrupts, version, number of lanes) 22
  23. 23. V4L2, Media Controller and Device Tree Putting it All Together •  When kernel starts, it will look at the device tree and configure it self •  The sub-device drivers will be loaded if present in the device tree and will get the necessary details from it •  The device driver HW pipeline will be formed •  Application interacts with video device using high level commands •  The video device is accessible under /dev •  The application will open /dev/videoX and interact using the high level commands (such as change resolution, shutdown, power on, start capture, etc.) •  Commands will be translated into low-level commands, specific to each individual sub-device (CSI-2 controller, camera, etc.) 23
  24. 24. V4L2, Media Controller and Device Tree Putting it All Together •  If a change in the platform is required (replace camera or controller, etc.) •  The new individual sub-devices need to be added to the kernel •  The device tree needs to be updated and recompiled to reflect the HW changes •  By re-starting the Kernel, the Device driver HW pipeline will be updated automatically 24
  25. 25. OUR EXPERIENCE
  26. 26. CSI-2 SW Development The Media Device HW Pipeline 26
  27. 27. CSI-2 SW Development The Device Tree Camera PHY Video DMA CSI Controller
  28. 28. CSI-2 SW Development Simple to Change CSI-2 Camera Camera
  29. 29. MIPI CSI-2 Host IP Prototyping Kit •  Synopsys offers an integrated HW/SW solution that takes advantage of all these concepts Visit Synopsys booth to see demo!
  30. 30. Leverage Kernel Tools to Accelerate CSI-2 SW Development •  Application interacts with driver only using high level Commands •  No need to use specific commands for each HW element •  The same kernel can serve for several HW platforms by simple changing the Device Tree •  No effort in recompiling the kernel •  Modular code is easily split among team members •  Each driver just needs to provide standard interaction commands •  Easy to maintain and add support for new HW All this accelerates driver development by reducing the code complexity and turnaround time! 30
  31. 31. Thank You! Synopsys’ broad porkolio of DesignWare MIPI IP soluIons synopsys.com/mipi

×