GMSL in Linux

GMSL in Linux
Eight Cameras, Two Busses, One Address
Automotive Linux Summit, 2018-03-21
Kieran Bingham
kieran.bingham@ideasonboard.com

about:me
Kieran Bingham
● Embedded Linux since 2006
● Open source focus since 2014
● Ideas on Board
● Linux-media / V4L2
● Renesas

about:us
R-Car Gen3
Upstream Multimedia Team
● Jacopo Mondi
● Kieran Bingham
● Laurent Pinchart
● Niklas Söderlund
● Ulrich Hecht
● Wolfram Sang

GMSL in Linux
GMSL
R-Car ADAS platform
Trials and Tribulations
Future Work
Summary
Question Time

Serialiser De-serialiser
I2C Control channel
Video
GMSL

GMSL
● High speed data link
– Serialiser
● Convert parallel data to a high data rate serial link
– Deserialiser
● Convert high data rate serial link to parallel data
– Forward Channel
● High bandwidth data
– Reverse Channel
● Control data
● (Low bandwidth) Video
● EMI Resistant
Gigabit Multimedia Serial Link

https://www.maximintegrated.com/en/products/interface/high-speed-signaling/MAX9271.html
GMSL

Serialisers – Deserialisers
● Serialiser converts parallel data to a
higher rate serial stream
– Pixel Clock:
– 8b/10b encoding
– 30bit / 40bit packets
● Deserialiser converts data back to a
parallel stream
● AC Coupled
– Uses 8b/10b to ensure ‘average’ bit use
– No more than 5 consecutive bits
GMSL

Serialisers – Deserialisers
● EMI reduction
– Pre-emphasis, de-emphasis
– Spread spectrum
● Fault detection
– Open cable, Short to Battery, Short to Ground
– Status registers on both sides of the link
– Possible to transmit to two deserialisers
● Power over Coax
GMSL

Data Transmission
● Packets are highly configurable
● DBL data mode
● Forward channel
– Parallel Video, Pixel Clock, I2S, GPIO, I2C, UART
● Reverse Channel
– GPIO, I2C, (reduced bandwidth) video
● Host can be either side
GMSL

GMSL
● 3 Gbps forward channel
● UART 1 mbps I2C 400 kbps
● STP or Coax, with PoC
● GMSL v2
– 6 Gbps forward channel (4K)
– 1.5 Gbps reverse
– Lossless compression
– HDCP
Data Transmission

GMSL Competitors / Alternatives
● GMSL is proprietary from Maxim
● Alternatives
– Automotive Ethernet
● In use for more than just multimedia
– MOST
● Token ring based system
– FPD-Link III
● Bridging of Control Interfaces Over Multimedia Serial Links
Vladimir Zapolskiy, Mentor Graphics
Friday, June 22 • 16:20 - 17:00
GMSL

Use cases
● ADAS
– Reversing cameras
– Surround view
– Lane departure detection
– Road sign detection
– Side mirror replacements
– Blind spots
– Recording Accidents
– Driver and passenger monitoring
– Driverless cars … ?
R-Car ADAS platform

Rcar VIN Gen 3 / CSI2
● Gen3 brings in CSI2 support
● Media controller support
● Supports up to 4K
● 8 Inputs
● Supported in Mainline v4.18
● Supports RGB and YUV video
● External synchronization and BT.656 for the YUV
● Gen3 platforms implement a CSI-2 receiver
R-Car ADAS platform
Documentation/devicetree/bindings/media/rcar_vin.txt
Documentation/devicetree/bindings/media/renesas,rcar-csi2.txt

MAX9286 – Quad GMSL Deserialiser
● 4 channels
● I2C Mux, with ‘address translation and broadcast’
● 1.5Gbps per channel
● Low-Cost 50Ω Coax Cable and FAKRA Connectors or 100Ω STP
● Data from Image Sensors Are Synchronized to the Same Pixel
● Automatic Internal/External Generation of Camera Sync
● Equalization Allows 15m Length Cable Operation at Full Speed
● Two Dedicated GPIO Ports
R-Car ADAS platform

MAX9286 – Quad GMSL Deserialiser
R-Car ADAS platform
Camera max9271
Camera max9271
Camera max9271
Camera max9271
Decode
and
Fifo
max9286
Line
Memory
and
Sync
GMSL to
parallel
convertor
GMSL to
parallel
convertor
GMSL to
parallel
convertor
GMSL to
parallel
convertor
I2C
MCU
Parallel
to
CSI2

RDACM20 / RDACM21
● GMSL Camera using FAKRA Coax connector
● Very small size (20.5mm x 20.5mm x 21.5mm)
● Ultra lightweight (< 10 g excluding lens)
● 10-bit raw, 8-/10-bit YUV422
● 1280 x 800 pixels
● OV10635 with DSL267
● Horizontal field of view: 194.6°
● Vertical field of view: 129.92°
● Lens aperture: F# 2.2
max9271 ov10635
mcu
R-Car ADAS platform

● [RFC PATCH v1 0/4] GMSL Drivers
– https://lwn.net/ml/linux-media/20180605233435.18102-1-kieran.bingham+renesas@ideasonboard.com/
– ` [RFC PATCH v1 1/4] media: dt-bindings: max9286: add device tree binding Kieran Bingham
– ` [RFC PATCH v1 2/4] media: i2c: Add MAX9286 driver Kieran Bingham
– ` [RFC PATCH v1 3/4] media: dt-bindings: rdacm20: add device tree binding Kieran Bingham
– ` [RFC PATCH v1 4/4] media: i2c: Add RDACM20 driver Kieran Bingham
● [RFC 0/2] GMSL bindings description update
– https://lwn.net/ml/linux-media/1528543805-23945-1-git-send-email-jacopo%2Brenesas%40jmondi.org/
` [RFC 1/2] dt/bindings: media: Add DT bindings for Maxim Integrated MAX9286 Jacopo Mondi
` [RFC 2/2] dt-bindings: media: i2c: Add bindings for IMI RDACM20 Jacopo Mondi
Upstreaming

MAX9286 - Bindings
The MAX9286 deserializes receives video data on up to 4 Gigabit
Multimedia Serial Links (GMSL) and outputs them on a CSI-2 port
using up to 4 data lanes.
In addition to video data, the GMSL links carry a bidirectional
control channel that encapsulates I2C messages. The MAX9286
forwards all I2C traffic not addressed to itself to the other side of
the links, where a GMSL serializer will output it on a local I2C bus.
In the other direction all I2C traffic received over GMSL by the
MAX9286 is output on the local I2C bus.
R-Car ADAS platform

Required Properties:
- compatible: Shall be "maxim,max9286"
- reg: I2C device address
Optional Properties:
- poc-supply: Regulator providing Power over Coax to the
cameras
- pwdn-gpios: GPIO connected to the #PWDN pin
Required endpoint nodes:
----------------------------------
The connections to the MAX9286 GMSL and its endpoint
nodes are modeled using the OF graph bindings in
accordance with the video interface bindings defined in
Documentation/devicetree/bindings/media/video-
interfaces.txt.
The following table lists the port number corresponding to
each device port.
Port Description
----------------------------------------
Port 0 GMSL Input 0
Port 1 GMSL Input 1
Port 2 GMSL Input 2
Port 3 GMSL Input 3
Port 4 CSI-2 Output
MAX9286 - Bindings
Optional Endpoint Properties for GSML Input Ports (Port [0-3]):
- remote-endpoint: phandle to the remote GMSL source
endpoint subnode in the remote node port.
Required Endpoint Properties for CSI-2 Output Port (Port 4):
- data-lanes: array of physical CSI-2 data lane indexes.
- clock-lanes: index of CSI-2 clock lane.
Required i2c-mux nodes:
----------------------------------
Each remote GMSL endpoint is modeled as child busses of an
i2c bus multiplexer/switch, in accordance with bindings
described in Documentation/devicetree/bindings/i2c/i2c-mux.txt.
Required i2c child bus properties:
- all properties described as required i2c child bus nodes
properties in Documentation/devicetree/bindings/i2c/i2c-mux.txt.

MAX9286 - Bindings
gmsl-deserializer@2c {
compatible = "maxim,max9286";
reg = <0x2c>;
poc-supply = <&camera_poc_12v>;
pwdn-gpios = <&gpio 13 GPIO_ACTIVE_LOW>;
#address-cells = <1>;
#size-cells = <0>;
ports {
#size-cells = <0>;
port@0 {
reg = <0>;
max9286_in0: endpoint {
remote-endpoint = <&rdacm20_out0>;
};
};
port@1 { … };
port@2 { … }
port@3 { … };
port@4 {
reg = <4>;
max9286_out: endpoint {
clock-lanes = <0>;
data-lanes = <1 2 3 4>;
remote-endpoint = <&csi40_in>;
};
};
};
… <continued> ...
i2c@0 {
#size-cells = <0>;
reg = <0>;
camera@51 {
compatible = "imi,rdacm20";
reg = <0x51 0x61>;
port {
rdacm20_out0: endpoint {
remote-endpoint = <&max9286_in0>;
};
};
};
};
i2c@1 {
#size-cells = <0>;
reg = <0>;
camera@52 {
compatible = "imi,rdacm20";
reg = <0x52 0x62>;
port {
rdacm20_out1: endpoint {
remote-endpoint = <&max9286_in1>;
};
};
};
};
i2c@2 { ... };
i2c@3 { ... };
};

MCU
8 Cameras, Two Busses, One Address
GMSL – Max9286 I2C Mux
Host
I2C

8 Cameras, Two Busses, One Address
Host
I2C
max9286
max9286
max9271
max9271
max9271
max9271
max9271
max9271
max9271
max9271
mcuov10635
mcuov10635
mcuov10635
mcuov10635
mcuov10635
mcuov10635
mcuov10635
mcuov10635

Difficulties faced
● Subdevice Notifiers
● Drivers merged into single module or perform cross writes to the
other
● I2C muxing – making sure that one MUX is disabled, while the
other is active.
● MCU camera programming
● Fsync … took forever
● Faulty cameras (non-probing – MCU absentia)
● VC / Async framework / Endpoint Matching (ADV748x prototype)
● Rapidly developing VIN/CSI2 on RCar3

● RDACM21
● Fault Tolerant V4L2
– Allowing the system to function when one camera is broken
– Recovering cameras / ‘reboot’ing them (Re-negotiate, on error?)
– Redundant links…
– Hotplug style events
● Automatic I2C addresses
● FWNode Endpoint Matching improvements
● GMSL as a bus
● Bi-directional
GMSL in Linux : Future Work

GMSL in Linux
GMSL
R-Car ADAS platform
Future Work
Summary
Question Time?

GMSL in Linux
GMSL
R-Car ADAS platform
Future Work
Summary
Question Time!

GMSL in Linux

More Related Content

What's hot

Similar to GMSL in Linux

Recently uploaded

GMSL in Linux