REAL-TIME SYSTEMS II                                                                                                      ...
Basic principles – retry in case of
                    Basic principles - emission                                       ...
Collisions                                                                                                          Parame...
Ethernet Frame                                                                                                            ...
Problems                                                                                                                  ...
Switched Ethernet                                                                                                         ...
Complementaty aspects                                                                                                  Qua...
Traffic adaptation                                                                                                     Use...
EtherCat                                                                                                                  ...
Ethernet IP with time
                  synchronization                                                                   ...
SERCOS III - Topology                                                                                                     ...
Ethernet Powerlink in short                                                                                               ...
Real-Time Ethernet EPA - Ethernet
                  for Plant Automation – in short                                       ...
Real-Time Ethernet EPA - analysis                                                                                         ...
REAL-TIME SYSTEMS II Real-Time Networking Ethernet
REAL-TIME SYSTEMS II Real-Time Networking Ethernet
REAL-TIME SYSTEMS II Real-Time Networking Ethernet
REAL-TIME SYSTEMS II Real-Time Networking Ethernet
Upcoming SlideShare
Loading in …5

REAL-TIME SYSTEMS II Real-Time Networking Ethernet


Published on

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

REAL-TIME SYSTEMS II Real-Time Networking Ethernet

  1. 1. REAL-TIME SYSTEMS II Outline Real-Time Networking The good old Ethernet Myths and realities Ethernet Companion protocols Improvements Switched Ethernet Prof. J.-D. Decotignie How to improve temporal behaviour CSEM Centre Suisse d’Electronique et de Microtechnique SA Industrial Ethernet Jaquet-Droz 1, 2007 Neuchâtel Experiments on Switched Ethernet ©2006, J.-D. Decotignie Conclusion Real-Time Networks – Ethernet 2 IEEE 802 standards « Vintage Ethernet » application One segment of coaxial cable presentation All nodes are session hooked to the Logical Link Control 802.2 transport same cable Bridging 802.1 Network A medium access 802.3 Ethernet 802.4 token bus 802.5 token ring 802.11 Wireless LAN 802.6 DQDB 802.15 Wireless PAN 802.16 Broadband W Data link control Every station may listen physical what is emitted on the bus (cable segment) Each station may listen to what it emits ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie Packet transmission Real-Time Networks – Ethernet 3 Real-Time Networks – Ethernet 4
  2. 2. Basic principles – retry in case of Basic principles - emission collision A station that wants to transmit Not done immediately Listen the transmission medium A number is chosen randomly in the backoff interval If free during a given duration This number is multiplied by the slot duration (5µs à 100Mbit/s) Emits its message Starts to transmit after the given duration (if medium is busy) While listening what is sensed on the cable If there is a collision, the backoff window is doubled If there is a collision, it sends a jamming sequence et prepares to retry Maximum of 0..1023 If the medium is busy, the station waits until it becomes free After 16 attempts, transmission is cancelled and does as indicated above When transmission is successful, the backoff interval returns to 1-persistent CSMA/CD its original value [0..1] ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie Real-Time Networks – Ethernet 5 Real-Time Networks – Ethernet 6 CSMA/CD for Ethernet Backoff Message to transmit In case of collision yes no Channel Transmission is stopped after sending the jamming sequence Build frame busy yes the backoff interval is calculated Differing ON [0..2m-1] where m is the number of successive collisions Differing yes on A number is selected randomly in this interval no Emit jamming start transmission Incrment #attempts Channel free no Waiting time = selected number t . « slot time » yes Slot time = 512 bit time (4096 @ 1Gbit/s) yes #attempts yes Collision detected >16 Wait interframe silence After each collision, the interval is doubled no ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie no Calculate backoff & wait Differing off Maximum interval of [0..1023] after 10 attempts no End of transmission yes END yes Frame In case of success, m is reset to 0 waiting Real-Time Networks – Ethernet 7 Real-Time Networks – Ethernet 8
  3. 3. Collisions Parameters (10base5) Collision zone = 2 x propagation time 50 ohms coaxial cable (thick ethernet) This defines the « slot time » 10 Mbit/s Limits the cable length and defines the minimal size of a Interframe silence = 96 bit time (9.6µs) packet « slot time » = 512 bits (51.2µs) A B A B Distance < 2500 m. between 2 stations 5 segments of 500 m and 4 repeaters Transmission a = τ Tt Minimum frame size = 512 bits time Maximum frame size = 1518 octets ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie τ = 3.33ns / m Propagation time Jamming sequence size = 32 to 48 bits collision Real-Time Networks – Ethernet 9 Real-Time Networks – Ethernet 10 History Conventions 1970: ALOHAnet is developped at Hawai University 1976: Bob Metcalfe and David Boggs make a talk at Speed Transmission Medium the National Computer Conference 1 BASE -2 13 déc. 1977: U.S. Patent #4,063,220 - Multipoint data 10 BROAD -5 communication system with collision detection 100 -36 1982: 1st version published by Xerox, Intel and DEC 1000 -T, -T2 1985: first 802.3 standard 10G -F, -FB, -FL ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie -X Real-Time Networks – Ethernet 11 Real-Time Networks – Ethernet 12
  4. 4. Ethernet Frame Vintage Ethernet Performances preamble SFD Destination Source Type / 802.2 header and FCS ESD Difficult to establish in theory address address length data A maximum throughput of 37% is often mentionned 7 bytes 1 byte 6 bytes 6 bytes 2 bytes 46-1500 bytes 4 bytes 1 byte This is only valid under simplified asumptions and small packets MAC address 24 bits: identification of manufacturer (Organization Unique In practice Indentifier) Access delay are minimal in case of low traffic 24 bits: assigned by the manufacturer or the vendor Transmission delay increases linearly with the size of the packets and the number of nodes ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie The delay standard deviation also increases but more slowly Real-Time Networks – Ethernet 13 Real-Time Networks – Ethernet 14 What makes a difference between Measured performances at 10Mbit/s theory and practice Mean delay Calculations are complicated ⇒ simplified asumptions Its standard deviation Possion arrival law In practice, it is less smooth Infinite population and always waiting packets Source Boggs, 1988 Packets are dropped due to buffer overflow In reality Cable lengths are much smaller than the maximum allowed Packet sizes according to a bimodal distribution ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie Lengths close to min and lengths close to max. Real-Time Networks – Ethernet 15 Real-Time Networks – Ethernet 16
  5. 5. Problems Companion protocols « capture effect » IP: Internet Protocol RFC 791 Assume 2 stations A and B have a lot of messages to send TCP: Transport Control Protocol, RFC 793 At a given time, their emissions collide A chooses 1 as a random number and B elects 0 ARP: Address Resolution Protocol, RFC 826 Thus B succeeds at its first attempt IGMP: Internet Group Management Protocol As B has more traffic, it attempts to send the next message right the way ICMP: Internet Control Message Protocol, This will collide with A second attempt RARP: Reverse Address Resolution Protocol, RFC 903 A has to double the backoff interval while B has the initial one B has hence more chance to succeed than A, ….. And so on DHCP: Dynamic Host Configuration Protocol, RFC Unbounded transmission time 2131-2 ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie Not exact but !!! IPv6, BGP, OSPF, RIP, NAT, ….. Real-Time Networks – Ethernet 17 Real-Time Networks – Ethernet 18 Improvements Parameters 1987: 1BASE5: 1Mbit/s on twisted pairs with hub Distances on twisted pairs < 100m. 1990: 10 Mbit/s hub Distances with optical fibers < 40 km (10G) 1995: 100 Mbit/s Speed Interframe Slot time 1998: 1 Gbit/s hub hub space 10 Gbit/s 10 96 bits 512 bits Node Node Node Node Node Node 1997: full duplex 100 96 bits 512 bits Hub replaced by a bridge (switch) 1000 96 bits 4096 bits ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie 10G 96 bits NA Real-Time Networks – Ethernet 19 Real-Time Networks – Ethernet 20
  6. 6. Switched Ethernet Switched Ethernet (2) The original Ethernet operates in a half duplex mode With a switch, there are no longer collisions One station transmits or another but not both at the same time (except in An end node no longer needs switch case of collision) to detect the collisions With a hub (repeater), there are only 2 nodes on a cable (hub + As in a twisted pair cable, switch switch end node) there a pair for each direction However the behavior must be the same as if all nodes share the same Both directions may be cable (collision domain) exploited simultaneously Node Node Node Node Node Node This is necessary to detect possible collisions Each link may hence be potentialy exploited at twice the speed A switch The available bandwidth is potentialy higher (multiple domains) Receives on a port Re-transmit on the port(s) toward the destination(s) of the packet That does not mean that all problems are solved ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie In between, it stores the packet Real-Time Networks – Ethernet 21 Real-Time Networks – Ethernet 22 Switched Ethernet (3) Hubs and switches Switch buffers may overflow Sometimes misleading If two nodes send traffic to a third one ⇒ PAUSE frame Hub 10/100 is a switch 80 Mbit/s switch Multilayer switches 160 Mbit/s ??? Include a routing module and a switching module 80 Mbit/s What about multiple paths between two nodes switch switch Spanning tree ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie switch switch switch Node Node Node Node Node Node Node Node Node Real-Time Networks – Ethernet 23 Real-Time Networks – Ethernet 24
  7. 7. Complementaty aspects Quality of service management Auto negociation According 802.1D (MAC bridges) every 16ms +/-8 Not only for Ethernet 8 traffic priorities VLAN – IEEE 802.1Q Most switches only implement « best effort », « real-time » and A way to isolate traffic among a group of nodes « management » on each egress Uses the « type/length » field Packets arriving in a ingress port are inserted in the queue corresponding to the priority at the egress port (FIFO order within a priority level) Add a LAN identifier (12 bits) and a priority field (3 bits) When the output link is free, the oldest packet of the highest priority is Quality of service – IEEE 802.1D (includes 802.1p) transmitted This may cause priority inversions which are quite long (1500 bytes) Self learning Packets that have no priority field or those that enter through selected ports Switches may learn on which ports the nodes are reachable may be forced at a given priority level ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie Very useful to filter the traffic coming from outside Multicast raises a problem IGMP snooping !!! This is not to be confused with QoS at network level such as IntServ or DiffServ Real-Time Networks – Ethernet 25 Real-Time Networks – Ethernet 26 How to improve QoS in Ethernet ? Add another MAC Access time A number of such proposals Modify the MAC Nearly all MAC types have been suggested Add another MAC on top of CSMA/CD The MAC used in many fieldbussses has been used Pure TDMA, master/slave, token, reservation, etc. Adapt the traffic Generaly speaking, rather inefficient Use of switches Pure TDMA @ 1Gbit/s ⇒ 4% utilization Synchronisation, dating, consistency Clock synchronization algorithms ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie Real-Time Networks – Ethernet 27 Real-Time Networks – Ethernet 28
  8. 8. Traffic adaptation Use of switches Avoid bursts due to non real-time traffic Avoids collisions smoothing (limit traffic instantaneous intensity by delaying Queues in switches may however overflow part of it) 80 Mbit/s shaping (periodic emissions by blocks) switch 160 Mbit/s ??? This only adds statistical guarantees 80 Mbit/s Reduces drop rate and jitter Increases mean delay Increases mean delays and increases jitter ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie Real-Time Networks – Ethernet 29 Real-Time Networks – Ethernet 30 Clock synchronization Current contenders Using messages to synchronize and provide Non standard hardware Standard hardware, simultaneous sampling is not possible (unless EtherCAT compatible with regular SynqNet 802.3 nodes exception) SERCOS III PROFINET Accuracy γ of synchronization depends on the PowerDNA ? JetSync uncertainty ε in the transmission delay Standard hardware but not EtherNet/IP (+CIP) γ ≥ ε (1 – 1/n) where n is the number of participating nodes compatible with regular Modbus-TCP 802.3 nodes Real-Time Publish-Subscribe Ethernet Powerlink And a number of academic proposals ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie FTT-Ethernet TTP over Ethernet Real-Time Networks – Ethernet 31 Real-Time Networks – Ethernet 32
  9. 9. EtherCat EtherCat – Architecture & protocols General topology with 65535 nodes / segment Wide choice of cable from twisted pair to optical fibers Synchronisation based on distributed synchronized clocks Claim accuracy to microseconds Quite performing 100 axes in 100µs; 1000 points E/S in 30 µs Compliant with IEEE 802.3u Fast Ethernet All protocols based on IP are supported (TCP, UDP,...) Dedicated hardware (except for the master) Integration with CANopen and SERCOS profiles for ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie EtherCat technical introduction compatibility with legacy and overview, dec. 2004 Node to node communication only via master Real-Time Networks – Ethernet 33 Real-Time Networks – Ethernet 34 EtherCat – Principles EtherCat - analysis Master sends an Ethernet frame Advantages That contains one or more EtherCat datagrams Very fast (is this useful ?) Contains data for the outputs and has room for the input data Nodes read and update part of the frame on the fly Standard hardware for the master Location in datagram is independent from the physical localization Compatible with legacy CAN Open or SERCOS Inside each slave, there is a sync manager Drawbacks That informs of the reception of new data Targeted to inputs and outputs which is not sufficent in many That manages the data read and write order cases Ensures that data has been updated Incompatible with Ethernet Requires specialized hardware ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie Restrictive topology Source: EtherCat the Ethernet Ethernet cable fieldbus, Real-Time Networks – Ethernet 35 Real-Time Networks – Ethernet 36
  10. 10. Ethernet IP with time synchronization Ethernet IP No limit in node number Topology according to Ethernet No modification to 802.3 It is possible to use 802.1p priorities Synchronisation according to IEEE 1588 Compliant with IEEE 802.3u Fast Ethernet IP based protocols are supported (TCP, UDP,...) Implementation on standard 802.3 hardware Integration with CIP for compatibility with DeviceNet and ControlNet ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie Node to node communication along a producer/consumer model Source ODVA Real-Time Networks – Ethernet 37 Real-Time Networks – Ethernet 38 Ethernet IP - analysis SERCOS III Advantages Up to 254 nodes on a single segment Standard hardware Line or ring topology Most of the used protocols are standard Medium redundancy capability Can be mixed with « pure » Ethernet nodes So-called « guaranteed and deterministic communication » Compatibility with DeviceNet and ControlNet Cycle time from 31.25µs (62.5, 125, 250 and n x 250µs) Synchronisation based on IEEE 1588 jitter <1µs (or 50µs for the low performance class) Drawbacks Compliant with IEEE 802.3u Fast Ethernet (fiber and copper) Cycle time is not constant, jitter is not controlled IP based protocols are supported (TCP, UDP,...) Only statistical guarantees Implementation on dedicated hardware ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie Using priorities in switches Integration SERCOS profiles Filtering external traffic Possibility of real-time node to node communication Real-Time Networks – Ethernet 39 Real-Time Networks – Ethernet 40
  11. 11. SERCOS III - Topology SERCOS - cycle Ring Real-time channel present in each cycle to Primary channel Master Secondary channel p1 p2 Synchronize the slaves with the master Exchange data and commands between the master and the slaves In each cycle (real-time) processing p1 p2 p1 p2 p1 p2 On request (service channel) processing Slave 1 Slave 2 Slave 3 Exchange data between slaves Line IP channel for on demand transfers (optionnal) Master Base on IP Primary channel p1 p2 Secondary channel Permet to exchange data between master and slaves Operator display data, files, configurations, … Exchange between slaves ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie processing p1 p2 p1 p2 p1 p2 processing processing Slave 1 Slave 2 Slave 3 Transparent communication with standard Ethernet nodes (PC, …) Real-Time Networks – Ethernet 41 Real-Time Networks – Ethernet 42 SERCOS III – cycle SERCOS III - analysis Up to 4 Master Data Telegrams Advantages Efficient / possibility of line topology Up to 4 AT Device Telegrams hotplug A number of IP messages (max. duration) Redundancy capability A node may be hooked to a regular Ethernet network (one port must be MDT AT MDT left open) H M H M H M D S D S D S Guranteed cycle time and reduced jitter R T R T R T Uses regular IP frames (no encapsulation) RT channel IP channel cycle Drawbacks MDT AT MDT Protected by patents H M H M H M Requires non standard hardware ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie D S D S D S R T R T R T Real-time behavior impossible in presence of non compliant nodes RT channel IP channel RT channel cycle Requires a bridge to regular Ethernet Real-Time Networks – Ethernet 43 Real-Time Networks – Ethernet 44
  12. 12. Ethernet Powerlink in short Ethernet Powerlink – Model Up to 240 nodes on a segment Topology according to Ethernet + line topology possible N Application layer EPL HTTP / FTP e So-called « guaranteed and deterministic communication » t w PDO Command SDO Command application Cycle time from 200µs o r Sequence SDO jitter <1µs for precise node synchronisation k transport IAONA real-time class 4 (highest performance one) M UDP TCP a Compliant with IEEE 802.3u Fast Ethernet n a IP network IP based protocols are supported (TCP, UDP,...) g e Data Link EPL Implementation on regular 802.3 hardware m data link e Integration with CANopen EN50325-4 for interoperability n 802.3 MAC ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie t Node to node communication along a producer/consumer mode 802.3 physical physical Real-Time Networks – Ethernet 45 Real-Time Networks – Ethernet 46 Ethernet Powerlink – cycle Ethernet Powerlink - Analysis Synchronous part made of slots Advantages Continuous: CN is polled at each cycle Constant cycle duration (in absence of error) Multiplexed: a given CN is polled every n cycles Line topology possible Synchronous exchanges Presence of an application layer Sequence of PollRequest followed by PollResponse Response is sent in broadcast (available to all) Drawbacks Producer -> consumer Only contains a limited subset of the necessary concepts (cf. WorldFIP) Idle part (to keep a constant duration to the cycle) Low efficiency (25µs lost in each transaction) Cycle time Cannot coexist with regular Ethernet nodes Slot Requires specialy designed routers and hubs Start_of Poll Poll Poll StartOf ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie MN _cycle Request Request Respons Async. Model does not take errors into account e Poll Poll Async Quite sensitive to timing errors (collisions !) CN Response Response Send Isochronous window Asynchronous window Idle Real-Time Networks – Ethernet 47 Real-Time Networks – Ethernet 48
  13. 13. Real-Time Ethernet EPA - Ethernet for Plant Automation – in short EPA – Topology Distant operation station Number of nodes only limited by physical considerations Operator Engineering EPA agent Topology according to Ethernet (switch and hub) station station So-called « guaranteed and deterministic communication » Level L2 Cycle time in multiples of milliseconds jitter depends on the selected IEEE 1588 class EPA EPA EPA bridge bridge bridge Compliant with IEEE 802.3 (all variants) IP based protocols are used (TCP, UDP,...) EPA field EPA field EPA field device device device Implementation on regular 802.3 hardware Level L1 Based upon IEEE 1588 for clock synchronisation EPA field EPA field EPA field device device device Device capabilities described in XML ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie EPA field EPA field EPA field Conventional client-server communication model device device device Process variables communicated according to IEC 61499 and IEC 61804 (function blocks) EPA microsegment Real-Time Networks – Ethernet 49 Real-Time Networks – Ethernet 50 Real-Time Ethernet EPA - architecture EPA - scheduling macrocycle EPA function blocks AP AP non temps réel Periodic transfer phase Tp Non periodic transfer phase Tn D1 EPA EPA SM Entity EPA Application HTTP / FTP / MIB access Entity SNMP / SNTP application D2 EPA Socket Mapping Entity D3 transport UDP TCP D4 IP D5 network EPA Communication Scheduling Management Entity (ECSME) D6 data link 802.3 MAC ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie 802.3 physique physical Non periodic data announciation Periodic packet Aperiodic packet EndNonperiodic data sending announciation Real-Time Networks – Ethernet 51 Real-Time Networks – Ethernet 52
  14. 14. Real-Time Ethernet EPA - analysis Others Advantages Profinet Support functional blocks V3 will introduce real-time guarantees suitable application layer P-NET on IP PNET is a virtual token MAC Drawbacks Same on IP without any guarantee Only client-server Real-Time publish-subscribe Single polling cycle Just a publish subscribe layer on top of TCP without any guarantee Difficult to calculate guarantees (even in absence of error) MODBUS on TCP/IP Some parts of the standard are quite obscure Nothing real-time, just the protocol on top of TCP Configuration must be centralized Vnet/IP ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie To ensure that parameters are compatible and consistent Based on temporal windows Proposed by Yokogawa ( Time Critical Control Network Real-Time Networks – Ethernet 53 Real-Time Networks – Ethernet 54 Switched Ethernet Ingress Egress Switched Ethernet (2) Ingress port Egress port port port Egress Temporal definitions Let us look at one output port port Switch Egress port Service curve β(t) = C (t-tmux)+ Ingress fabric tmux : multiplexing delay Egress port Egress port Ingress port Switch port tqueue: queueing delay fabric + means that the function is equal to 0 Ingress Ingress for t-tmux < 0 ttrans : packet transmission port port C is the maximum transmission rate Ingress Egress port port delay = tframe + tmux + tqueue + tos on the output port tos : operating system delay Ingress Egress Arrival curve port port tframe: frame tx delay Sum of arrival curves of all incoming traffic whose destination is that 4 port switch output port 121µs for 1514 bytes @ 100Mbit/s One station on each ingress port : α(t) = min [Ct + M, rkt + bk] How to calculate the delay ? M maximum frame size ©2006, J.-D. Decotignie ©2006, J.-D. Decotignie Use of the network calculus rk long term average rate (Σ rk ≤ C should hold for output) bk burstiness of the traffic (M< bk and rk ≤ C) Real-Time Networks – Ethernet 55 Real-Time Networks – Ethernet 56