Dynamic Classification in a Silicon-Based Forwarding EngineTal Lavian Ph.D.
Implement flow performance enhancement mechanisms without introducing software into data forwarding path
Service defined packet processing in a silicon-based forwarding engine
Policy-based Dynamic packet classifier
Create OPEN platform for introduction of new services
Specify OPEN interfaces for Java applications to control a generic, platform-neutral forwarding plane
Enable downloading of services to network node
Allow object sharing and inter-service communication
Enabling Active Flow Manipulation In Silicon-based Network Forwarding EnginesTal Lavian Ph.D.
A significant challenge in today’s Internet is the ability to efficiently incorporate customizable network intelligence in commercial high performance network devices.
Framework for introducing services
API for programming network devices
Open Networking Better Networking Through ProgrammabilityTal Lavian Ph.D.
1st Degree of Emancipation:
Introverted APIs Emerge.
Modular code is native, local, and trusted. port required.
2nd Degree of Emancipation:
Extroverted APIs expose object capabilities to ISV code.
3th Degree of Emancipation:
Extroverted APIs extend a commodity Java runtime.
4th Degree of Emancipation:
ISV code is local/non-local, non-native, non-trusted, loaded on demand, and can teleport itself.
Dynamic Classification in a Silicon-Based Forwarding EngineTal Lavian Ph.D.
Implement flow performance enhancement mechanisms without introducing software into data forwarding path
Service defined packet processing in a silicon-based forwarding engine
Policy-based Dynamic packet classifier
Create OPEN platform for introduction of new services
Specify OPEN interfaces for Java applications to control a generic, platform-neutral forwarding plane
Enable downloading of services to network node
Allow object sharing and inter-service communication
Enabling Active Flow Manipulation In Silicon-based Network Forwarding EnginesTal Lavian Ph.D.
A significant challenge in today’s Internet is the ability to efficiently incorporate customizable network intelligence in commercial high performance network devices.
Framework for introducing services
API for programming network devices
Open Networking Better Networking Through ProgrammabilityTal Lavian Ph.D.
1st Degree of Emancipation:
Introverted APIs Emerge.
Modular code is native, local, and trusted. port required.
2nd Degree of Emancipation:
Extroverted APIs expose object capabilities to ISV code.
3th Degree of Emancipation:
Extroverted APIs extend a commodity Java runtime.
4th Degree of Emancipation:
ISV code is local/non-local, non-native, non-trusted, loaded on demand, and can teleport itself.
Data Quality Analysis of Different Receivers Based on Static Base StationIJRESJOURNAL
ABSTRACT: With the global navigation and positioning system technology in various industries more and more widely used. As a result, higher requirements are made for the performance of GNSS receiver and the quality of receiving data. The quality of the received data is analyzed and compared to the receivers produced by different manufacturers. By comparison, The processing ability and data quality of different receivers for GNSS signals are analyzed. Data is collected for two different receiver from production M300Pro of ComNav and SPS 855 of Trimble. The quality analysis and discussion were carried out by using TEQC and CRU software respectively. The results show that the data quality of the two receivers conforms to the detection experience value, and is analyzed and compared by data quality. The two quality analysis software has Respective advantages. Combining different observation factors analysis that is important for the stable operation of the system and improving the positioning accuracy.
Inter-controller Traffic in ONOS Clusters for SDN Networks Paolo Giaccone
In distributed SDN architectures, the network is controlled by a cluster of multiple controllers. This distributed ap- proach permits to meet the scalability and reliability requirements of large operational networks. Despite that, a logical centralized view of the network state should be guaranteed, enabling the simple development of network applications. Achieving a consis- tent network state requires a consensus protocol, which generates control traffic among the controllers whose timely delivery is crucial for network performance.
We focus on the state-of-art ONOS controller, designed to scale to large networks, based on a cluster of self-coordinating controllers, and concentrate on the inter-controller control traffic. Based on real traffic measurements, we develop a model to quan- tify the traffic exchanged among the controllers, which depends on the topology of the controlled network. This model is useful to design and dimension the control network interconnecting the controllers.
본 발표에서는 OCP 하드웨어 및 소프트웨어에 대한 소개를 진행할 예정이다. 특히 페이스북에서 제공한 Wedge ToR 스위치, Open Network Linux, FBOSS, Indigo OpenFlow agent 를 갖고 삽질한 지난 두달 간의 경험을 공유할 예정이다.
Analytical Modeling of End-to-End Delay in OpenFlow Based NetworksAzeem Iqbal
OpenFlow enabled networks split and separate the data and control planes of traditional networks. This design commodifies network switches and enables centralized control of the network. Control decisions are made by an OpenFlow controller, and locally cached by switches, as directed by controllers. Since controllers are not necessarily co-located with switches that can significantly impact the forwarding delay incurred by packets in switches. Only very few studies have been conducted to evaluate the performance of OpenFlow in terms of end-to-end delay. In this work we develop a stochastic model for the end to end delay in OpenFlow switches based on measurements made in Internetscale experiments performed on three different platforms, i.e. Mininet, the GENI testbed and the OF@TEIN testbed.
Enabling Active Flow Manipulation In Silicon-based Network Forwarding EnginesTal Lavian Ph.D.
Lack of industrial-strength Active Network devices that dispel major concerns:
AN requires substantial supports from a NOS
AN introduces substantial software component, hence delay on the data path
AN lacks adequate measures to addressing integrity and security of network devices.
Many thanks to Nick McKeown (Stanford), Jennifer Rexford (Princeton), Scott Shenker (Berkeley), Nick Feamster (Princeton), Li Erran Li (Columbia), Yashar Ganjali (Toronto)
Data Quality Analysis of Different Receivers Based on Static Base StationIJRESJOURNAL
ABSTRACT: With the global navigation and positioning system technology in various industries more and more widely used. As a result, higher requirements are made for the performance of GNSS receiver and the quality of receiving data. The quality of the received data is analyzed and compared to the receivers produced by different manufacturers. By comparison, The processing ability and data quality of different receivers for GNSS signals are analyzed. Data is collected for two different receiver from production M300Pro of ComNav and SPS 855 of Trimble. The quality analysis and discussion were carried out by using TEQC and CRU software respectively. The results show that the data quality of the two receivers conforms to the detection experience value, and is analyzed and compared by data quality. The two quality analysis software has Respective advantages. Combining different observation factors analysis that is important for the stable operation of the system and improving the positioning accuracy.
Inter-controller Traffic in ONOS Clusters for SDN Networks Paolo Giaccone
In distributed SDN architectures, the network is controlled by a cluster of multiple controllers. This distributed ap- proach permits to meet the scalability and reliability requirements of large operational networks. Despite that, a logical centralized view of the network state should be guaranteed, enabling the simple development of network applications. Achieving a consis- tent network state requires a consensus protocol, which generates control traffic among the controllers whose timely delivery is crucial for network performance.
We focus on the state-of-art ONOS controller, designed to scale to large networks, based on a cluster of self-coordinating controllers, and concentrate on the inter-controller control traffic. Based on real traffic measurements, we develop a model to quan- tify the traffic exchanged among the controllers, which depends on the topology of the controlled network. This model is useful to design and dimension the control network interconnecting the controllers.
본 발표에서는 OCP 하드웨어 및 소프트웨어에 대한 소개를 진행할 예정이다. 특히 페이스북에서 제공한 Wedge ToR 스위치, Open Network Linux, FBOSS, Indigo OpenFlow agent 를 갖고 삽질한 지난 두달 간의 경험을 공유할 예정이다.
Analytical Modeling of End-to-End Delay in OpenFlow Based NetworksAzeem Iqbal
OpenFlow enabled networks split and separate the data and control planes of traditional networks. This design commodifies network switches and enables centralized control of the network. Control decisions are made by an OpenFlow controller, and locally cached by switches, as directed by controllers. Since controllers are not necessarily co-located with switches that can significantly impact the forwarding delay incurred by packets in switches. Only very few studies have been conducted to evaluate the performance of OpenFlow in terms of end-to-end delay. In this work we develop a stochastic model for the end to end delay in OpenFlow switches based on measurements made in Internetscale experiments performed on three different platforms, i.e. Mininet, the GENI testbed and the OF@TEIN testbed.
Enabling Active Flow Manipulation In Silicon-based Network Forwarding EnginesTal Lavian Ph.D.
Lack of industrial-strength Active Network devices that dispel major concerns:
AN requires substantial supports from a NOS
AN introduces substantial software component, hence delay on the data path
AN lacks adequate measures to addressing integrity and security of network devices.
Many thanks to Nick McKeown (Stanford), Jennifer Rexford (Princeton), Scott Shenker (Berkeley), Nick Feamster (Princeton), Li Erran Li (Columbia), Yashar Ganjali (Toronto)
Workshop on TelegraphCQ:
Concept of Data Stram Management System.
TelegraphCQ: the DSMS developped at Berkley, internal architecture.
Differences between tradition database.
Adaptive QUery Processing using the new concept of Eddies like a routing operator.
Troubles about join Streams (with no statistical data) and Relations; and the two solution: STAIR and SteMs.
STAIR: a join operator that allow internal state changing using primite function visible to eddies.
SteMs: helf-join operator that keep homogeneous tuples, internal state is decision-indipendent.
Eddies Routing Policy implemented with the (Waldspurger & Weihl [1994]) Lottery Scheduling.
Soft Real-Time Guarantee for Control Applications Using Both Measurement and ...CSCJournals
In this paper, we present a probabilistic admission control algorithm over switched Ethernet to support soft real-time control applications with heterogeneous periodic flows. Our approach is purely end host based, and it enables real-time application-to-application QoS management over switched Ethernet without sophisticated packet scheduling or resource reservation mechanisms in Ethernet switches or middleware on end hosts. In designing the probabilistic admission control algorithm, we employ both measurement and analytical techniques. In particular, we provide a new and efficient method to identify and estimate the queueing delay probability inside Ethernet switches for heterogeneous periodic flows with variable message size and period. We implement the probabilistic admission control algorithm on the Windows operating system, and validated its efficacy through extensive experiments.
A NETWORK-BASED DAC OPTIMIZATION PROTOTYPE SOFTWARE 2 (1).pdfSaiReddy794166
The International Journal of Engineering and Science and Research is online journal in English published. The aim is to publish peer review and research articles without delay in the developing in engineering and science Research.The International Journal of Engineering and Science and Research is online journal in English published. The aim is to publish peer review and research articles without delay in the developing in engineering and science Research.
Enabling Active Flow Manipulation In Silicon-based Network Forwarding EnginesTal Lavian Ph.D.
Great Active Nets Community Solutions:
Active networks (AN) approach opens an exciting opportunity for individual applications to define the service provided by the network through programmability.
Active Networks technologies expose a novel approach that allows customer value-added services to be introduced to the network “on-the-fly”.
Active Nets program has produced a new network platform flexible and extensible at runtime to accommodate the rapid evolution and deployment of network technologies.
The exciting opportunity exists for network service providers and third parties, not just the network device providers, to program the network infrastructure and services.
OSMC 2012 | Monitoring at CERN by Christophe HaenNETWAYS
Das CERN, die Europäische Organisation für Kernforschung, ist das weltweit größte Forschungszentrum für Teilchenphysik. Es werden dort Experimente in der Hochenergiephysik mit Hilfe des Teilchenbeschleunigers durchgeführt sowie anderer bereitgestellter Infrastrukturen. Die Untersuchungen rund um den Large Hadron Collider (LHC), erfordern umfangreiche IT-Infrastrukturen, um die Daten, die durch die Kollisionen generiert werden, zu verarbeiten. Sogar die Überwachung des LHC selbst hängt von einer komplexen Infrastruktur ab. Die CERN-IT stellt den Mitarbeitern und den Usern viele unterschiedliche Services bereit und ist vor allem aber der Hauptakteur des LHC GRID. Das GRID ist das weltweit verteilte Rechen- und Speicher-Netzwerk, das die nötige Kapazität zur Verfügung stellt um die Menge an Daten, die anhand des Teilchenbeschleunigers gesammelt wird, analysieren zu können. Es besteht aus 200.000 Cores verteilt auf 34 Ländern. All diese großen Rechenzentren erfordern ein sorgfältiges Monitoring, aber jedes für sich hat Besonderheiten, was dazu führt, dass unterschiedliche Monitoring Strategien und Tools angewandt werden müssen. Die unzähligen Herangehensweisen an diese Herausforderung werden in diesem Vortrag aufgezeigt sowie ein Ausblick auf geplante künftige Entwicklungen.
CapellaDays2022 | ThermoFisher - ESI TNO | A method for quantitative evaluati...Obeo
Development of high-tech systems is a complex task done by diverse specialists distributed across the globe. Reference architectures including a clear functional breakdowns can support them and support their decisions. This presentation proposes an approach to improve the development of advanced electron microscopes by using Capella as an authoritative source of information. To support design decisions, a Capella AddOn has been developed to obtain quantitative information, such as throughput numbers, for a particular workflow. First, we will illustrate how functional and system decompositions can be captured and serve as company-wide architecting assets to inform design decisions. Next, we will outline how simulating Capella models can bring valuable insights to modelers. During a demo, we’ll simulate Capella’s Functional chains using the open-source simulation tool POOSL (https://github.com/eclipse/poosl) , and visualize results using the freely available TRACE4CPS tool (https://www.eclipse.org/trace4cps/). Re-using functions from the reference architecture allows us reason about design aspects such as the relation between throughput and design choices about function allocation and parallelism.
***
The open-source code of the solution is available at https://github.com/TNO/capella-workflow-dse
1. The Run Control and
Monitoring System of
the CMS Experiment
Presented by Andrea Petrucci
INFN, Laboratori Nazionali di Legnaro, Italy
On behalf of the DAQ Group of CMS collaboration
ACAT 2007, 23-27 April 2007, Amsterdam, Netherlands
2. 2
Outline
2ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
Run Control and Monitor System :
Architecture
• Logical Layer
• Services
• Components
• Technologies
At the Magnet Test and Cosmic Challenge (MTCC)
• Control structure
• Operation
• Components
• Results
GRICC Project
3. 3
What is CMS?
3ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
The Compact Muon Solenoid (CMS) experiment is one of two
large general-purpose particle physics detectors being built on the
proton-proton Large Hadron Collider (LHC) at CERN in Switzerland.
•to explore physics at the TeV
scale
•to discover the Higgs boson
•to look for evidence of
physics beyond the standard
model
•to be able to study aspects of
heavy ion collisions
The main goals of the experiment are:
4. 4
Run Control and Monitor System
4ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
• The Run Control and Monitor System (RCMS) is responsible for controlling
and monitoring the CMS experiment during the data taking.
• RCMS views the experiment as a set of partition, where a partition is a
grouping of entities that can be operated independently.
• Main operations are configuration, monitoring, error handling, logging and
synchronization with other subsystems.
5. 55ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
Baseline DAQ
Configuration
• 512 inputs
• 2024 outputs
CMS Data Acquisition
Control and Monitor requirements
• O(104
) distributed Objects to
– control
– configure
– monitor
• On-line diagnostics
• Interactive system
6. 6
RCMS is integrated in the CMS On-line system :
• It controls the “DAQ component”
– Data transport
– Event processing
• It monitors the “Detector Control System” DCS
– manages the slow controls of the whole experiment.
6ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
Run Control and Monitor System
The SOAP protocol and the Web
Services have been adopted as the
main means for communication .
The online process environment is XDAQ,
a C++ framework for a distributed Data
Acquisition System.
7. 7
RCMS Logical Structure
7ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
• A Session is the allocation of the
hardware and software of a CMS
partition needed to perform data-
taking.
• Multiple Sessions may coexist and
operate concurrently.
• Each Session is associated with a Top
Function Manager, that coordinates all
the actions.
TopTop
Sub-DetectorSub-Detector
Sub-DetectorSub-Detector
Sub-DetectorSub-Detector
Sub-DetectorSub-Detector
Sub-DetectorSub-Detector
Sub-DetectorSub-Detector
ServicesServices
ServicesServices
ServicesServices
Sub-System
(DAQ) Resources
8. 8
– SECURITY SERVICE
• login and user account
management;
– RESOURCE SERVICE (RS)
• information about DAQ
resources and partitions;
– INFORMATION AND MONITOR
SERVICE (IMS)
• Collects messages and monitor
data; distributes them to the
subscribers;
– JOB CONTROL
• Starts, monitors and stops the
software elements of RCMS,
including the DAQ components;
RCMS Services
8ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
9. 9
Function Manager
9ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
Input Handler : It handles all
the input events of the FM
(GUIs or other FMs, errors,
states, logs and monitor
messages)
Event Processor: It handles
all the incoming message and
decide where to send them. It
has processing capability
Finite State Machine
(FSM): The behavior of the
FM is driven by a FSM.
Resource Proxy: It handles
all the outgoing connections
with the resources.
The purpose of a Function Manager (FM) is to control a set of resources.
Input
Handler
Event
Processor
FSM
Engine
Resource
Proxy
Function Manager
Resources
State Flow
Error Flow
Monitor Flow
Control Flow
Customizable
10. 10
Resource Service
10ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
RS Manager tool
RS DB
RS API
DAQ Configurator
The Resource Service (RS) stores the process configuration
of the On-line System.
features
Flexible data store
Java API
Configuration documents can be
built on the fly from relational
schema
Versioning system
Oracle and MySQL
Implementation
11. 11
Log Collector
11ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
• Collects log information from log4j
compliant applications (i.e. on-line
process).
…
Publish
Subscriber
System
Display System
Storage System
Log
Collector
Relational DB
Oracle,MySQL
Message System
Access via
JDBC
Access via
TCP
RCMS
applications
and XDAQ
applications
• Send log information directly to a Display
System (Chainsaw) .
• Stores log information in a database and
visualizes them (LogDBViewer) .
• Distributes/publishes log information
through a message system (Java Message
Service).
12. 12
RCMS main components
12ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
Log Collector
WebAppChainsaw
Log Viewer
JobControl
Process
Control
Firefox
JSP/Ajax GUI
Config data
Config data
Conditions data
Process Config
Log Messages
Commands
Notifications
RCMS
RS Manager
Manager/Edit
or
DAQ Configurator
Configuration
Function
Manager
Framework
RS API
RunInfo
API
Hwcfg API
RS DB
GCK DB
RunInfo DB
Log DB
Hwcfg DB
User Interface
13. 13
RCMS Technologies
13ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
Technologies and tools:
• Web Applications,Java Servlets (Apache Tomcat)
• WebService (Axis, WSDL, SOAP)
• Web Tecnologies (Ajax,JSP)
• Databases
– Oracle
– MySQL
Architecture Implementation
Resource Service (RS) Resource Service
Information and Monitor Service (IMS) LogCollector
SubSystem Controllers (FMs) RCMS Framework
Top Function Manager RCMS Framework
GUIs Default JSP GUI - RCMS Framework
JobControl XDAQ Framework
14. 14
Magnet Test and Cosmic Challenge
14ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
The main goals of the Cosmic Challenge were:
•Test Muon alignment systems.
•Commission the several sub-detectors (Drift Tubes - DT, Hadron Calorimeter –
HCAL, Tracker, etc.) and Cosmic Trigger.
•demonstrate cosmic ray reconstruction with multiple sub-detectors.
The Magnet Test and Cosmic Challenge (MTCC) is a milestone of the CMS experiment,
it completes the commissioning of the magnet system (coil & yoke) before its lowering
into the cavern.
Scale MTCC versus
CMS
Data Sources:20 out of 600
3%
Filter Nodes: 14 out of 2000
0.3%
Trigger rate: 100 Hz out of 100 kHz
0.1%
Scale MTCC versus
CMS
Data Sources:20 out of 600
3%
Filter Nodes: 14 out of 2000
0.3%
Trigger rate: 100 Hz out of 100 kHz
0.1%
Event size: 200 kB out of 1 MB
15. 15
FMs Control Structure at MTCC I & II
15ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
Web Browser (GUI)
Level 0 FM
Level 1 FM
Level 2 FM
User interaction with Web Browser
connected to Level 0 FM.
Level 0 FM is entry point to
Run Control System.
Level 2 FMs are sub-system
specific custom
implementations.
Level 1 FM interface to
the Level 0 FM and have
to implement a standard
set of inputs and states.
TOP
LTC
CSC DAQ
RPC DT
TRK
ECAL
HCAL
FB RB FF
Resources
FEC FED
Resources are on-line
system components
16. 16
RCMS at MTCC I & II
ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
Top DAQ
LTC
HCAL
ECAL
DT
CSC
RPC
TRK
RCMS Operation Scenario
– Sub-system function managers were written
using the RCMS software
– The run configuration was communicated
via a global configuration key
– The Run Info DB was used to store end-of-
run summary information and status
information about the run.
It also contained the schema to generate
Run Numbers and Run Sequence Numbers.
N
Sub-Detector controlled 8
Function Managers used 14
Online resources controlled ~ 100
17. 17
RCMS Components at MTCC I & II
17ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
global key
local key
configuration
logmessage
Top DAQ
LTC-Trg
HCAL
ECAL
DT
CSC
RPC
Global
Configuration
Keys
Global
Configuration
Keys
RPC
RS
RPC
RS
LTC-Trg
RS
LTC-Trg
RS
EMU
RS
EMU
RS
DT
RS
DT
RS
DAQ
RS
DAQ
RS
global key
ECAL
RS
ECAL
RS
HCAL
RS
HCAL
RS
TRK TRK
RS
TRK
RS
LOG DBLOG DB
Top
RS
Top
RS
Collector Collector
Collector
Collector
Collector
Collector
Collector
Collector
Collector
Configuration
A Global configuration
Key identified a sub-
system configuration.
The configuration local
to the sub-system were
decouple from each
other and the top
configuration.
19. 19
MTCC result
19ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
• RCMS software was
stable.
• Separation of Subsystem
installations worked well.
• Recorded ~ 160 M
events on a period of one
month
20. 20
RCMS and GRIDCC
20ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
• It is a project of 3-years and started
in September 2004
• Web site: www.gridcc.org
What is GRIDCC ?
The Grid enabled Remote Instrumentation with Distributed Control and
Computation (GRIDCC) is a project funded by the European
community, aimed to provide access to and control of distributed
complex instrumentation.
The CMS RCMS is one of the main applications for the GRIDCC project .
The RCMS software is the core of the
Instrument Element of the GRIDCC.
21. 2121ACAT 2007 23-27 April 2007, Amsterdam Andrea Petrucci - LNL-INFN
Thank you for your attention.
• Any Questions?