DPDK Summit 2015 in San Francisco.
Presentation by Charles Shiflett, Aspera.
For additional details and the video recording please visit www.dpdksummit.com.
DPDK Summit 2015 - Sprint - Arun RajagopalJim St. Leger
DPDK Summit 2015 in San Francisco.
Presentation by Arun Rajagopal, Sprint, and Sameh Gobriel, Intel.
For additional details and the video recording please visit www.dpdksummit.com.
DPDK Summit 2015 - NTT - Yoshihiro NakajimaJim St. Leger
DPDK Summit 2015 in San Francisco.
NTT presentation by Yoshihiro Nakajima.
For additional details and the video recording please visit www.dpdksummit.com.
DPDK Summit 2015 - RIFT.io - Tim MortsolfJim St. Leger
DPDK Summit 2015 in San Francisco.
Presentation by RIFT.io's CTO Tim Mortsolf.
For additional details and the video recording please visit www.dpdksummit.com.
DPDK Summit 2015 - Sprint - Arun RajagopalJim St. Leger
DPDK Summit 2015 in San Francisco.
Presentation by Arun Rajagopal, Sprint, and Sameh Gobriel, Intel.
For additional details and the video recording please visit www.dpdksummit.com.
DPDK Summit 2015 - NTT - Yoshihiro NakajimaJim St. Leger
DPDK Summit 2015 in San Francisco.
NTT presentation by Yoshihiro Nakajima.
For additional details and the video recording please visit www.dpdksummit.com.
DPDK Summit 2015 - RIFT.io - Tim MortsolfJim St. Leger
DPDK Summit 2015 in San Francisco.
Presentation by RIFT.io's CTO Tim Mortsolf.
For additional details and the video recording please visit www.dpdksummit.com.
Accelerate Service Function Chaining Vertical Solution with DPDKOPNFV
Service Function Chaining (SFC) is one of top 5 NFV use case. Supporting SFC in provider and enterprise networks requires performance assurance. Specifically, the Classifier and the Service Function Forwarder which are typically implemented in software such as virtual switches need to match line rate requirement. DPDK (Data Plane Development Kit) is an open source project comprising a set of libraries and drivers for fast packet processing. In this presentation, we will discuss our experiences accelerating SFC with DPDK. In addition, Telco and Datacenter carriers demands dynamic SFC that requires new SFC wire protocols (e.g. VxLAN-GPE and NSH) support in both data and control planes. We intend to share our experiences and future works of a high performance, NSH-aware SFC vertical solution with open-source ingredients: Openstack, Opendaylight, OpenvSwitch with DPDK acceleration.
DPDK Summit 2015 in San Francisco.
Intel's presentation by Keith Wiles.
For additional details and the video recording please visit www.dpdksummit.com.
DPDK Summit - 08 Sept 2014 - 6WIND - High Perf Networking Leveraging the DPDK...Jim St. Leger
Thomas Monjalon, 6WIND, presents on where/how to use DPDK, the DPDK ecosystem, and the DPDK.org community.
Thomas is the community maintainer of DPDK.org.
Packet processing in the fast path involves looking up bit patterns and deciding on an actions at line rate. The complexity of these functions at Line Rate, have been traditionally handled by ASICs and NPUs. However with the availability of faster and cheaper CPUs and hardware/software accelerations, it is possible to move these functions onto commodity hardware. This tutorial will talk about the various building blocks available to speed up packet processing both hardware based e.g. SR-IOV, RDT, QAT, VMDq, VTD and software based e.g. DPDK, Fd.io/VPP, OVS etc and give hands on lab experience on DPDK and fd.io fast path look up with following sessions. 1: Introduction to Building blocks: Sujata Tibrewala
This presentation introduces Data Plane Development Kit overview and basics. It is a part of a Network Programming Series.
First, the presentation focuses on the network performance challenges on the modern systems by comparing modern CPUs with modern 10 Gbps ethernet links. Then it touches memory hierarchy and kernel bottlenecks.
The following part explains the main DPDK techniques, like polling, bursts, hugepages and multicore processing.
DPDK overview explains how is the DPDK application is being initialized and run, touches lockless queues (rte_ring), memory pools (rte_mempool), memory buffers (rte_mbuf), hashes (rte_hash), cuckoo hashing, longest prefix match library (rte_lpm), poll mode drivers (PMDs) and kernel NIC interface (KNI).
At the end, there are few DPDK performance tips.
Tags: access time, burst, cache, dpdk, driver, ethernet, hub, hugepage, ip, kernel, lcore, linux, memory, pmd, polling, rss, softswitch, switch, userspace, xeon
DPDK (Data Plane Development Kit) Overview by Rami Rosen
* Background and short history
* Advantages and disadvantages
- Very High speed networking acceleration in L2
- How this acceleration is achieved (hugepages, optimizations)
- rte_kni (and KCP)
- VPP (and FD.io project) , providing routing and switching.
- TLDK (Transport Layer Development Kit, TCP/UDP)
* Anatomy of a simple DPDK application.
* Development and governance model
* Testpmd: DPDK CLI tool
* DDP - Dynamic Device Profiles
Rami Rosen is a Linux Kernel expert, the author of "Linux Kernel Networking", Apress, 2014.
Rami had published two articles about DPDK in the last year:
"Network acceleration with DPDK"
https://lwn.net/Articles/725254/
"Userspace Networking with DPDK"
https://www.linuxjournal.com/content/userspace-networking-dpdk
Seven years ago at LCA, Van Jacobsen introduced the concept of net channels but since then the concept of user mode networking has not hit the mainstream. There are several different user mode networking environments: Intel DPDK, BSD netmap, and Solarflare OpenOnload. Each of these provides higher performance than standard Linux kernel networking; but also creates new problems. This talk will explore the issues created by user space networking including performance, internal architecture, security and licensing.
DPDK Summit - 08 Sept 2014 - Intel - Networking Workloads on Intel ArchitectureJim St. Leger
Venky Venkatesan presents information on the Data Plane Development Kit (DPDK) including an overview, background, methodology, and future direction and developments.
Tim O'Driscoll, Intel, closes out the first-ever DPDK Summit with a look at the DPDK open source journey and history and finishes with some next steps for the community.
DPDK Summit 2015 - Intro - Tim O'DriscollJim St. Leger
DPDK Summit 2015 in San Francisco.
Introductory comments and kick-off by Tim O'Driscoll, Intel.
For additional details and the video recording please visit www.dpdksummit.com.
Accelerate Service Function Chaining Vertical Solution with DPDKOPNFV
Service Function Chaining (SFC) is one of top 5 NFV use case. Supporting SFC in provider and enterprise networks requires performance assurance. Specifically, the Classifier and the Service Function Forwarder which are typically implemented in software such as virtual switches need to match line rate requirement. DPDK (Data Plane Development Kit) is an open source project comprising a set of libraries and drivers for fast packet processing. In this presentation, we will discuss our experiences accelerating SFC with DPDK. In addition, Telco and Datacenter carriers demands dynamic SFC that requires new SFC wire protocols (e.g. VxLAN-GPE and NSH) support in both data and control planes. We intend to share our experiences and future works of a high performance, NSH-aware SFC vertical solution with open-source ingredients: Openstack, Opendaylight, OpenvSwitch with DPDK acceleration.
DPDK Summit 2015 in San Francisco.
Intel's presentation by Keith Wiles.
For additional details and the video recording please visit www.dpdksummit.com.
DPDK Summit - 08 Sept 2014 - 6WIND - High Perf Networking Leveraging the DPDK...Jim St. Leger
Thomas Monjalon, 6WIND, presents on where/how to use DPDK, the DPDK ecosystem, and the DPDK.org community.
Thomas is the community maintainer of DPDK.org.
Packet processing in the fast path involves looking up bit patterns and deciding on an actions at line rate. The complexity of these functions at Line Rate, have been traditionally handled by ASICs and NPUs. However with the availability of faster and cheaper CPUs and hardware/software accelerations, it is possible to move these functions onto commodity hardware. This tutorial will talk about the various building blocks available to speed up packet processing both hardware based e.g. SR-IOV, RDT, QAT, VMDq, VTD and software based e.g. DPDK, Fd.io/VPP, OVS etc and give hands on lab experience on DPDK and fd.io fast path look up with following sessions. 1: Introduction to Building blocks: Sujata Tibrewala
This presentation introduces Data Plane Development Kit overview and basics. It is a part of a Network Programming Series.
First, the presentation focuses on the network performance challenges on the modern systems by comparing modern CPUs with modern 10 Gbps ethernet links. Then it touches memory hierarchy and kernel bottlenecks.
The following part explains the main DPDK techniques, like polling, bursts, hugepages and multicore processing.
DPDK overview explains how is the DPDK application is being initialized and run, touches lockless queues (rte_ring), memory pools (rte_mempool), memory buffers (rte_mbuf), hashes (rte_hash), cuckoo hashing, longest prefix match library (rte_lpm), poll mode drivers (PMDs) and kernel NIC interface (KNI).
At the end, there are few DPDK performance tips.
Tags: access time, burst, cache, dpdk, driver, ethernet, hub, hugepage, ip, kernel, lcore, linux, memory, pmd, polling, rss, softswitch, switch, userspace, xeon
DPDK (Data Plane Development Kit) Overview by Rami Rosen
* Background and short history
* Advantages and disadvantages
- Very High speed networking acceleration in L2
- How this acceleration is achieved (hugepages, optimizations)
- rte_kni (and KCP)
- VPP (and FD.io project) , providing routing and switching.
- TLDK (Transport Layer Development Kit, TCP/UDP)
* Anatomy of a simple DPDK application.
* Development and governance model
* Testpmd: DPDK CLI tool
* DDP - Dynamic Device Profiles
Rami Rosen is a Linux Kernel expert, the author of "Linux Kernel Networking", Apress, 2014.
Rami had published two articles about DPDK in the last year:
"Network acceleration with DPDK"
https://lwn.net/Articles/725254/
"Userspace Networking with DPDK"
https://www.linuxjournal.com/content/userspace-networking-dpdk
Seven years ago at LCA, Van Jacobsen introduced the concept of net channels but since then the concept of user mode networking has not hit the mainstream. There are several different user mode networking environments: Intel DPDK, BSD netmap, and Solarflare OpenOnload. Each of these provides higher performance than standard Linux kernel networking; but also creates new problems. This talk will explore the issues created by user space networking including performance, internal architecture, security and licensing.
DPDK Summit - 08 Sept 2014 - Intel - Networking Workloads on Intel ArchitectureJim St. Leger
Venky Venkatesan presents information on the Data Plane Development Kit (DPDK) including an overview, background, methodology, and future direction and developments.
Tim O'Driscoll, Intel, closes out the first-ever DPDK Summit with a look at the DPDK open source journey and history and finishes with some next steps for the community.
DPDK Summit 2015 - Intro - Tim O'DriscollJim St. Leger
DPDK Summit 2015 in San Francisco.
Introductory comments and kick-off by Tim O'Driscoll, Intel.
For additional details and the video recording please visit www.dpdksummit.com.
Asegure el cumplimiento de Normativas Regulatorias, Evite Errores Costosos y ...Rocket Software ALM
Asegure el cumplimiento de Normativas Regulatorias, Evite Errores Costosos y Automatice el Ciclo de Vida del Desarrollo de Aplicaciones con Rocket Aldon ALM Hub
• Control de Versiones
• Control de Cambios
• Ciclo de Vida del Software
• Rocket Aldon ALM
• Application Lifecycle Management
L'Éco Actu est la revue économique du Pays de Lorient
Son objectif : parler des hommes, des entreprises, des structures, des filières qui façonnent et animent le paysage économique du Pays de Lorient
Guida lavoro estero.... cosa fare prima di partireLinda Belloni
Andare all'estero per cercare lavoro è la scelta giusta per tutti? Sai cosa ti aspetta in un altro paese? Sei pronto? Hai tutte le informazioni necessarie per partire? Leggi quello che ho scritto per te (consigli pratici, indirizzi e informazioni indispensabili) per chiarirti le idee e capire se l'estero è veramente l'opzione più valida per te!
New Ethernet standards, such as 40 GbE or 100 GbE, are already being deployed commercially along with their corresponding Network Interface Cards (NICs) for the servers. However, network measurement solutions are lagging behind: while there are several tools available for monitoring 10 or 20 Gbps networks, higher speeds pose a harder challenge that requires more new ideas, different from those applied previously, and so there are less applications available. In this paper, we show a system capable of capturing, timestamping and storing 40 Gbps network traffic using a tailored network driver together with Non-Volatile Memory express (NVMe) technology and the Storage Performance Development Kit (SPDK) framework. Also, we expose core ideas that can be extended for the capture at higher rates: a multicore architecture capable of synchronization with minimal overhead that reduces disordering of the received frames, methods to filter the traffic discarding unwanted frames without being computationally expensive, and the use of an intermediate buffer that allows simultaneous access from several applications to the same data and efficient disk writes. Finally, we show a testbed for a reliable benchmarking of our solution using custom DPDK traffic generators and replayers, which
have been made freely available for the network measurement
community.
OpenPOWER Acceleration of HPCC SystemsHPCC Systems
JT Kellington, IBM and Allan Cantle, Nallatech present at the 2015 HPCC Systems Engineering Summit Community Day about porting HPCC Systems to the POWER8-based ppc64el architecture.
HPC and cloud distributed computing, as a journeyPeter Clapham
Introducing an internal cloud brings new paradigms, tools and infrastructure management. When placed alongside traditional HPC the new opportunities are significant But getting to the new world with micro-services, autoscaling and autodialing is a journey that cannot be achieved in a single step.
Introduction to HPC & Supercomputing in AITyrone Systems
Catch up with our live webinar on Natural Language Processing! Learn about how it works and how it applies to you. We have provided all the information in our video recording you would not miss out on.
Watch the Natural Language Processing webinar here!
The overall volume of Internet traffic has been growing in a tremendous rate day-by-day which also contains unwanted malicious traffic. It has been a continuous challenge for the network operators to effectively identify the threats from line rate traffic. Hyperscan is a pattern (in terms of regular expression) matching software ideal for applications such as intrusion prevention/detection system, antivirus, unified threat management, deep packet inspection systems, etc.
Hyperscan works in two phases. At first, the customer patterns are parsed and compiled into databases in terms of bytecode. During runtime, these bytecode are used to search for patterns against blocks/streams data. Hyperscan library runs entirely in software and scales with IA processors to provide the maximum throughput of 293 Gbps.
Dataplane networking acceleration with OpenDataplane / Максим Уваров (Linaro)Ontico
HighLoad++ 2017
Зал «Москва», 7 ноября, 13:00
Тезисы:
http://www.highload.ru/2017/abstracts/2909.html
OpenDataPlane (ODP, https://www.opendataplane.org) является open-source-разработкой API для сетевых data plane-приложений, представляющий абстракцию между сетевым чипом и приложением. Сейчас вендоры, такие как TI, Freescale, Cavium, выпускают SDK с поддержкой ODP на своих микросхемах SoC. Если проводить аналогию с графическим стеком, то ODP можно сравнить с OpenGL API, но только в области сетевого программирования.
...
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storageMayaData Inc
Webinar Session - https://youtu.be/_5MfGMf8PG4
In this webinar, we share how the Container Attached Storage pattern makes performance tuning more tractable, by giving each workload its own storage system, thereby decreasing the variables needed to understand and tune performance.
We then introduce MayaStor, a breakthrough in the use of containers and Kubernetes as a data plane. MayaStor is the first containerized data engine available that delivers near the theoretical maximum performance of underlying systems. MayaStor performance scales with the underlying hardware and has been shown, for example, to deliver in excess of 10 million IOPS in a particular environment.
IBM Power9 Servers are here! Launched this week, the AC922 POWER9 servers will form the basis of the world’s fastest “Coral” supercomputers coming to ORNL and LLNL. Built specifically for compute-intensive AI workloads, the new POWER9 systems are capable of improving the training times of deep learning frameworks by nearly 4x allowing enterprises to build more accurate AI applications, faster.
Listen to the Radio Free HPC podcast on Power9: https://insidehpc.com/2017/12/radio-free-hpc-looks-new-power9-titan-v-snapdragon-845/
Learn more: https://www.ibm.com/us-en/marketplace/power-systems-ac922
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
A Dataflow Processing Chip for Training Deep Neural Networksinside-BigData.com
In this deck from the Hot Chips conference, Chris Nicol from Wave Computing presents: A Dataflow Processing Chip for Training Deep Neural Networks.
Watch the video: https://wp.me/p3RLHQ-k6W
Learn more: https://wavecomp.ai/
and
http://www.hotchips.org/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Ariel Waizel discusses the Data Plane Development Kit (DPDK), an API for developing fast packet processing code in user space.
* Who needs this library? Why bypass the kernel?
* How does it work?
* How good is it? What are the benchmarks?
* Pros and cons
Ariel worked on kernel development at the IDF, Ben Gurion University, and several companies. He is interested in networking, security, machine learning, and basically everything except UI development. Currently a Solution Architect at ConteXtream (an HPE company), which specializes in SDN solutions for the telecom industry.
Globus Connect Server Deep Dive - GlobusWorld 2024Globus
We explore the Globus Connect Server (GCS) architecture and experiment with advanced configuration options and use cases. This content is targeted at system administrators who are familiar with GCS and currently operate—or are planning to operate—broader deployments at their institution.
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns
Unlocking Business Potential: Tailored Technology Solutions by Prosigns
Discover how Prosigns, a leading technology solutions provider, partners with businesses to drive innovation and success. Our presentation showcases our comprehensive range of services, including custom software development, web and mobile app development, AI & ML solutions, blockchain integration, DevOps services, and Microsoft Dynamics 365 support.
Custom Software Development: Prosigns specializes in creating bespoke software solutions that cater to your unique business needs. Our team of experts works closely with you to understand your requirements and deliver tailor-made software that enhances efficiency and drives growth.
Web and Mobile App Development: From responsive websites to intuitive mobile applications, Prosigns develops cutting-edge solutions that engage users and deliver seamless experiences across devices.
AI & ML Solutions: Harnessing the power of Artificial Intelligence and Machine Learning, Prosigns provides smart solutions that automate processes, provide valuable insights, and drive informed decision-making.
Blockchain Integration: Prosigns offers comprehensive blockchain solutions, including development, integration, and consulting services, enabling businesses to leverage blockchain technology for enhanced security, transparency, and efficiency.
DevOps Services: Prosigns' DevOps services streamline development and operations processes, ensuring faster and more reliable software delivery through automation and continuous integration.
Microsoft Dynamics 365 Support: Prosigns provides comprehensive support and maintenance services for Microsoft Dynamics 365, ensuring your system is always up-to-date, secure, and running smoothly.
Learn how our collaborative approach and dedication to excellence help businesses achieve their goals and stay ahead in today's digital landscape. From concept to deployment, Prosigns is your trusted partner for transforming ideas into reality and unlocking the full potential of your business.
Join us on a journey of innovation and growth. Let's partner for success with Prosigns.
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...Anthony Dahanne
Les Buildpacks existent depuis plus de 10 ans ! D’abord, ils étaient utilisés pour détecter et construire une application avant de la déployer sur certains PaaS. Ensuite, nous avons pu créer des images Docker (OCI) avec leur dernière génération, les Cloud Native Buildpacks (CNCF en incubation). Sont-ils une bonne alternative au Dockerfile ? Que sont les buildpacks Paketo ? Quelles communautés les soutiennent et comment ?
Venez le découvrir lors de cette session ignite
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
Unleash Unlimited Potential with One-Time Purchase
BoxLang is more than just a language; it's a community. By choosing a Visionary License, you're not just investing in your success, you're actively contributing to the ongoing development and support of BoxLang.
Accelerate Enterprise Software Engineering with PlatformlessWSO2
Key takeaways:
Challenges of building platforms and the benefits of platformless.
Key principles of platformless, including API-first, cloud-native middleware, platform engineering, and developer experience.
How Choreo enables the platformless experience.
How key concepts like application architecture, domain-driven design, zero trust, and cell-based architecture are inherently a part of Choreo.
Demo of an end-to-end app built and deployed on Choreo.
How to Position Your Globus Data Portal for Success Ten Good PracticesGlobus
Science gateways allow science and engineering communities to access shared data, software, computing services, and instruments. Science gateways have gained a lot of traction in the last twenty years, as evidenced by projects such as the Science Gateways Community Institute (SGCI) and the Center of Excellence on Science Gateways (SGX3) in the US, The Australian Research Data Commons (ARDC) and its platforms in Australia, and the projects around Virtual Research Environments in Europe. A few mature frameworks have evolved with their different strengths and foci and have been taken up by a larger community such as the Globus Data Portal, Hubzero, Tapis, and Galaxy. However, even when gateways are built on successful frameworks, they continue to face the challenges of ongoing maintenance costs and how to meet the ever-expanding needs of the community they serve with enhanced features. It is not uncommon that gateways with compelling use cases are nonetheless unable to get past the prototype phase and become a full production service, or if they do, they don't survive more than a couple of years. While there is no guaranteed pathway to success, it seems likely that for any gateway there is a need for a strong community and/or solid funding streams to create and sustain its success. With over twenty years of examples to draw from, this presentation goes into detail for ten factors common to successful and enduring gateways that effectively serve as best practices for any new or developing gateway.
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
Why React Native as a Strategic Advantage for Startup Innovation.pdfayushiqss
Do you know that React Native is being increasingly adopted by startups as well as big companies in the mobile app development industry? Big names like Facebook, Instagram, and Pinterest have already integrated this robust open-source framework.
In fact, according to a report by Statista, the number of React Native developers has been steadily increasing over the years, reaching an estimated 1.9 million by the end of 2024. This means that the demand for this framework in the job market has been growing making it a valuable skill.
But what makes React Native so popular for mobile application development? It offers excellent cross-platform capabilities among other benefits. This way, with React Native, developers can write code once and run it on both iOS and Android devices thus saving time and resources leading to shorter development cycles hence faster time-to-market for your app.
Let’s take the example of a startup, which wanted to release their app on both iOS and Android at once. Through the use of React Native they managed to create an app and bring it into the market within a very short period. This helped them gain an advantage over their competitors because they had access to a large user base who were able to generate revenue quickly for them.
Into the Box Keynote Day 2: Unveiling amazing updates and announcements for modern CFML developers! Get ready for exciting releases and updates on Ortus tools and products. Stay tuned for cutting-edge innovations designed to boost your productivity.
Cyaniclab : Software Development Agency Portfolio.pdfCyanic lab
CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
Developing Distributed High-performance Computing Capabilities of an Open Sci...Globus
COVID-19 had an unprecedented impact on scientific collaboration. The pandemic and its broad response from the scientific community has forged new relationships among public health practitioners, mathematical modelers, and scientific computing specialists, while revealing critical gaps in exploiting advanced computing systems to support urgent decision making. Informed by our team’s work in applying high-performance computing in support of public health decision makers during the COVID-19 pandemic, we present how Globus technologies are enabling the development of an open science platform for robust epidemic analysis, with the goal of collaborative, secure, distributed, on-demand, and fast time-to-solution analyses to support public health.
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...Juraj Vysvader
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I didn't get rich from it but it did have 63K downloads (powered possible tens of thousands of websites).
Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production.
Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process?
In this session we will cover:
- The Art of Effective Code Reviews
- Streamlining the Review Process
- Elevating Reviews with Automated Tools
By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces
SOCRadar Research Team: Latest Activities of IntelBrokerSOCRadar
The European Union Agency for Law Enforcement Cooperation (Europol) has suffered an alleged data breach after a notorious threat actor claimed to have exfiltrated data from its systems. Infamous data leaker IntelBroker posted on the even more infamous BreachForums hacking forum, saying that Europol suffered a data breach this month.
The alleged breach affected Europol agencies CCSE, EC3, Europol Platform for Experts, Law Enforcement Forum, and SIRIUS. Infiltration of these entities can disrupt ongoing investigations and compromise sensitive intelligence shared among international law enforcement agencies.
However, this is neither the first nor the last activity of IntekBroker. We have compiled for you what happened in the last few days. To track such hacker activities on dark web sources like hacker forums, private Telegram channels, and other hidden platforms where cyber threats often originate, you can check SOCRadar’s Dark Web News.
Stay Informed on Threat Actors’ Activity on the Dark Web with SOCRadar!
Experience our free, in-depth three-part Tendenci Platform Corporate Membership Management workshop series! In Session 1 on May 14th, 2024, we began with an Introduction and Setup, mastering the configuration of your Corporate Membership Module settings to establish membership types, applications, and more. Then, on May 16th, 2024, in Session 2, we focused on binding individual members to a Corporate Membership and Corporate Reps, teaching you how to add individual members and assign Corporate Representatives to manage dues, renewals, and associated members. Finally, on May 28th, 2024, in Session 3, we covered questions and concerns, addressing any queries or issues you may have.
For more Tendenci AMS events, check out www.tendenci.com/events
2. ASPERA HIGH-SPEED FILE TRANSFER
2
80 GBIT/S OVER IP USING DPDK
Performance, Code, and Architecture
Charles Shiflett
Developer of next-generation FASP
bear @ us . IBM . com
bear@asperasoft.com
3. 3
RAPID PROTYPING OF BARE METAL SOLUTIONS
3
Benefits of using DPDK
• Lightweight API that exposes Network and Acceleration Cards
• Rapid Iteration on high performance transport designs as
compared to equivalent code developed as Kernel Module
• Set of performance best practices
- Time, Locking, Memory Alignment, and Synchronization
- Avoids libc and system call interaction when possible
• NUMA aware solution
• Framework for High Performance Userland Direct I/O
4. FASP NX DESIGN GOAL
4
Goal is to bring industry closer to the data they
are using both within the data center and across
the globe.
• Effectively utilize PCIe based IO on Intel Server platform.
• Solve storage bottleneck using Direct I/O w/ multiple controllers
• Solve network bottleneck using DPDK and link aggregation
• Provide a secure transport solution using Intel AES-NI GCM
• Significantly reduce time spent getting your data set to the CPU
6. ASPERA’S MISSION
Creating next-generation transport technologies
that move the world’s digital assets at maximum speed,
regardless of file size, transfer distance and network conditions.
6
7. CHALLENGES WITH TCP AND ALTERNATIVE TECHNOLOGIES
Distance degrades conditions on all networks
• Latency (or Round Trip Times) increase
• Packet losses increase
• Fast networks just as prone to degradation
TCP performance degrades with distance
• Throughput bottleneck becomes more severe with
increased latency and packet loss
TCP does not scale with bandwidth
• TCP designed for low bandwidth
• Adding more bandwidth does not improve throughput
Alternative Technologies
• TCP-based - Network latency and packet loss must be low
• UDP traffic blasters - Inefficient and waste bandwidth
• Data caching - Inappropriate for many large file transfer workflows
• Modified TCP - Improves on TCP performance but insufficient for fast networks
• Data compression - Time consuming and impractical for certain file types
• CDNs & co-lo build outs - High overhead and expensive to scale
7
8. FASP™: HIGH-PERFORMANCE TRANSPORT
Maximum transfer speed
• Optimal end-to-end throughput efficiency
• Transfer performance scales with bandwidth independent of transfer distance
and resilient to packet loss
Congestion Avoidance and Policy Control
• Automatic, full utilization of available bandwidth
• On-the-fly prioritization and bandwidth allocation
Uncompromising security and reliability
• Secure, user/endpoint authentication
• AES-128 cryptography in transit and at-rest
Scalable management, monitoring and control
• Real-time progress, performance and bandwidth utilization
• Detailed transfer history, logging, and manifest
Low Overhead
• Less than 0.1% overhead on 30% packet loss
• High performance with large files or large sets of small files
Resulting in
• Transfers up to thousands of times faster than FTP
• Precise and predictable transfer times
• Extreme scalability (concurrency and throughput)
8
9. LAN TRANSFER RESULTS IN GBIT PER SECOND
9
0
10
20
30
40
50
60
70
80
SSH
AES-128 CTR
FTP
iperf
FASP 4x10Gbit
(Intel® Xeon®
E5-2650 v2)
FASP
(Intel® Xeon®
E5-2697 v3)
Disk to Disk
Wire Utilization w/o disk
Results from SC 14 showing the relative performance of Network Transfer Technologies
12. AVOIDING STALLS
• Synchronization primitives aligned to cache-line, lockless. Example for circular buffer queues;
Dequeue, claim block: block_id = __sync_fetch_and_add ( &sess->block_tail, 1 ) % block_count;
Dequeue, consume block: while ( !__sync_val_compare_and_swap( &sess->block[block_id].state, 0, 1) ) PAUSE;
• Zero copy paradigm critical to good memory performance. Memory copies are VERY expensive!
• Memory copies should be avoided at all costs, writes to Memory must be page aligned to avoid read of
destination cacheline
• Solution is to operate on datasets which fit into L3 cache. New tools provide better insight into L3 cache
utilization such that code can be optimized.
• Low latency direct I/O.
• Intel DDIO critical in bringing data into L3 from Storage, Network Adapters, and PCIe Accelerators.
• DPDK provides framework for operating on data streamed into L3 cache
• Avoid TLB misses by using Hugepages
• Avoid NUMA issues and minimize thread preemption.
• Userland IO drivers where possible.
12
13. EXAMPLE OF ALIGNED/UNALIGNED MEMORY COPY
13
5.7 GB/S copy performance constrained by RFO
9.0 GB/s copy avoids RFO, close to per core limit
* Per channel results via Intel Performance Counter Monitor
14. OPTIMIZING NETWORK THROUGHPUT
14
• Create network queues which fit into L3. Each queue tied to a specific core.
• Experiment with values to minimize memory, while retaining useful properties:
• Should support bulk allocation
• MAX_RX_BURST < rx_free_thresh < nb_rx_desc (set when assigning queue to port)
• Nb_rx_desc % rx_free_thresh == 0
• Ideally pkts_in_cache % nb_pkts == 0 and nb_pkts == 2^n-1
• PTHRESH/HTHRESH/WTHRESH values feel like black magic, but can have a huge
effect on performance. Start by copying values from example applications.
15. IMPROVING STORAGE THROUGHPUT
• Traditional storage is built around the idea of moving slow data from disk to memory and then from
memory to application. Memory is used to cache data to improve access speeds.
• Cache structure quickly becomes a bottleneck as transfer speeds exceed 10 gbit/s.
• While individual spinning disks are slow, JBOD’s of 100’s of disk have very high aggregate bandwidth
• Modern SSDs (especially NVMe) is very fast.
• Two solutions for fast data
• Use XFS with direct IO (and/or MMAP)
• Have shown performance at about 40gbit/s with hardware raid and direct attach disks.
• Have shown performance at about 70gbit/s with NVMe SSD.
• Limited by how many devices you can connect to PCIe data link.
• Clustered parallel storage
• Aspera is targeting IBM GPFS (Spectrum) or Lustre based solutions
• Individual nodes can be slow, but by aggregating nodes high performance is achieved
• Both offer direct I/O solutions
15
16. DC P3700 PERFORMANCE (SINGLE DRIVE)
16
Performance relative to block size (SI Units in MB)
2678 MB/s
1869 MB/s
0
500
1000
1500
2000
2500
3000
128 16 4 1
Write
Synchronous Read
Async Read (AIO)
2800 MB/s
17. ENCRYPTION IS NOT A BOTTLENECK WITH HW AES-GCM
17
0
0.5
1
1.5
2
2.5
Core i7 X 980
3.33GHz
Xeon E5-2650 v2
2.60GHz
Xeon E5-2697 v3
2.60GHz
AES 128 GCM Encryption Rate in GB/s per Core
18. NEW ON THE WIRE PROTOCOL
• Built around AES-128 GCM, which is similar to DTLS ( Datagram TLS ).
18
AES-128 GCM {
uint64 IV;
uint8[] payload;
uint8[16] MAC;
}
DTLSRecord {
ContentType type;
ProtocolVersion version;
uint16 epoch;
uint48 sequence_number;
uint16 length;
uint8_t[length] payload;
}
MAC {
uint8[16] hash;
}
FASP4 {
uint16 stream_id;
uint8 ver_and_pkt_type;
uint40 sequence_number;
uint8[] payload;
uint8[16] hash; // optional
}
• In the same way it is possible to encapsulate FASP NX header in UDP, will also support
Encapsulating FASP4 in DTLS.
20. FASP NX API’S
• Native FASP NX API
• Share hugepage sized regions of memory with client application
• Usable from VM or Native machine
• Enables Zero Copy transfer solutions of non file data
• Dynamically set rate policy, destination IP, and so on.
• Traditional Aspera (ascp) command line tools
• Similar command line to SCP, with FASP NX performance.
• Existing Aspera API
• Faspmgmt
• Web Services
• Rest based Queries
20
21. FUTURE DIRECTION
• Aspera’s Goal: Transfer Solution to Petascale Datasets
- 1 SI Terabyte in 2 Minutes
- 1 SI Petabyte in 1⅜ Days
- Performance improvements expected to scale relative to PCIe interconnect.
• Better integration with storage systems
- Take advantage of native APIs to avoid kernel cache.
• Better integration with network hardware
- Expected to show 100gbit/s transfers using Mellanox ConnectX®-4
- Query network switch and routers?
• Support wider use cases
- Compression, Forward Error Correcting Codecs, Berkeley Sockets API
21