"JBoss clustering solutions Mission Critical Enterprise" by Mircea Markus @ e...eLiberatica
This is a presentation held at eLiberatica 2009.
http://www.eliberatica.ro/2009/
One of the biggest events of its kind in Eastern Europe, eLiberatica brings community leaders from around the world to discuss about the hottest topics in FLOSS movement, demonstrating the advantages of adopting, using and developing Open Source and Free Software solutions.
The eLiberatica organizational committee together with our speakers and guests, have graciously allowed media representatives and all attendees to photograph, videotape and otherwise record their sessions, on the condition that the photos, videos and recordings are licensed under the Creative Commons Share-Alike 3.0 License.
"Apache Spark is today’s fastest growing Big Data analysis platform. Spark workloads typically maintain a persistent data set in memory, which is accessed multiple times over the network. Consequently, networking IO performance is a critical component in Spark systems. RDMA’s performance characteristics, such as high bandwidth, low latency, and low CPU overhead, offer a good opportunity for accelerating Spark by improving its data transfer facilities."
"In this talk, we present a Java-based, RDMA network layer for Apache Spark. The implementation optimized both the RPC and the Shuffle mechanisms for RDMA. Initial benchmarking shows up to 25% improvement for Spark Applications."
Watch the video presentation: http://wp.me/p3RLHQ-gzN
Learn more: http://mellanox.com
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Greg Casey from Dell EMC presented this talk at the OpenFabrics Workshop: GEN-Z: An Overview and Use Cases.
“This session will focus on the new Gen-Z memory-semantic fabric. The speaker will show the audience why Gen-Z is needed, how Gen-Z operates, what is expected in first products that employ Gen-Z, and encourage participation in finalizing the Gen-Z specifications. Gen-Z will be connecting components inside of servers as well as connecting servers with pools of memory, storage, and acceleration devices through a switch environment.”
Watch the video: http://insidehpc.com/2017/04/gen-z-overview-use-cases/
Learn more: http://genzconsortium.org/
and
https://www.openfabrics.org/index.php/2017-ofa-workshop-presentations.html
Accelerating Hadoop, Spark, and Memcached with HPC Technologiesinside-BigData.com
DK Panda from Ohio State University presented this deck at the OpenFabrics Workshop.
"Modern HPC clusters are having many advanced features, such as multi-/many-core architectures, highperformance RDMA-enabled interconnects, SSD-based storage devices, burst-buffers and parallel file systems. However, current generation Big Data processing middleware (such as Hadoop, Spark, and Memcached) have not fully exploited the benefits of the advanced features on modern HPC clusters. This talk will present RDMA-based designs using OpenFabrics Verbs and heterogeneous storage architectures to accelerate multiple components of Hadoop (HDFS, MapReduce, RPC, and HBase), Spark and Memcached. An overview of the associated RDMA-enabled software libraries (being designed and publicly distributed as a part of the HiBD project for Apache Hadoop (integrated and plug-ins for Apache, HDP, and Cloudera distributions), Apache Spark and Memcached will be presented. The talk will also address the need for designing benchmarks using a multi-layered and systematic approach, which can be used to evaluate the performance of these Big Data processing middleware."
Watch the video presentation: http://wp.me/p3RLHQ-gzg
Learn more: http://hibd.cse.ohio-state.edu/
and
https://www.openfabrics.org/index.php/abstracts-agenda.html
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Building Efficient HPC Clouds with MCAPICH2 and RDMA-Hadoop over SR-IOV Infin...inside-BigData.com
Xiaoyi Lu from Ohio State University presented this deck at the OpenFabrics Workshop.
"Single Root I/O Virtualization (SR-IOV) technology has been steadily gaining momentum for high performance interconnects such as InfiniBand. SR-IOV can deliver near native performance but lacks locality-aware communication support. This talk presents an efficient approach to building HPC clouds based on MVAPICH2 and RDMA-Hadoop with SR-IOV. We discuss high-performance designs of the
virtual machine and container aware MVAPICH2 library over SR-IOV enabled HPC Clouds."
This talk will also present a high-performance virtual machine migration framework for MPI applications on SR-IOV enabled InfiniBand clouds. The MVAPICH2 software for building HPC Clouds presented in this talk is publicly available. We will also discuss how to leverage the high-performance networking features (e.g., RDMA, SR-IOV) on cloud environments to accelerate data processing through RDMAHadoop package, which is publicly available. Comprehensive performance evaluations on NSF-supported Chameleon Cloud show that our design can deliver the near bare-metal performance."
Watch the video: http://wp.me/p3RLHQ-gB3
Learn more: http://%20mvapich.cse.ohio-state.edu/
and
https://www.openfabrics.org/index.php/abstracts-agenda.html
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
"JBoss clustering solutions Mission Critical Enterprise" by Mircea Markus @ e...eLiberatica
This is a presentation held at eLiberatica 2009.
http://www.eliberatica.ro/2009/
One of the biggest events of its kind in Eastern Europe, eLiberatica brings community leaders from around the world to discuss about the hottest topics in FLOSS movement, demonstrating the advantages of adopting, using and developing Open Source and Free Software solutions.
The eLiberatica organizational committee together with our speakers and guests, have graciously allowed media representatives and all attendees to photograph, videotape and otherwise record their sessions, on the condition that the photos, videos and recordings are licensed under the Creative Commons Share-Alike 3.0 License.
"Apache Spark is today’s fastest growing Big Data analysis platform. Spark workloads typically maintain a persistent data set in memory, which is accessed multiple times over the network. Consequently, networking IO performance is a critical component in Spark systems. RDMA’s performance characteristics, such as high bandwidth, low latency, and low CPU overhead, offer a good opportunity for accelerating Spark by improving its data transfer facilities."
"In this talk, we present a Java-based, RDMA network layer for Apache Spark. The implementation optimized both the RPC and the Shuffle mechanisms for RDMA. Initial benchmarking shows up to 25% improvement for Spark Applications."
Watch the video presentation: http://wp.me/p3RLHQ-gzN
Learn more: http://mellanox.com
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Greg Casey from Dell EMC presented this talk at the OpenFabrics Workshop: GEN-Z: An Overview and Use Cases.
“This session will focus on the new Gen-Z memory-semantic fabric. The speaker will show the audience why Gen-Z is needed, how Gen-Z operates, what is expected in first products that employ Gen-Z, and encourage participation in finalizing the Gen-Z specifications. Gen-Z will be connecting components inside of servers as well as connecting servers with pools of memory, storage, and acceleration devices through a switch environment.”
Watch the video: http://insidehpc.com/2017/04/gen-z-overview-use-cases/
Learn more: http://genzconsortium.org/
and
https://www.openfabrics.org/index.php/2017-ofa-workshop-presentations.html
Accelerating Hadoop, Spark, and Memcached with HPC Technologiesinside-BigData.com
DK Panda from Ohio State University presented this deck at the OpenFabrics Workshop.
"Modern HPC clusters are having many advanced features, such as multi-/many-core architectures, highperformance RDMA-enabled interconnects, SSD-based storage devices, burst-buffers and parallel file systems. However, current generation Big Data processing middleware (such as Hadoop, Spark, and Memcached) have not fully exploited the benefits of the advanced features on modern HPC clusters. This talk will present RDMA-based designs using OpenFabrics Verbs and heterogeneous storage architectures to accelerate multiple components of Hadoop (HDFS, MapReduce, RPC, and HBase), Spark and Memcached. An overview of the associated RDMA-enabled software libraries (being designed and publicly distributed as a part of the HiBD project for Apache Hadoop (integrated and plug-ins for Apache, HDP, and Cloudera distributions), Apache Spark and Memcached will be presented. The talk will also address the need for designing benchmarks using a multi-layered and systematic approach, which can be used to evaluate the performance of these Big Data processing middleware."
Watch the video presentation: http://wp.me/p3RLHQ-gzg
Learn more: http://hibd.cse.ohio-state.edu/
and
https://www.openfabrics.org/index.php/abstracts-agenda.html
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Building Efficient HPC Clouds with MCAPICH2 and RDMA-Hadoop over SR-IOV Infin...inside-BigData.com
Xiaoyi Lu from Ohio State University presented this deck at the OpenFabrics Workshop.
"Single Root I/O Virtualization (SR-IOV) technology has been steadily gaining momentum for high performance interconnects such as InfiniBand. SR-IOV can deliver near native performance but lacks locality-aware communication support. This talk presents an efficient approach to building HPC clouds based on MVAPICH2 and RDMA-Hadoop with SR-IOV. We discuss high-performance designs of the
virtual machine and container aware MVAPICH2 library over SR-IOV enabled HPC Clouds."
This talk will also present a high-performance virtual machine migration framework for MPI applications on SR-IOV enabled InfiniBand clouds. The MVAPICH2 software for building HPC Clouds presented in this talk is publicly available. We will also discuss how to leverage the high-performance networking features (e.g., RDMA, SR-IOV) on cloud environments to accelerate data processing through RDMAHadoop package, which is publicly available. Comprehensive performance evaluations on NSF-supported Chameleon Cloud show that our design can deliver the near bare-metal performance."
Watch the video: http://wp.me/p3RLHQ-gB3
Learn more: http://%20mvapich.cse.ohio-state.edu/
and
https://www.openfabrics.org/index.php/abstracts-agenda.html
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
"Applications, programming languages, and libraries that leverage sophisticated network hardware capabilities have a natural advantage when used in today’s and tomorrow’s high-performance and data center computer environments. Modern RDMA based network interconnects provides incredibly rich functionality (RDMA, Atomics, OS-bypass, etc.) that enable low-latency and high-bandwidth communication services. The functionality is supported by a variety of interconnect technologies such as InfiniBand, RoCE, iWARP, Intel OPA, Cray’s Aries/Gemini, and others. OFA organization and LinuxRDMA community have been playing a predominant role in the enablement efficient and vendor agnostic software stack for those interconnects. Over the last decade, the community has developed variety user/kernel level protocols and libraries that enable a variety of applications over RDMA including MPI, SHMEM, NFS over RDMA, IPoIB, and many others."
"With the emerging availability server platforms based on ARM CPU architecture, it is important to understand ARM integrates with RDMA hardware and software eco-system. In this talk, we will overview ARM architecture and system software stack. We will discuss how ARM CPU interacts with network devices and accelerators. In addition, we will share our experience in enabling RDMA software stack (OFED/MOFED Verbs) and one-sided communication libraries (Open UCX, OpenSHMEM/SHMEM) on ARM and share preliminary evaluation results."
Watch the video presentation: http://wp.me/p3RLHQ-gyO
Learn more: https://www.openfabrics.org/index.php/abstracts-agenda.html
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Five cool ways the JVM can run Apache Spark fasterTim Ellison
The IBM JVM runs Apache Spark fast! This talk explains some of the findings and optimizations from our experience of running Spark workloads.
The talk was originally presented at the SparkEU Summit 2015 in Amsterdam.
This is the slide deck of the Zend webinar "Introduction to column oriented databases in PHP".
This webinar is a quick crash-course and practical session that will explain:
* What are column oriented databases and how do they differ from the standard rational (row oriented) databases.
* Advantages and disadvantages
* Is a column oriented database for me?
* How to use it from PHP
Presenter – Slavey Karadzhov
"Session ID: BUD17-503
Session Name: The HPE Machine and Gen-Z - BUD17-503
Speaker:
Grant Likely
Track:
★ Session Summary ★
With the exponential rise in quantity of data to manage, the modern data centre is increasingly limited by the capacity of individual machines. Since storage and compute demand more capacity than can be provided by a single machine, we distribute both over large clusters and use the network to transfer data between where it is stored and where it is processed. Moving all that data around uses deep storage stacks which incur a significant performance impact. If we could somehow flatten the storage stack and provide applications with direct access to data, then we could improve performance by orders of magnitude.
Hewlett Packard Enterprise recently demonstrated that we can do exactly with their research project, ""The Machine"". Instead of moving data around with a network, The Machine uses multi terabytes of persistent memory and a next generation fabric-attached memory interconnect to provide a single pool of storage which can be accessed by any processor in the cluster. It shows that we can provide applications with immediate load/store access to huge data sets in a model called Memory-Driven Computing.
Proof in hand, now it is time to bring Memory-Defined Computing to the data centre. Gen-Z is an open systems interconnect designed to provide memory semantic access to data and devices via direct attached, switched or fabric topologies. HPE has joined the Gen-Z consortium and is using the knowledge gained with The Machine to help shape Gen-Z to set the stage for true Memory-Driven Computing. With putting memory at the centre, this enables us to overcome the limitations of today's computing systems and power innovations.
This session will cover two topics. It will start with a status update on The Machine and an overview of how it works. Then we'll shift into an introduction of Gen-Z, and how it can reshape the architecture of computing in the years to come.
---------------------------------------------------
★ Resources ★
Event Page: http://connect.linaro.org/resource/bud17/bud17-503/
Presentation:
Video: https://youtu.be/1BVtChDQVyQ
---------------------------------------------------
★ Event Details ★
Linaro Connect Budapest 2017 (BUD17)
6-10 March 2017
Corinthia Hotel, Budapest,
Erzsébet krt. 43-49,
1073 Hungary
---------------------------------------------------
Keyword: HPE, Gen-Z
http://www.linaro.org
http://connect.linaro.org
---------------------------------------------------
Follow us on Social Media
https://www.facebook.com/LinaroOrg
https://twitter.com/linaroorg
https://www.youtube.com/user/linaroorg?sub_confirmation=1
https://www.linkedin.com/company/1026961"
Kerberos is the system which underpins the vast majority of strong authentication across the Apache HBase/Hadoop application stack. Kerberos errors have brought many to their knees and it is often referred to as “black magic” or “the dark arts”; a long-standing joke that there are so few who understand how it works. This talk will cover the types of problems that Kerberos solves and doesn’t solve for HBase, decrypt some jargon on related libraries and technology that enable Kerberos authentication in HBase and Hadoop, and distill some basic takeaways designed to ease users in developing an application that can securely communicate with a “kerberized” HBase installation.
Open Source Backup Conference 2014: Bakup to and of the cloud, by Marco van W...NETWAYS
Newest developments in being able to use Bareos for backing up and restoring data of a Gluster or/and CEPH storage cluster. For this we use GFAPI for Gluster and librados/libcephfs (2 plugins) for CEPH. Next to that we also added support for
using Gluster and CEPH as a backing store for storing backup data using the native APIs not a FUSE mount (so without all the additional overhead.)
What do we support now and what is planned. (Currently some is internal development and some is as technology preview in our current bleeding edge code base.)
Under the Armor of Knights Corner: Intel MIC Architecture at Hotchips 2012Intel IT Center
George Chrysos, the leading architect of Intel Xeon Phi co-processor shared the new architecture details of upcoming Intel's HPC powerhouse. Designed for highly-parallel applications, Intel Xeon Phi co-processor based on Intel Mani Integrated Core architecture will deliver the combination of industry leading performance per watt with the ability to re-use the existing code and applications without necessity of re-writing them. Equipped with more than 50 cores and built using Intel's latest 22nm 3D Tri-gate transistor technology, new co-processors will be in production this year with first supercomputers from top500 list already taking advantage of this technology.
Augmenting Flow Operations and Feedback on the Model Driven MD_SAL Approach i...Brent Salisbury
Augmenting Flow Operations and Feedback on the Model Driven MD_SAL Approach in OpenDaylight. Will post more details in a blog entry at http://networkstatic.net as soon as time permits for those looking for more information.
Cheers,
-Brent
A real-time speech to text conversion system converts the spoken words into text . Speech-to-Text technology enables us to convert audio to text by applying powerful neural network models. It has a number of applications for users with and without disabilities. Speech-to-text has been used for voice search, help writers boost their productivity, and to provide alternate access to a computer for individuals with physical impairments. Other applications include speech recognition for foreign language learning, voice-activated products for the blind and many familiar mainstream technologies. It is a driving force behind the success of new age voice-controlled speakers like Amazon Echo and Google Home.
"Applications, programming languages, and libraries that leverage sophisticated network hardware capabilities have a natural advantage when used in today’s and tomorrow’s high-performance and data center computer environments. Modern RDMA based network interconnects provides incredibly rich functionality (RDMA, Atomics, OS-bypass, etc.) that enable low-latency and high-bandwidth communication services. The functionality is supported by a variety of interconnect technologies such as InfiniBand, RoCE, iWARP, Intel OPA, Cray’s Aries/Gemini, and others. OFA organization and LinuxRDMA community have been playing a predominant role in the enablement efficient and vendor agnostic software stack for those interconnects. Over the last decade, the community has developed variety user/kernel level protocols and libraries that enable a variety of applications over RDMA including MPI, SHMEM, NFS over RDMA, IPoIB, and many others."
"With the emerging availability server platforms based on ARM CPU architecture, it is important to understand ARM integrates with RDMA hardware and software eco-system. In this talk, we will overview ARM architecture and system software stack. We will discuss how ARM CPU interacts with network devices and accelerators. In addition, we will share our experience in enabling RDMA software stack (OFED/MOFED Verbs) and one-sided communication libraries (Open UCX, OpenSHMEM/SHMEM) on ARM and share preliminary evaluation results."
Watch the video presentation: http://wp.me/p3RLHQ-gyO
Learn more: https://www.openfabrics.org/index.php/abstracts-agenda.html
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Five cool ways the JVM can run Apache Spark fasterTim Ellison
The IBM JVM runs Apache Spark fast! This talk explains some of the findings and optimizations from our experience of running Spark workloads.
The talk was originally presented at the SparkEU Summit 2015 in Amsterdam.
This is the slide deck of the Zend webinar "Introduction to column oriented databases in PHP".
This webinar is a quick crash-course and practical session that will explain:
* What are column oriented databases and how do they differ from the standard rational (row oriented) databases.
* Advantages and disadvantages
* Is a column oriented database for me?
* How to use it from PHP
Presenter – Slavey Karadzhov
"Session ID: BUD17-503
Session Name: The HPE Machine and Gen-Z - BUD17-503
Speaker:
Grant Likely
Track:
★ Session Summary ★
With the exponential rise in quantity of data to manage, the modern data centre is increasingly limited by the capacity of individual machines. Since storage and compute demand more capacity than can be provided by a single machine, we distribute both over large clusters and use the network to transfer data between where it is stored and where it is processed. Moving all that data around uses deep storage stacks which incur a significant performance impact. If we could somehow flatten the storage stack and provide applications with direct access to data, then we could improve performance by orders of magnitude.
Hewlett Packard Enterprise recently demonstrated that we can do exactly with their research project, ""The Machine"". Instead of moving data around with a network, The Machine uses multi terabytes of persistent memory and a next generation fabric-attached memory interconnect to provide a single pool of storage which can be accessed by any processor in the cluster. It shows that we can provide applications with immediate load/store access to huge data sets in a model called Memory-Driven Computing.
Proof in hand, now it is time to bring Memory-Defined Computing to the data centre. Gen-Z is an open systems interconnect designed to provide memory semantic access to data and devices via direct attached, switched or fabric topologies. HPE has joined the Gen-Z consortium and is using the knowledge gained with The Machine to help shape Gen-Z to set the stage for true Memory-Driven Computing. With putting memory at the centre, this enables us to overcome the limitations of today's computing systems and power innovations.
This session will cover two topics. It will start with a status update on The Machine and an overview of how it works. Then we'll shift into an introduction of Gen-Z, and how it can reshape the architecture of computing in the years to come.
---------------------------------------------------
★ Resources ★
Event Page: http://connect.linaro.org/resource/bud17/bud17-503/
Presentation:
Video: https://youtu.be/1BVtChDQVyQ
---------------------------------------------------
★ Event Details ★
Linaro Connect Budapest 2017 (BUD17)
6-10 March 2017
Corinthia Hotel, Budapest,
Erzsébet krt. 43-49,
1073 Hungary
---------------------------------------------------
Keyword: HPE, Gen-Z
http://www.linaro.org
http://connect.linaro.org
---------------------------------------------------
Follow us on Social Media
https://www.facebook.com/LinaroOrg
https://twitter.com/linaroorg
https://www.youtube.com/user/linaroorg?sub_confirmation=1
https://www.linkedin.com/company/1026961"
Kerberos is the system which underpins the vast majority of strong authentication across the Apache HBase/Hadoop application stack. Kerberos errors have brought many to their knees and it is often referred to as “black magic” or “the dark arts”; a long-standing joke that there are so few who understand how it works. This talk will cover the types of problems that Kerberos solves and doesn’t solve for HBase, decrypt some jargon on related libraries and technology that enable Kerberos authentication in HBase and Hadoop, and distill some basic takeaways designed to ease users in developing an application that can securely communicate with a “kerberized” HBase installation.
Open Source Backup Conference 2014: Bakup to and of the cloud, by Marco van W...NETWAYS
Newest developments in being able to use Bareos for backing up and restoring data of a Gluster or/and CEPH storage cluster. For this we use GFAPI for Gluster and librados/libcephfs (2 plugins) for CEPH. Next to that we also added support for
using Gluster and CEPH as a backing store for storing backup data using the native APIs not a FUSE mount (so without all the additional overhead.)
What do we support now and what is planned. (Currently some is internal development and some is as technology preview in our current bleeding edge code base.)
Under the Armor of Knights Corner: Intel MIC Architecture at Hotchips 2012Intel IT Center
George Chrysos, the leading architect of Intel Xeon Phi co-processor shared the new architecture details of upcoming Intel's HPC powerhouse. Designed for highly-parallel applications, Intel Xeon Phi co-processor based on Intel Mani Integrated Core architecture will deliver the combination of industry leading performance per watt with the ability to re-use the existing code and applications without necessity of re-writing them. Equipped with more than 50 cores and built using Intel's latest 22nm 3D Tri-gate transistor technology, new co-processors will be in production this year with first supercomputers from top500 list already taking advantage of this technology.
Augmenting Flow Operations and Feedback on the Model Driven MD_SAL Approach i...Brent Salisbury
Augmenting Flow Operations and Feedback on the Model Driven MD_SAL Approach in OpenDaylight. Will post more details in a blog entry at http://networkstatic.net as soon as time permits for those looking for more information.
Cheers,
-Brent
A real-time speech to text conversion system converts the spoken words into text . Speech-to-Text technology enables us to convert audio to text by applying powerful neural network models. It has a number of applications for users with and without disabilities. Speech-to-text has been used for voice search, help writers boost their productivity, and to provide alternate access to a computer for individuals with physical impairments. Other applications include speech recognition for foreign language learning, voice-activated products for the blind and many familiar mainstream technologies. It is a driving force behind the success of new age voice-controlled speakers like Amazon Echo and Google Home.
06 - Proposition d’Adoption de la Stratégie de Spécialisation Intelligente en...Mohamed Larbi BEN YOUNES
Proposition d’Adoption de la Stratégie de Spécialisation Intelligente en Tunisie / Adoption of a Strategy for Smart Specialisation in Tunisia
Ms. Sana MRIZAK, Télécom École de management, Évry, France
Séminaire sur la Stratégie de Spécialisation Intelligente / S3 organisé par l'ANPR avec le support de l'UE les 17 et 18 mai 2016 à Hammamet.
Défis pour les Régions à Faible Innovation / Challenges for Low Innovation Regions
Mr. Adrian HEALY, SmartSpec project, Cardiff University
Séminaire sur la Stratégie de Spécialisation Intelligente / S3 organisé par l'ANPR avec le support de l'UE les 17 et 18 mai 2016 à Hammamet.
How we plan to implement a Content Repository in pure PHP for use with TYPO3 5.0. Presented at 3rd International TYPO3 Conference 2007 in Karlsruhe, Germany.
Code for Startup MVP (Ruby on Rails) Session 1Henry S
First Session on Learning to Code for Startup MVP's using Ruby on Rails.
This session covers the web architecture, Git/GitHub and makes a real rails app that is deployed to Heroku at the end.
Thanks,
Henry
High Performance Enterprise Data Processing with Apache Spark with Sandeep Va...Spark Summit
Data engineering to support reporting and analytics for commercial Lifesciences groups consists of very complex interdependent processing with highly complex business rules (thousands of transformations on hundreds of data sources). We will talk about our experiences in building a very high performance data processing platform powered by Spark that balances the considerations of extreme performance, speed of development, and cost of maintenance. We will touch upon optimizing enterprise grade Spark architecture for data warehousing and data mart type applications, optimizing end to end pipelines for extreme performance, running hundreds of jobs in parallel in Spark, orchestrating across multiple Spark clusters, and some guidelines for high speed platform and application development within enterprises. Key takeaways: – example architecture for complex data warehousing and data mart applications on Spark – architecture to build high performance Spark platforms for enterprises that balance functionality with total cost of ownership – orchestrating multiple elastic Spark clusters while running hundreds of jobs in parallel – business benefits of high performance data engineering, especially for Lifesciences.
Las nuevas arquitecturas, servicios y micro-servicios web, aplicaciones y apps, Bots, IoT, AI, etc., que demandan las organizaciones, necesitan cada vez más del talento y experiencia de los Administradores de Bases de Datos para dar consejos, sugerencias y respuestas que aporten un valor diferencial a los grupos de desarrollo y usuarios de negocio.
Te mostramos las claves del nuevo rol del DBA, que complementa la “A” de Administrar con: Analizar, Aconsejar, Automatizar y crear Arquitecturas eficientes y Autónomas para la gestión Avanzada de datos, colaborando con los desarrolladores y usuarios desde un conocimiento profundo de las base de datos.
Building and deploying LLM applications with Apache AirflowKaxil Naik
Behind the growing interest in Generate AI and LLM-based enterprise applications lies an expanded set of requirements for data integrations and ML orchestration. Enterprises want to use proprietary data to power LLM-based applications that create new business value, but they face challenges in moving beyond experimentation. The pipelines that power these models need to run reliably at scale, bringing together data from many sources and reacting continuously to changing conditions.
This talk focuses on the design patterns for using Apache Airflow to support LLM applications created using private enterprise data. We’ll go through a real-world example of what this looks like, as well as a proposal to improve Airflow and to add additional Airflow Providers to make it easier to interact with LLMs such as the ones from OpenAI (such as GPT4) and the ones on HuggingFace, while working with both structured and unstructured data.
In short, this shows how these Airflow patterns enable reliable, traceable, and scalable LLM applications within the enterprise.
https://airflowsummit.org/sessions/2023/keynote-llm/
Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...Odinot Stanislas
Après la petite intro sur le stockage distribué et la description de Ceph, Jian Zhang réalise dans cette présentation quelques benchmarks intéressants : tests séquentiels, tests random et surtout comparaison des résultats avant et après optimisations. Les paramètres de configuration touchés et optimisations (Large page numbers, Omap data sur un disque séparé, ...) apportent au minimum 2x de perf en plus.
1. Intro to SRE role
2. SRE vs DevOps vs SDE
3. How to prepare for SRE interviews ?
4. What specific skills to acquire for working as a SRE ?
5. How should we start our career as SRE straight out of college?
6. Study materials that can help
Zach Shelby, Director of Technology for IoT at ARM and previously the co-founder of Sensinode gives and an in-depth tutrorial of the Constrained Application Protocol (CoAP) for the Internet of Things. Updates to this tutorial made on April 30th, 2014.
During the Neos Conference 2024 I talked – again – about updating Neos. Next to why, when and how you should be updating I talked about going to Neos 9…
There are many ways to start a new Neos-based project and there is a multitude of tools involved. These slides introduce a "perfect" way to set up a Neos project. A recording of the presentation is available at https://youtu.be/hWJSK0pE1H0
The presentation explains how multi-language websites can be built with Neos. But it doesn't stop there, since Neos allows for arbitrary content dimensions - be it for country-specific content, targeting different customer segments, personalization, …
This presentation was given during Inspiring Conference in Kolbermoor on March 28th, 2015.
FLOW3 spearheaded a move towards Git adoption within the TYPO3 project, and we are more pleased every day with the decision to turn away from Subversion and toward GIt.
In this session I explain the workflow we adopted using Git and the code review system Gerrit. I will show how it makes collaborative development more productive and improves code quality at the same time.
Ein Überblick über die Mittel zu Kommunikation und Kollaboration in der TYPO3 Community. Vorgestellt beim 15. "Wissensmanagement Stammtisch Braunschweig" im Mai 2007.
Introduction to what Unicode support in PHP6 means and how it will change the way PHP developers work. Presented at the 3rd International TYPO3 Conference 2007 in Karlsruhe, Germany.
Introduction to the persistent layer implemented for FLOW3. The presentation was held on the 4th International TYPO3 Conference in Berlin, October 2008.
Introduction to Test-Driven Development and why using Dependency Injection as provided by FLOW3 makes it an even better development approach. Presented at the 4th International TYPO3 Conference 2008 in Berlin.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
3. What is a Content Repository
A Content Repository (CR) allows the storage and retrieval of
arbitrary content as nodes and properties in a tree structure
The node types as well as the tree structure can be freely defined
by the user of the CR
Binary content can stored and queried as effectively as textual
content
The API for using a CR is standardized in the JSR-170 and JSR-283
specifications
The API abstracts the actual data storage used (RDBMS, ODBMS,
files, ...)
Inspiring people to
share
5. Reasons for using a CR
Flexible and extensible data structure
Object based storage and retrieval
Combines advantages of navigational and relational databases
Security can be enforced on a higher level
Cleaner and easier to use for the developer
No vendor lock-in
Powerful data storage for a truckload of applications
Inspiring people to
share
6. Existing implementations
Jackrabbit is the reference implementation, available as open
source from the Apache Foundation
Day CRX is the commercial CR implementation from the quot;inventorquot;
of JSR-170, Day Software
Other implementations are eXo JCR and Jeceira, the latter also
being dead, and others
JSR-170 connectors exist for Alfresco, BEA Portal Server, IBM
Domino and others
Inspiring people to
share
7. PHP ports of the JSR-170 API
Travis Swicegood ported the API to PHP in 2005 - project seems
dead
There is a port of the API available in the Jackrabbit sources,
added 2005 - no relevant changes since then
• This could be a sign of a well-done port of the API, but it
rather seems to be a lack of usage and feedback
No JSR-283 port of the API today
• Well, since the specification isn’t final yet, this may not come
as a surprise...
Inspiring people to
share
8. TYPO3 Content Repository
No standalone PHP implementation of a Content Repository exists
as of now
Our goal is a pure PHP implementation of JSR-283, but
functionality needed for the TYPO3 CMS has priority
Will take advantage of the TYPO3 Framework, but not be tied to
the TYPO3 CMS.
Could eventually become the standard CR for the PHP
community?!
• This depends also on you, the PHP developers out there...
Inspiring people to
share
9. Summary
A Content Repository (CR) promises to solve a lot of the problems
(not only) vendors of CMS currently have
A stable standard with a fresh version in the making
Various implementations exist, mostly in Java
No standalone PHP implementation of a Content Repository exists
We want to change that!
Inspiring people to
share
11. TYPO3 Framework
Provides a robust and advanced programming framework with
features like Dependency Injection, Aspect Oriented Programming,
MVC, Component and Package Management, enhanced Reflection
and more
Inspired by the most popular frameworks and toolkits from
Smalltalk, Python, Ruby and Java available today, picking the
best concepts, skipping the annoyances
Has already come a long way, check out the session by Robert
Lemke today at 17:30 in Rhein-Main I!
Not tied to TYPO3 CMS, can be used for any PHP6-based project
Inspiring people to
share
12. PHP 6
PHP 6 will be used for development of all projects around TYPO3
5.0
Currently unclear when PHP 6 will be released
PHP 5.3 = PHP 6 - Unicode?
• This could mean our code runs on PHP 5.3...
• ... but will lack Unicode support
Still needs to be investigated
Inspiring people to
share
13. Development model
A few acronyms for your reading pleasure:
• DDD – Domain Driven Design
• TDD – Test Driven Development
• CI – Continuous Integration
• AOP – Aspect Oriented Programming
We follow these principles and use those paradigms not because
they sound good – but because they improve design and code
Open Source, see http://typo3.org/gimmefive/ for more
Inspiring people to
share
14. Summary
The TYPO3 Framework serves as a solid base for development
PHP 6 will be a prerequisite for using the TYPO3 CR
• Following PHP roadmap changes we will evaluate possible
escape routes
Using state-of-the-art development principles
Inspiring people to
share
16. Porting the JSR-283 API
The API needs to be ported before writing the code
Interfaces will not be ported up-front, but as we need them
Facing typing issues, some Java types simply do not exist in PHP
Binary data will probably be Resource Manager handles instead of
streams
• Alternatively simple stream handlers, still needs research
Features prioritized according to the needs of the TYPO3 CMS
project
Inspiring people to
share
17. Actual data storage
The underlying storage of the TYPO3CR will be a RDBMS for the
start
Currently only SQLite through PDO is used
• Easy to use for development and unit testing
• The use of PDO already enables any PDO-supported database
Specialized DB connectors will follow, using optimized queries,
stored procedures, ...
• This also opens the door for implementing other connectors,
like to LDAP or object databases or ...
Inspiring people to
share
18. Data storage techniques
Basically we need to store a simple tree
Read access must be fast, as the majority of requests are read
requests, write access should be fast
Traditional approach as used – not only – in TYPO3 today is to
store a triplet (uid, pid, sorting)
Alternative & faster method: Pre/Post Plane Encoding
Inspiring people to
share
19. Pre/Post Plane Encoding
Stores number determined by pre-order and post-order tree
traversal
Allows to partition the nodes into
four regions, as shown for node ƒ
Very fast read access, e.g. a single
SELECT to query all ancestors to
a node ƒ
SELECT * FROM table WHERE
pre < ƒ.pre AND post > ƒ.post
Inspiring people to
share
20. Performance considerations
Nodes can already be fetched very fast
• But attributes must be stored seperately, to keep node table
sizes inside sensible limits
• Binary data will be stored in the underlying database for
easier replication – a transparent cache to the filesystem will
be added
Write access can be sped up by various approaches like spacing
and variable length indices for the pre/post numbers or by
partitioning the data over more tables
Inspiring people to
share
21. Querying the TYPO3 CR
Using getRootNode() and friends from the API
Using XPath queries
Using SQL queries
Inspiring people to
share
22. API usage for querying
$repository = $this->componentManager->getComponent
('T3_phpCR_RepositoryInterface');
$session = $repository->login();
$newsEntries = $session->getNodeByUUID
('96bca35d-1ef5-4a47-8b0c-0bfc79507d08')->getNodes();
foreach($newsEntries as $newsEntry) {
$title = $newsEntry->getProperty('title');
$text = $newsEntry->getProperty('text');
// more code here...
}
Inspiring people to
share
23. XPath support for TYPO3R
To enable XPath we need
• a XPath parser
• an efficient way to transform a XPath query into SQL for the
used low-level data structure
The latter is a lot easier when storing the tree pre/post plane
encoded
Inspiring people to
share
24. SQL support for TYPO3R
Using SQL we need
• a (simple) SQL parser
• an efficient way to transform that SQL into equivalent SQL for
the used low-level data structure
This still needs to be investigated, possible approaches include
storing a reference to the parent node or even using the pre/post
plane only as a cache for XPath read queries, optimizing the
native storage for SQL read queries
Inspiring people to
share
25. Extensions to JSR-283
A vendor may choose to offer additional features in his CR
implementation
The TYPO3CR will offer support for
• Data persistency through code annotations
• Automatic node type generation based on class members
• Rules for setting up virtual root nodes based on node types
Inspiring people to
share
26. Current status
Currently the code supports a subset of the required features of
levels 1 & 2 and the optional parts of the JSR-283 specification
• Basic read & write access
• Namespace registration
• Node type discovery and registration
Data storage uses the naive approach known from TYPO3 4.x
Inspiring people to
share
27. Future plans
Implementing missing required and optional features according
to the JSR-283 specification
• Workspaces and versioning will require special attention, as
they are key features for the CMS as well!
Implement native RDBMS connectors making use of specifically
tuned queries and product-specific features
Performance tests and tuning as needed
Inspiring people to
share
28. Summary
Implementing the specification is not an easy task, but doable
For the various parts a lot of research has already been done in
the past
The repository is a major improvement over the current way of
storing data
The whole PHP community could^Wcan^Wwill benefit!
Inspiring people to
share