Calsoft presented following papers at SNIA Storage Developer Conference (SDC), Santa Clara, CA, USA, 2012 –
Cloud File System and Cloud Data Management Interface (CDMI) - The cloud file system presented and implemented by Calsoft integrates with many cloud storage providers using Cloud Data Management Interface(CDMI). This helps enterprises store file based data to cloud storage and provides throughput similar to local NAS by using efficient caching techniques.
A brief overview of caching mechanisms in a web application. Taking a look at the different layers of caching and how to utilize them in a PHP code base. We also compare Redis and MemCached discussing their advantages and disadvantages.
Apache Flume is a simple yet robust data collection and aggregation framework which allows easy declarative configuration of components to pipeline data from upstream source to backend services such as Hadoop HDFS, HBase and others.
In just a few years, Open Policy Agent (OPA) has emerged as one of the hotter technologies for policy management and fine grained access control in the cloud native ecosystem. Now it’s coming for your APIs!
In this session we will explore the underlying concepts and some of the components involved in OPA before we get hands on in live coding test driven authorization policies to protect API endpoints.
ApacheCon 2022: From Column-Level to Cell-Level_ Towards Finer-grained Encryp...XinliShang1
This talk is about Apache Parquet cell-level encryption feature. It allows encryption can happen at the cell(intersection of column and row) level, which is finer-grained than the column level.
Exactly-once Stream Processing with Kafka StreamsGuozhang Wang
I will present the recent additions to Kafka to achieve exactly-once semantics (0.11.0) within its Streams API for stream processing use cases. This is achieved by leveraging the underlying idempotent and transactional client features. The main focus will be the specific semantics that Kafka distributed transactions enable in Streams and the underlying mechanics to let Streams scale efficiently.
A brief overview of caching mechanisms in a web application. Taking a look at the different layers of caching and how to utilize them in a PHP code base. We also compare Redis and MemCached discussing their advantages and disadvantages.
Apache Flume is a simple yet robust data collection and aggregation framework which allows easy declarative configuration of components to pipeline data from upstream source to backend services such as Hadoop HDFS, HBase and others.
In just a few years, Open Policy Agent (OPA) has emerged as one of the hotter technologies for policy management and fine grained access control in the cloud native ecosystem. Now it’s coming for your APIs!
In this session we will explore the underlying concepts and some of the components involved in OPA before we get hands on in live coding test driven authorization policies to protect API endpoints.
ApacheCon 2022: From Column-Level to Cell-Level_ Towards Finer-grained Encryp...XinliShang1
This talk is about Apache Parquet cell-level encryption feature. It allows encryption can happen at the cell(intersection of column and row) level, which is finer-grained than the column level.
Exactly-once Stream Processing with Kafka StreamsGuozhang Wang
I will present the recent additions to Kafka to achieve exactly-once semantics (0.11.0) within its Streams API for stream processing use cases. This is achieved by leveraging the underlying idempotent and transactional client features. The main focus will be the specific semantics that Kafka distributed transactions enable in Streams and the underlying mechanics to let Streams scale efficiently.
Supporting Over a Thousand Custom Hive User Defined FunctionsDatabricks
Over the years, Facebook has used Hive as the primary query engine to be used by our data engineers. Since Hive uses SQL-like query language called HQL, the list of built-in User Defined Functions (UDFs) did not always satisfy our customer requirements and as a result, an extensive list of custom UDFs was developed over time. As we started migrating pipelines from Hive to Spark SQL, a number of custom UDFs appeared incompatible with Spark, and many others showed bad performance. In this talk will first take a deep dive into how Hive UDFs work with Spark. We will then share what challenges we overcame on the way to support 99.99% of the custom UDFs in Spark.
Speakers: Sergey Makagonov, Xin Yao
Complex architectures for authentication and authorization on AWSBoyan Dimitrov
In this talk we discuss key architecture patterns for designing authentication and authorization solutions in complex microservices environments. We focus on the key advantages and capabilities of AWS Cognito User Pools and Federated Identities and explore how this service can address the challenges of implementing client to service, service to service and service to infrastructure auth.
In addition, we discuss patterns and best practices around building a highly available and resilient decentralised authorization solution for microservices environments based on OIDC. We present a simple RBAC implementation together with fine-grained permissions and end to end automation.
Data Security at Scale through Spark and Parquet EncryptionDatabricks
Big data presents new challenges for protection of privacy and integrity of sensitive information. Straightforward application of traditional file encryption and MAC techniques can’t cope with staggering volumes of data, flowing in modern analytic pipelines.
Apple addresses these challenges by leveraging the new capabilities in the Apache Parquet format. We work with the Apache Parquet community on a modular data security mechanism, that provides privacy and integrity guarantees for sensitive information at scale; the encryption specification has been approved and released by the Apache Parquet Format project. Today, there are two open source implementations of this specification – in Apache Arrow (C++) and in Apache Parquet-MR (Java) repositories. The latter has just been released in the parquet-mr-1.12 version – which means the Apache Spark and other Java/Scala based analytic frameworks can start working with Apache Parquet encryption.
In this talk, Gidon Gershinsky and Tim Perelmutov will outline the challenges of protecting the privacy of data at scale and describe the Apache Parquet encryption technology security approach. We will give a quick intro to usage of Apache Parquet encryption API in pure Java and in Apache Spark applications. We will also discuss the roadmap of the community work on new encryption features and on deeper integration with Apache Spark and other analytic frameworks. Finally, we will show a demo of the Apache Parquet modular encryption in action, sharing our learnings using it at scale.
This presentation discusses the following topics:
Hadoop Distributed File System (HDFS)
How does HDFS work?
HDFS Architecture
Features of HDFS
Benefits of using HDFS
Examples: Target Marketing
HDFS data replication
Troubleshooting Kerberos in Hadoop: Taming the BeastDataWorks Summit
Kerberos is the ubiquitous authentication mechanism when it comes to secure any Hadoop Services. With recent updates in Hadoop core and various Apache Hadoop components, inherent Kerberos support has matured and has come a long way.
Understanding & configuring Kerberos is still a challenge but even more painful & frustrating is troubleshooting a Kerberos issue. There are lot of things (small & big) that can go wrong (and will go wrong!). This talk covers the Kerberos debugging part in detail and discusses the tools & tricks that can be used to narrow down any Kerberos issue.
Rather than discussing the issues and their resolution, we will focus on how to approach a Kerberos problem and do's / dont's in Kerberos scene. This talk will provide a step by step guide that will equip the audience for troubleshooting future Kerberos problems.
Agenda is to discuss:
- Systematic approach to Kerberos troubleshooting
- Kerberos Tools available in Hadoop arsenal
- Tips & Tricks to narrow down Kerberos issues quickly
- Some nasty Kerberos issues from Support trenches
Some prior knowledge on Kerberos basics will be appreciated but is not a prerequisite.
Speaker:
Vipin Rathor, Sr. Product Specialist (HDP Security), Hortonworks
Orchestrating workflows Apache Airflow on GCP & AWSDerrick Qin
Working in a cloud or on-premises environment, we all somehow move data from A to B on-demand or on schedule. It is essential to have a tool that can automate recurring workflows. This can be anything from an ETL(Extract, Transform, and Load) job for a regular analytics report all the way to automatically re-training a machine learning model.
In this talk, we will introduce Apache Airflow and how it can help orchestrate your workflows. We will cover key concepts, features, and use cases of Apache Airflow, as well as how you can enjoy Apache Airflow on GCP and AWS by demo-ing a few practical workflows.
Mario Molina, Software Engineer
CDC systems are usually used to identify changes in data sources, capture and replicate those changes to other systems. Companies are using CDC to sync data across systems, cloud migration or even applying stream processing, among others.
In this presentation we’ll see CDC patterns, how to use it in Apache Kafka, and do a live demo!
https://www.meetup.com/Mexico-Kafka/events/277309497/
Organizations need to perform increasingly complex analysis on data — streaming analytics, ad-hoc querying, and predictive analytics — in order to get better customer insights and actionable business intelligence. Apache Spark has recently emerged as the framework of choice to address many of these challenges. In this session, we show you how to use Apache Spark on AWS to implement and scale common big data use cases such as real-time data processing, interactive data science, predictive analytics, and more. We will talk about common architectures, best practices to quickly create Spark clusters using Amazon EMR, and ways to integrate Spark with other big data services in AWS.
Learning Objectives:
• Learn why Spark is great for ad-hoc interactive analysis and real-time stream processing.
• How to deploy and tune scalable clusters running Spark on Amazon EMR.
• How to use EMR File System (EMRFS) with Spark to query data directly in Amazon S3.
• Common architectures to leverage Spark with Amazon DynamoDB, Amazon Redshift, Amazon Kinesis, and more.
Kafka for Real-Time Replication between Edge and Hybrid CloudKai Wähner
Not all workloads allow cloud computing. Low latency, cybersecurity, and cost-efficiency require a suitable combination of edge computing and cloud integration.
This session explores architectures and design patterns for software and hardware considerations to deploy hybrid data streaming with Apache Kafka anywhere. A live demo shows data synchronization from the edge to the public cloud across continents with Kafka on Hivecell and Confluent Cloud.
EKS에서 Opentelemetry로 코드실행 모니터링하기 - 신재현 (인덴트코퍼레이션) :: AWS Community Day Online...AWSKRUG - AWS한국사용자모임
EKS환경에서 Opentelemetry와 Jaeger를 활용하여 서버의 코드가 잘 동작하는지 어떤로직에서 문제가 발생했는지 모니터링 하는 방법을 알아봅니다. 마지막으로 Grafana를 이용해 쉽게 원하는 코드를 조회 하는 방법도 실습해볼 예정입니다. K8S를 모르셔도 참석할 수 있습니다.
Guaranteed Event Delivery with Kafka and NodeJS | Amitesh Madhur, NutanixHostedbyConfluent
The business systems of an organization are a continuous source of events. Each system also needs to know about events happening in the other systems. Exchanging these events through direct API calls creates a web of inter-dependencies, is fragile and fails to scale. We examine how this problem can be solved through the use of right integration patterns implemented as a light-weight event hub that leverages the power of Kafka and Confluent to operate at enterprise scale. We demonstrate how JavaScript with its event-driven programming model can be a good fit for implementing an event hub that ensures guaranteed message delivery in the face of failures within the individual subscriber systems.
Many organizations having large engineering teams skilled in NodeJS and a multitude of NodeJs applications. We show how these teams can easily leverage the power of Kafka and scale their applications with the right architectural building blocks. We also offer insights from our own experience of building NodeJS based Kafka applications.
Video: https://data-artisans.com/flink-forward-berlin/resources/monitoring-flink-with-prometheus
Live Demo Code: https://github.com/mbode/flink-prometheus-example
Prometheus is a cloud-native monitoring system prioritizing reliability and simplicity – and Flink works really well with it! This session will show you how to leverage the Flink metrics system together with Pronetheus to improve the observability of your jobs. There will be a live demo establishing how everything ties in together. The talk is aimed at people already building and running Flink jobs who would like to gain more insight into them. It is fine if you are not familiar with Prometheus yet as the basic concepts will be introduced. If you have ever wondered how you could use modern monitoring tools to be alerted in the middle of the night in case your Flink job‘s 99th percentile end-to-end latency degraded for some reason, this might just be the talk you are looking for.
Supporting Over a Thousand Custom Hive User Defined FunctionsDatabricks
Over the years, Facebook has used Hive as the primary query engine to be used by our data engineers. Since Hive uses SQL-like query language called HQL, the list of built-in User Defined Functions (UDFs) did not always satisfy our customer requirements and as a result, an extensive list of custom UDFs was developed over time. As we started migrating pipelines from Hive to Spark SQL, a number of custom UDFs appeared incompatible with Spark, and many others showed bad performance. In this talk will first take a deep dive into how Hive UDFs work with Spark. We will then share what challenges we overcame on the way to support 99.99% of the custom UDFs in Spark.
Speakers: Sergey Makagonov, Xin Yao
Complex architectures for authentication and authorization on AWSBoyan Dimitrov
In this talk we discuss key architecture patterns for designing authentication and authorization solutions in complex microservices environments. We focus on the key advantages and capabilities of AWS Cognito User Pools and Federated Identities and explore how this service can address the challenges of implementing client to service, service to service and service to infrastructure auth.
In addition, we discuss patterns and best practices around building a highly available and resilient decentralised authorization solution for microservices environments based on OIDC. We present a simple RBAC implementation together with fine-grained permissions and end to end automation.
Data Security at Scale through Spark and Parquet EncryptionDatabricks
Big data presents new challenges for protection of privacy and integrity of sensitive information. Straightforward application of traditional file encryption and MAC techniques can’t cope with staggering volumes of data, flowing in modern analytic pipelines.
Apple addresses these challenges by leveraging the new capabilities in the Apache Parquet format. We work with the Apache Parquet community on a modular data security mechanism, that provides privacy and integrity guarantees for sensitive information at scale; the encryption specification has been approved and released by the Apache Parquet Format project. Today, there are two open source implementations of this specification – in Apache Arrow (C++) and in Apache Parquet-MR (Java) repositories. The latter has just been released in the parquet-mr-1.12 version – which means the Apache Spark and other Java/Scala based analytic frameworks can start working with Apache Parquet encryption.
In this talk, Gidon Gershinsky and Tim Perelmutov will outline the challenges of protecting the privacy of data at scale and describe the Apache Parquet encryption technology security approach. We will give a quick intro to usage of Apache Parquet encryption API in pure Java and in Apache Spark applications. We will also discuss the roadmap of the community work on new encryption features and on deeper integration with Apache Spark and other analytic frameworks. Finally, we will show a demo of the Apache Parquet modular encryption in action, sharing our learnings using it at scale.
This presentation discusses the following topics:
Hadoop Distributed File System (HDFS)
How does HDFS work?
HDFS Architecture
Features of HDFS
Benefits of using HDFS
Examples: Target Marketing
HDFS data replication
Troubleshooting Kerberos in Hadoop: Taming the BeastDataWorks Summit
Kerberos is the ubiquitous authentication mechanism when it comes to secure any Hadoop Services. With recent updates in Hadoop core and various Apache Hadoop components, inherent Kerberos support has matured and has come a long way.
Understanding & configuring Kerberos is still a challenge but even more painful & frustrating is troubleshooting a Kerberos issue. There are lot of things (small & big) that can go wrong (and will go wrong!). This talk covers the Kerberos debugging part in detail and discusses the tools & tricks that can be used to narrow down any Kerberos issue.
Rather than discussing the issues and their resolution, we will focus on how to approach a Kerberos problem and do's / dont's in Kerberos scene. This talk will provide a step by step guide that will equip the audience for troubleshooting future Kerberos problems.
Agenda is to discuss:
- Systematic approach to Kerberos troubleshooting
- Kerberos Tools available in Hadoop arsenal
- Tips & Tricks to narrow down Kerberos issues quickly
- Some nasty Kerberos issues from Support trenches
Some prior knowledge on Kerberos basics will be appreciated but is not a prerequisite.
Speaker:
Vipin Rathor, Sr. Product Specialist (HDP Security), Hortonworks
Orchestrating workflows Apache Airflow on GCP & AWSDerrick Qin
Working in a cloud or on-premises environment, we all somehow move data from A to B on-demand or on schedule. It is essential to have a tool that can automate recurring workflows. This can be anything from an ETL(Extract, Transform, and Load) job for a regular analytics report all the way to automatically re-training a machine learning model.
In this talk, we will introduce Apache Airflow and how it can help orchestrate your workflows. We will cover key concepts, features, and use cases of Apache Airflow, as well as how you can enjoy Apache Airflow on GCP and AWS by demo-ing a few practical workflows.
Mario Molina, Software Engineer
CDC systems are usually used to identify changes in data sources, capture and replicate those changes to other systems. Companies are using CDC to sync data across systems, cloud migration or even applying stream processing, among others.
In this presentation we’ll see CDC patterns, how to use it in Apache Kafka, and do a live demo!
https://www.meetup.com/Mexico-Kafka/events/277309497/
Organizations need to perform increasingly complex analysis on data — streaming analytics, ad-hoc querying, and predictive analytics — in order to get better customer insights and actionable business intelligence. Apache Spark has recently emerged as the framework of choice to address many of these challenges. In this session, we show you how to use Apache Spark on AWS to implement and scale common big data use cases such as real-time data processing, interactive data science, predictive analytics, and more. We will talk about common architectures, best practices to quickly create Spark clusters using Amazon EMR, and ways to integrate Spark with other big data services in AWS.
Learning Objectives:
• Learn why Spark is great for ad-hoc interactive analysis and real-time stream processing.
• How to deploy and tune scalable clusters running Spark on Amazon EMR.
• How to use EMR File System (EMRFS) with Spark to query data directly in Amazon S3.
• Common architectures to leverage Spark with Amazon DynamoDB, Amazon Redshift, Amazon Kinesis, and more.
Kafka for Real-Time Replication between Edge and Hybrid CloudKai Wähner
Not all workloads allow cloud computing. Low latency, cybersecurity, and cost-efficiency require a suitable combination of edge computing and cloud integration.
This session explores architectures and design patterns for software and hardware considerations to deploy hybrid data streaming with Apache Kafka anywhere. A live demo shows data synchronization from the edge to the public cloud across continents with Kafka on Hivecell and Confluent Cloud.
EKS에서 Opentelemetry로 코드실행 모니터링하기 - 신재현 (인덴트코퍼레이션) :: AWS Community Day Online...AWSKRUG - AWS한국사용자모임
EKS환경에서 Opentelemetry와 Jaeger를 활용하여 서버의 코드가 잘 동작하는지 어떤로직에서 문제가 발생했는지 모니터링 하는 방법을 알아봅니다. 마지막으로 Grafana를 이용해 쉽게 원하는 코드를 조회 하는 방법도 실습해볼 예정입니다. K8S를 모르셔도 참석할 수 있습니다.
Guaranteed Event Delivery with Kafka and NodeJS | Amitesh Madhur, NutanixHostedbyConfluent
The business systems of an organization are a continuous source of events. Each system also needs to know about events happening in the other systems. Exchanging these events through direct API calls creates a web of inter-dependencies, is fragile and fails to scale. We examine how this problem can be solved through the use of right integration patterns implemented as a light-weight event hub that leverages the power of Kafka and Confluent to operate at enterprise scale. We demonstrate how JavaScript with its event-driven programming model can be a good fit for implementing an event hub that ensures guaranteed message delivery in the face of failures within the individual subscriber systems.
Many organizations having large engineering teams skilled in NodeJS and a multitude of NodeJs applications. We show how these teams can easily leverage the power of Kafka and scale their applications with the right architectural building blocks. We also offer insights from our own experience of building NodeJS based Kafka applications.
Video: https://data-artisans.com/flink-forward-berlin/resources/monitoring-flink-with-prometheus
Live Demo Code: https://github.com/mbode/flink-prometheus-example
Prometheus is a cloud-native monitoring system prioritizing reliability and simplicity – and Flink works really well with it! This session will show you how to leverage the Flink metrics system together with Pronetheus to improve the observability of your jobs. There will be a live demo establishing how everything ties in together. The talk is aimed at people already building and running Flink jobs who would like to gain more insight into them. It is fine if you are not familiar with Prometheus yet as the basic concepts will be introduced. If you have ever wondered how you could use modern monitoring tools to be alerted in the middle of the night in case your Flink job‘s 99th percentile end-to-end latency degraded for some reason, this might just be the talk you are looking for.
Virtualization is the creation of a virtual (rather than actual) version of something, such as an operating system, a server, a storage device or network resources.
Class lecture by Prof. Raj Jain on Storage Virtualization. The talk covers Disk Arrays, Data Access Methods, SCSI (Small Computer System Interface), Advanced Technology Attachment (ATA), ESCON and FICON, Fibre Chanel, Fibre Channel Devices, Fibre Channel Protocol Layers, Fibre Channel Flow Control, Fibre Channel Classes of Service, What is Storage Virtualization?, Benefits of Storage Virtualization, Virtualizing Storage, RAID Levels, Nested RAIDs, Synchronous vs. Asynchronous Replication, Virtual Storage Area Network (VSAN), Physical Storage Network, Virtual Storage Network, SAN vs. NAS, iSCSI (Internet Small Computer System Interface), iFCP (Internet Fiber Channel Protocol), FCIP (Fibre Channel over IP), FCoE (Fibre Channel over Ethernet), Virtual File Systems. Video recording available in YouTube.
Leo Zhadanovsky, Senior Solutions Architect at Amazon Web Services, shows how to run content management systems such as Drupal, WordPress and Jekyll on Amazon Web Services in a way that is scalable, highly-available and economical.
The slides feature how to architect websites in the cloud so they are secure and allow for rapid iteration and change without downtime.
Hybrid clouds are quickly forming on the horizon, and they are transforming the way that organizations do business. Join David Butler, SVP of Marketing at Eucalyptus, Judith Hurwitz, President and CEO, and Marcia Kaufman, COO and partner, of Hurwitz & Associates and co-authors of “Hybrid Cloud For Dummies” to learn what this new cloud deployment model is all about.
Hybrid clouds are quickly forming on the horizon, and they are transforming the way that organizations do business. Join David Butler, SVP of Marketing at Eucalyptus, Judith Hurwitz, President and CEO, and Marcia Kaufman, COO and partner, of Hurwitz & Associates and co-authors of “Hybrid Cloud For Dummiesto learn what this new cloud deployment model is all about.
Cloud management is an immature space with very few industry standards guiding the selection, implementation and use of Cloud management tools in support of complex Cloud environments.
Cloud service providers enable consumers to create complex computing environments by leveraging emerging Cloud orchestration and management tools that support the concept of "templates," or pre-defined Cloud management profiles. Under this approach, pre-built, pre-tested profiles are managed in a catalog, which can contain tens to hundreds of pre-defined IT templates, with the ability to customize them to meet specific business needs.
Cloud service providers are starting to appreciate the complex Cloud management requirements associated with implementing, or configuring and renting a Cloud computing environment for mission critical business requirements. The maturation of Cloud management profiles, or "templates," is essential for accessing and consuming Cloud computing capabilities from your Cloud service providers in a cost-effective and governance-aligned fashion.
Interestingly, these Cloud templates make it easy, in some respects too easy, to create or access Cloud computing environments optimized for a specific business user's needs. Hundreds or even thousands of "optimized" VMs stand at the ready for easy access, provisioning and consumption. But just like life in a gold rush town, where lawlessness and greed prevailed, we will eventually reach a critical juncture where self-interests, stovepipe behaviors, lack of standardization, and refusal to share Cloud resources can no longer be sustained.
Cloud governance and Cloud management are essential for implementing and managing an optimized Cloud computing environment, regardless of whether you implement private, public or hybrid Cloud models.
During this upcoming webinar we will discuss:
• The relationship of Cloud governance and Cloud management frameworks, and the associated concepts and terminology associated with these disciplines
• The current state of cloud management approaches, tooling and sandbox frameworks, and how templates are essential to enable orchestration and management of highly-virtualized Cloud computing environments
• Relevant industry standards that enable Cloud management to be realized leveraging Cloud management profiles or "templates"
• A conceptual architecture for selecting, implementing and operationalizing Cloud management in support of your Cloud strategy
• The critical need for companies to couple a strong Cloud governance model and body of Cloud policies with Cloud management frameworks and the associated tools
Cloud governance and Cloud management frameworks are critical to successful enterprise adoption of Cloud computing capabilities. This webinar will provide the foundation for implementing Cloud governance and management, and help you manage your success with Cloud computing.
Cloud computing is the next evolution of systems architecture that promises to deliver computing infrastructure
at lower cost, greater flexibility and significantly higher levels of scalability than ever before. The benefits of
cloud computing are primarily built on the notion that cloud environments allow you to achieve these results
by virtualizing the different work-loads that you run, and deploying them within an environment that can
automatically scale up, or down, as work-load demands change over ti me.
This presentation will discuss concerns and opportunities, business drivers, cisco solutions for enterprise cloud, the compute platform, cisco automation solutions and cloud enablement workshop.
Cloud-based systems have brought a new, scalable application
delivery service model to the market. They can help clients
save both capital and operational costs, and further shift capital
expenses (CAPEX) to operational expenses (OPEX) while
providing increased flexibility for their organizations. However,
a key aspect of any type of Cloud deployment is reliable service
delivery, on which ready access to the digital information (data)
in the Cloud is foundational requirement.
Cloud Storage Adoption, Practice, and DeploymentGlusterFS
In this webinar, leading storage analyst firm Storage Strategies NOW, will discuss the findings from their comprehensive outlook report on the state of the cloud storage market and storage services that are layered on top of it. We will review: the definition of cloud storage, requirements, deployment, the market and its trends, API’s, cloud computing initiatives, best practices and infrastructure providers. Tom Trainer, Director of Product Marketing at Gluster, will provide an overview of Gluster’s storage products along with case studies demonstrating the strategic deployment of Gluster storage in both the public and private cloud.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.