SpringOne Platform 2017
Christian Tzolov, Pivotal
"When working with BigData & IoT systems we often feel the need for an established, Common Query Language.
To fill this gap some NoSql vendors are building SQL access to their systems. Building SQL engine from scratch is a daunting job and frameworks like Apache Calcite can help you with the heavy lifting. It allows you to integrate SQL parser, Cost-Based Optimizer, and JDBC with your NoSql system. Calcite has been used to empower many BigData platforms such as Hive, Spark, Flink, Drill, HBase/Phoenix to name some.
In this session I will walk you through the process of building a SQL access layer for Apache Geode (GemFire). I will share my experience, pitfalls and technical consideration like balancing between the SQL/RDBMS semantics and the design choices and limitations of In-Memory-Data-Grid systems like Geode.
Hopefully this will enable you to add SQL capabilities to your preferred NoSQL data system."
Spring boot microservice metrics monitoringOracle Korea
This document summarizes a presentation on monitoring microservices with Spring Boot. It discusses evolving architectures from monolithic to microservices and challenges in microservices. It then covers different monitoring techniques like metrics, tracing and logging. It provides an overview of tools like Prometheus, Grafana, Spring Boot Admin, Eureka and Consul for monitoring microservices. Finally, it outlines hands-on labs to set up monitoring of a sample application with different tool combinations.
This document discusses various ways to interface with Oracle Cloud Infrastructure (OCI) Object Storage using different tools and SDKs. It covers using the OCI CLI to manage buckets and upload/download files, using Java and Python SDKs to programmatically interact with Object Storage, integrating Object Storage with Hadoop via the HDFS Connector, and using Object Storage as a data source for services like Oracle Autonomous Data Warehouse. The goal is to provide a common interface for data via virtualization regardless of where the data is physically stored.
The document discusses Oracle NoSQL Database and its features. It provides an overview of NoSQL databases and data models in Oracle NoSQL including key-value, table, and JSON. It also describes Oracle NoSQL's architecture, which uses automatic data sharding and replication across storage nodes for high availability and scalability. Configuration and usage is simplified with libraries and command line tools.
This document discusses Redis, MongoDB, and Amazon DynamoDB. It begins with an overview of NoSQL databases and the differences between SQL and NoSQL databases. It then covers Redis data types like strings, hashes, lists, sets, sorted sets, and streams. Examples use cases for Redis are also provided like leaderboards, geospatial queries, and message queues. The document also discusses MongoDB design patterns like embedding data, embracing duplication, and relationships. Finally, it provides a high-level overview of DynamoDB concepts like tables, items, attributes, and primary keys.
Connecting your .Net Applications to NoSQL Databases - MongoDB & CassandraLohith Goudagere Nagaraj
The document discusses various ways to connect .NET applications to NoSQL databases like MongoDB and Cassandra. It covers client SDK APIs, REST/SOAP APIs, and SQL-based connectivity options. For SQL connectivity, the document explains that Progress DataDirect drivers normalize the NoSQL data model to expose it through SQL. Examples demonstrate connecting to MongoDB and Cassandra using the MongoDB and Cassandra .NET drivers, their REST APIs, and Progress DataDirect's ODBC drivers with SQL. The document concludes that SQL connectivity requires data normalization but offers familiar skills and easy BI integration.
Apache Ignite is an integrated and distributed In-Memory Data Fabric for computing and transacting on large-scale data sets in real-time, orders of magnitude faster than possible with traditional disk-based or flash technologies. It is designed to easily power both existing and new applications in a distributed, massively parallel architecture on affordable, industry-standard hardware. Apache Ignite addresses today's Fast Data and Big Data needs by providing a comprehensive in-memory data fabric, which includes a data grid with SQL and transactional capabilities, in-memory streaming, an in-memory file system, and more.
Reactive Java EE - Let Me Count the Ways!Reza Rahman
As our industry matures there are pockets of increased demand for high-throughput, low-latency systems heavily utilizing event-driven programming and asynchronous processing. This trend is gradually converging on the somewhat well established but so-far not well understood term "Reactive".
This session explores how vanilla Java SE and Java EE aligns with this movement via features and APIs like JMS, MDB, EJB @Asynchronous, JAX-RS/Servlet/WebSocket async, CDI events, Java EE concurrency utilities and so on. We will also see how these robust facilities can be made digestible even in the most complex cases for mere mortal developers through Java SE 8 Lambdas and Completable Futures.
JDBC Next: A New Asynchronous API for Connecting to a Database Yolande Poirier
This new API is completely nonblocking. It is not intended to be an extension to, or a replacement for, JDBC but, rather, an entirely separate API that provides completely nonblocking access to the same databases as JDBC.
Spring boot microservice metrics monitoringOracle Korea
This document summarizes a presentation on monitoring microservices with Spring Boot. It discusses evolving architectures from monolithic to microservices and challenges in microservices. It then covers different monitoring techniques like metrics, tracing and logging. It provides an overview of tools like Prometheus, Grafana, Spring Boot Admin, Eureka and Consul for monitoring microservices. Finally, it outlines hands-on labs to set up monitoring of a sample application with different tool combinations.
This document discusses various ways to interface with Oracle Cloud Infrastructure (OCI) Object Storage using different tools and SDKs. It covers using the OCI CLI to manage buckets and upload/download files, using Java and Python SDKs to programmatically interact with Object Storage, integrating Object Storage with Hadoop via the HDFS Connector, and using Object Storage as a data source for services like Oracle Autonomous Data Warehouse. The goal is to provide a common interface for data via virtualization regardless of where the data is physically stored.
The document discusses Oracle NoSQL Database and its features. It provides an overview of NoSQL databases and data models in Oracle NoSQL including key-value, table, and JSON. It also describes Oracle NoSQL's architecture, which uses automatic data sharding and replication across storage nodes for high availability and scalability. Configuration and usage is simplified with libraries and command line tools.
This document discusses Redis, MongoDB, and Amazon DynamoDB. It begins with an overview of NoSQL databases and the differences between SQL and NoSQL databases. It then covers Redis data types like strings, hashes, lists, sets, sorted sets, and streams. Examples use cases for Redis are also provided like leaderboards, geospatial queries, and message queues. The document also discusses MongoDB design patterns like embedding data, embracing duplication, and relationships. Finally, it provides a high-level overview of DynamoDB concepts like tables, items, attributes, and primary keys.
Connecting your .Net Applications to NoSQL Databases - MongoDB & CassandraLohith Goudagere Nagaraj
The document discusses various ways to connect .NET applications to NoSQL databases like MongoDB and Cassandra. It covers client SDK APIs, REST/SOAP APIs, and SQL-based connectivity options. For SQL connectivity, the document explains that Progress DataDirect drivers normalize the NoSQL data model to expose it through SQL. Examples demonstrate connecting to MongoDB and Cassandra using the MongoDB and Cassandra .NET drivers, their REST APIs, and Progress DataDirect's ODBC drivers with SQL. The document concludes that SQL connectivity requires data normalization but offers familiar skills and easy BI integration.
Apache Ignite is an integrated and distributed In-Memory Data Fabric for computing and transacting on large-scale data sets in real-time, orders of magnitude faster than possible with traditional disk-based or flash technologies. It is designed to easily power both existing and new applications in a distributed, massively parallel architecture on affordable, industry-standard hardware. Apache Ignite addresses today's Fast Data and Big Data needs by providing a comprehensive in-memory data fabric, which includes a data grid with SQL and transactional capabilities, in-memory streaming, an in-memory file system, and more.
Reactive Java EE - Let Me Count the Ways!Reza Rahman
As our industry matures there are pockets of increased demand for high-throughput, low-latency systems heavily utilizing event-driven programming and asynchronous processing. This trend is gradually converging on the somewhat well established but so-far not well understood term "Reactive".
This session explores how vanilla Java SE and Java EE aligns with this movement via features and APIs like JMS, MDB, EJB @Asynchronous, JAX-RS/Servlet/WebSocket async, CDI events, Java EE concurrency utilities and so on. We will also see how these robust facilities can be made digestible even in the most complex cases for mere mortal developers through Java SE 8 Lambdas and Completable Futures.
JDBC Next: A New Asynchronous API for Connecting to a Database Yolande Poirier
This new API is completely nonblocking. It is not intended to be an extension to, or a replacement for, JDBC but, rather, an entirely separate API that provides completely nonblocking access to the same databases as JDBC.
Self-Service Data Ingestion Using NiFi, StreamSets & KafkaGuido Schmutz
Many of the Big Data and IoT use cases are based on combining data from multiple data sources and to make them available on a Big Data platform for analysis. The data sources are often very heterogeneous, from simple files, databases to high-volume event streams from sensors (IoT devices). It’s important to retrieve this data in a secure and reliable manner and integrate it with the Big Data platform so that it is available for analysis in real-time (stream processing) as well as in batch (typical big data processing). In past some new tools have emerged, which are especially capable of handling the process of integrating data from outside, often called Data Ingestion. From an outside perspective, they are very similar to a traditional Enterprise Service Bus infrastructures, which in larger organization are often in use to handle message-driven and service-oriented systems. But there are also important differences, they are typically easier to scale in a horizontal fashion, offer a more distributed setup, are capable of handling high-volumes of data/messages, provide a very detailed monitoring on message level and integrate very well with the Hadoop ecosystem. This session will present and compare Apache NiFi, StreamSets and the Kafka Ecosystem and show how they handle the data ingestion in a Big Data solution architecture.
Big Data, Data Lake, Fast Data - Dataserialiation-FormatsGuido Schmutz
The concept of "Data Lake" is in everyone's mind today. The idea of storing all the data that accumulates in a company in a central location and making it available sounds very interesting at first. But Data Lake can quickly turn from a clear, beautiful mountain lake into a huge pond, especially if it is inexpertly entrusted with all the source data formats that are common in today's enterprises, such as XML, JSON, CSV or unstructured text data. Who, after some time, still has an overview of which data, which format and how they have developed over different versions? Anyone who wants to help themselves from the Data Lake must ask themselves the same questions over and over again: what information is provided, what data types do they have and how has the content changed over time?
Data serialization frameworks such as Apache Avro and Google Protocol Buffer (Protobuf), which enable platform-independent data modeling and data storage, can help. This talk will discuss the possibilities of Avro and Protobuf and show how they can be used in the context of a data lake and what advantages can be achieved. The support on Avro and Protobuf by Big Data and Fast Data platforms is also a topic.
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaGuido Schmutz
Apache Kafka is a popular distributed streaming data platform and more and more is the architectural backbone for integrating streaming data with a Data Lake, Microservices and Stream Processing. A lot of data necessary in stream processing is stored in traditional systems backed by relational databases. This session will present different approaches for integrating relational databases with Kafka, such as Kafka Connect, Oracle GoldenGate, ORDS APIs and bridging Kafka with Oracle AQ.
Stream Processing in the Cloud With Data Microservicesmarius_bogoevici
The future of scalable data processing is event-driven microservices! They provide a powerful paradigm that solves issues typically associated with distributed applications such as availability, data consistency, or communication complexity, and allows the creation of sophisticated and extensible data processing pipelines.
Building on the ease of development and deployment provided by Spring Boot and the cloud native capabilities of Spring Cloud, the Spring Cloud Stream project provides a simple and powerful framework for creating event-driven microservices. They make it easy to develop data-processing Spring Boot applications that build upon the capabilities of Spring Integration. At a higher level of abstraction, Spring Cloud Data Flow is an integrated orchestration layer that provides a highly productive experience for deploying and managing sophisticated data pipelines consisting of standalone microservices. Streams are defined using a DSL abstraction and can be managed via shell and a web UI. Furthermore, a pluggable runtime SPI allows Spring Cloud Data Flow to coordinate these applications across a variety of distributed runtime platforms such as Apache YARN, Cloud Foundry, Kubernetes, or Apache Mesos.
The big data platforms of many organisations are underpinned by a technology that is soon to celebrate its 45th birthday: SQL. This industry stalwart is applied in a multitude of critical points in business data flows; the results that these processes generate may significantly influence business and financial decision making. However, the SQL ecosystem has been overlooked and ignored by more recent innovations in the field of software engineering best practices such as fine grained automated testing and code quality metrics. This exposes organisations to poor application maintainability, high bug rates, and ultimately corporate risk.
We present the work we’ve been doing at Hotels.com to address these issues by bringing some advanced software engineering practices and open source tools to the realm of Apache Hive SQL. We first define the relevance of such approaches and demonstrate how automated testing can be applied to Hive SQL using HiveRunner, a JUnit based testing framework. We next consider how best to structure Hive queries to yield meaningful test scenarios that are maintainable and performant. Finally, we demonstrate how test coverage reports can highlight areas of risk in SQL codebases and weaknesses in the testing process. We do this using Mutant Swarm, an open source mutation testing tool for SQL languages developed by Hotels.com that can deliver insights similar to those produced by Java focused tools such as Jacoco and PIT.
This document discusses how to deliver a multi-tenant and PCI compliant Exalogic platform. It outlines the challenges of a shared platform including compliance with regulations like PCI-DSS. It then explains how Exalogic addresses these challenges through automation, isolation across storage, network and virtualization layers, and other security controls. Key aspects covered include per-tenant provisioning, encryption, firewalling, patching, and auditing to ensure isolation and compliance for different tenants including those processing credit card data.
Java EE 8 Presentation given at NYC Java SIG on May 4, 2017. This presentation provides the latest information on the forthcoming release of Java EE 8 in June.
How web works and browser works ? (behind the scenes)Vibhor Grover
how web and browser works, this presentation can help you in understanding what happens when you enter a URL in your browser and how the page is displayed by the browser, and how we can improve the performance of our applications.
This document provides an overview of Sybase BAM (Business Activity Monitoring). It discusses the technology background of BAM, CEP, and RTBI. It then describes Sybase BAM's analytic model, architecture, main features, and a demo. The analytic model uses fields, rules, actions and timers to process events. The architecture includes components like the BAM engine and tools. Main features include support for complex event processing, real-time BI, alerts, visualization, metadata-driven design, and high volume processing.
Enterprise Java Web Application Frameworks Sample Stack ImplementationMert Çalışkan
This document provides an overview of enterprise Java web application frameworks and sample stack implementations. It discusses choosing between various UI, controller, model, and integration frameworks like JSF, Spring, Hibernate, and Apache CXF. It then demonstrates a sample stack using these technologies along with Maven, Eclipse, and other tools. The aim is to provide a scalable and high-performance MVC architecture using proven open source solutions.
Enterprise Application Architectures by Dr. Indika KumaraThejan Wijesinghe
Enterprise Applications/Computing
Architecture Styles for Enterprise Applications
Method-oriented
Message-oriented
Resource-oriented
REST (representational state transfer)
Event-oriented
SOA (service-oriented architecture)
Basic and extended SOA
Implementing SOA
RESTful
WS-* (web services stack)
ESB (enterprise service bus)
Business processes and service compositions
Migrating ETL Workflow to Apache Spark at Scale in PinterestDatabricks
The document summarizes Pinterest's migration of ETL workflows from Cascading and Scalding to Spark. Key points:
- Pinterest runs Spark on AWS but manages its own clusters to avoid vendor lock-in. They have multiple Spark clusters with hundreds to thousands of nodes.
- The migration plan is to move remaining workloads from Hive, Cascading/Scalding, and Hadoop streaming to SparkSQL, PySpark, and native Spark over time. An automatic migration service helps with the process.
- Technical challenges included secondary sorting, accumulators behaving differently between frameworks, and output committer issues. Performance profiling and tuning was also important.
- Results of migrating so far include
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaGuido Schmutz
Apache Kafka is a popular distributed streaming data platform. A Kafka cluster stores streams of records (messages) in categories called topics. It is the architectural backbone for integrating streaming data with a Data Lake, Microservices and Stream Processing. Data sources flowing into Kafka are often native data streams such as social media streams, telemetry data, financial transactions and many others. But these data stream only contain part of the information. A lot of data necessary in stream processing is stored in traditional systems backed by relational databases. To implement new and modern, real-time solutions, an up-to-date view of that information is needed. So how do we make sure that information can flow between the RDBMS and Kafka, so that changes are available in Kafka as soon as possible in near-real-time? This session will present different approaches for integrating relational databases with Kafka, such as Kafka Connect, Oracle GoldenGate and bridging Kafka with Oracle Advanced Queuing (AQ).
Stream and Batch Processing in the Cloud with Data Microservicesmarius_bogoevici
The future of scalable data processing is microservices! Building on the ease of development and deployment provided by Spring Boot and the cloud native capabilities of Spring Cloud, the Spring Cloud Stream and Spring Cloud Task projects provide a simple and powerful framework for creating microservices for stream and batch processing. They make it easy to develop data-processing Spring Boot applications that build upon the capabilities of Spring Integration and Spring Batch, respectively. At a higher level of abstraction, Spring Cloud Data Flow is an integrated orchestration layer that provides a highly productive experience for deploying and managing sophisticated data pipelines consisting of standalone microservices. Streams and tasks are defined using a DSL abstraction and can be managed via shell and a web UI. Furthermore, a pluggable runtime SPI allows Spring Cloud Data Flow to coordinate these applications across a variety of distributed runtime platforms such as Apache YARN, Cloud Foundry, or Apache Mesos. This session will provide an overview of these projects, including how they evolved out of Spring XD. Both streaming and batch-oriented applications will be deployed in live demos on different platforms ranging from local cluster to a remote Cloud to show the simplicity of the developer experience.
Druid is a high performance, column-oriented distributed data store that is widely used at Oath for big data analysis. Druid has a JSON schema as its query language, making it difficult for new users unfamiliar with the schema to start querying Druid quickly. The JSON schema is designed to work with the data ingestion methods of Druid, so it can provide high performance features such as data aggregations in JSON, but many are unable to utilize such features, because they not familiar with the specifics of how to optimize Druid queries. However, most new Druid users at Yahoo are already very familiar with SQL, and the queries they want to write for Druid can be converted to concise SQL.
We found that our data analysts wanted an easy way to issue ad-hoc Druid queries and view the results in a BI tool in a way that's presentable to nontechnical stakeholders. In order to achieve this, we had to bridge the gap between Druid, SQL, and our BI tools such as Apache Superset. In this talk, we will explore different ways to query a Druid datasource in SQL and discuss which methods were most appropriate for our use cases. We will also discuss our open source contributions so others can utilize our work. GURUGANESH KOTTA, Software Dev Eng, Oath and JUNXIAN WU, Software Engineer, Oath Inc.
Apache Druid Auto Scale-out/in for Streaming Data Ingestion on KubernetesDataWorks Summit
Apache Druid supports auto-scaling of Middle Manager nodes to handle changes in data ingestion load. On Kubernetes, this can be implemented using Horizontal Pod Autoscaling based on custom metrics exposed from the Druid Overlord process, such as the number of pending/running tasks and expected number of workers. The autoscaler scales the number of Middle Manager pods between minimum and maximum thresholds to maintain a target average load percentage.
Druid: Sub-Second OLAP queries over Petabytes of Streaming DataDataWorks Summit
When interacting with analytics dashboards in order to achieve a smooth user experience, two major key requirements are sub-second response time and data freshness. Cluster computing frameworks such as Hadoop or Hive/Hbase work well for storing large volumes of data, although they are not optimized for ingesting streaming data and making it available for queries in realtime. Also, long query latencies make these systems sub-optimal choices for powering interactive dashboards and BI use-cases.
In this talk we will present Druid as a complementary solution to existing hadoop based technologies. Druid is an open-source analytics data store, designed from scratch, for OLAP and business intelligence queries over massive data streams. It provides low latency realtime data ingestion and fast sub-second adhoc flexible data exploration queries.
Many large companies are switching to Druid for analytics, and we will cover how druid is able to handle massive data streams and why it is a good fit for BI use cases.
Agenda -
1) Introduction and Ideal Use cases for Druid
2) Data Architecture
3) Streaming Ingestion with Kafka
4) Demo using Druid, Kafka and Superset.
5) Recent Improvements in Druid moving from lambda architecture to Exactly once Ingestion
6) Future Work
NoSQL no more: SQL on Druid with Apache Calcitegianmerlino
Druid is an analytics-focused, distributed, scale-out data store. Existing Druid clusters have scaled to petabytes of data and trillions of events, ingesting millions of events every second. Up until version 0.10, Druid could only be queried in a JSON-based language that many users found unfamiliar.
Enter Apache Calcite. It includes an industry-standard SQL parser, validator, and JDBC driver, as well as a cost-based relational optimizer. Calcite bills itself as “the foundation for your next high-performance database” and is used by Hive, Drill, and a variety of other projects. Druid uses Calcite to power Druid SQL, a standards-based query API that vaults Druid out of the NoSQL world and into the SQL world.
Gian Merlino offers an overview of Druid SQL and explains how Druid and Calcite are integrated and why you should stop worrying and learn to love relational algebra in your own projects.
This document discusses migrating databases from Oracle to PostgreSQL using AWS Database Migration Service (DMS) and AWS Schema Conversion Tool (SCT). It provides an overview of DMS and SCT, the migration process which involves assessing the database with SCT and then using DMS to replicate the data, and resources available to customers for both services. It also provides background on PostgreSQL, describing it as an open-source, object-relational database management system.
Modularization With Project Jigsaw in JDK 9Simon Ritter
The document discusses Project Jigsaw and modularization in JDK 9. It introduces modularization and modules, explaining that modules group code and declare dependencies. It outlines changes in JDK 9 like encapsulating internal APIs and changing the binary structure. The goals of modularization are to make Java more scalable, flexible, secure and maintainable for large applications. Modules, compilation, execution and linking with modular JAR files are also summarized.
Enable SQL/JDBC Access to Apache Geode/GemFire Using Apache CalciteChristian Tzolov
https://springoneplatform.io/sessions/enable-sql-jdbc-access-to-apache-geode-gemfire-using-apache-calcite
When working with BigData & IoT systems we often feel the need for an established, Common Query Language.
To fill this gap some NoSql vendors are building SQL access to their systems. Building SQL engine from scratch is a daunting job and frameworks like Apache Calcite can help you with the heavy lifting. It allows you to integrate SQL parser, Cost-Based Optimizer, and JDBC with your NoSql system. Calcite has been used to empower many BigData platforms such as Hive, Spark, Flink, Drill, HBase/Phoenix to name some.
In this session I will walk you through the process of building a SQL access layer for Apache Geode (GemFire). I will share my experience, pitfalls and technical consideration like balancing between the SQL/RDBMS semantics and the design choices and limitations of In-Memory-Data-Grid systems like Geode.
Hopefully this will enable you to add SQL capabilities to your preferred NoSQL data system.
Cloud-Native Streaming and Event-Driven MicroservicesVMware Tanzu
MARIUS BOGOEVICI SPRING CLOUD STREAM LEAD
Join us for an introduction to Spring Cloud Stream, a framework for creating event-driven microservices that builds on on the ease of development and execution of Spring Boot, the cloud-native capabilities of Spring Cloud, and the message-driven programming model of Spring Integration. See how Spring Cloud Stream’s abstractions and opinionated primitives allow you to easily build applications that can interchangeably use RabbitMQ, Kafka or Google PubSub without changing the application logic. Finally, we will show how these applications can be orchestrated and deployed on different modern runtimes such as Cloud Foundry, Kubernetes or Mesos using Spring Cloud Data Flow.
Self-Service Data Ingestion Using NiFi, StreamSets & KafkaGuido Schmutz
Many of the Big Data and IoT use cases are based on combining data from multiple data sources and to make them available on a Big Data platform for analysis. The data sources are often very heterogeneous, from simple files, databases to high-volume event streams from sensors (IoT devices). It’s important to retrieve this data in a secure and reliable manner and integrate it with the Big Data platform so that it is available for analysis in real-time (stream processing) as well as in batch (typical big data processing). In past some new tools have emerged, which are especially capable of handling the process of integrating data from outside, often called Data Ingestion. From an outside perspective, they are very similar to a traditional Enterprise Service Bus infrastructures, which in larger organization are often in use to handle message-driven and service-oriented systems. But there are also important differences, they are typically easier to scale in a horizontal fashion, offer a more distributed setup, are capable of handling high-volumes of data/messages, provide a very detailed monitoring on message level and integrate very well with the Hadoop ecosystem. This session will present and compare Apache NiFi, StreamSets and the Kafka Ecosystem and show how they handle the data ingestion in a Big Data solution architecture.
Big Data, Data Lake, Fast Data - Dataserialiation-FormatsGuido Schmutz
The concept of "Data Lake" is in everyone's mind today. The idea of storing all the data that accumulates in a company in a central location and making it available sounds very interesting at first. But Data Lake can quickly turn from a clear, beautiful mountain lake into a huge pond, especially if it is inexpertly entrusted with all the source data formats that are common in today's enterprises, such as XML, JSON, CSV or unstructured text data. Who, after some time, still has an overview of which data, which format and how they have developed over different versions? Anyone who wants to help themselves from the Data Lake must ask themselves the same questions over and over again: what information is provided, what data types do they have and how has the content changed over time?
Data serialization frameworks such as Apache Avro and Google Protocol Buffer (Protobuf), which enable platform-independent data modeling and data storage, can help. This talk will discuss the possibilities of Avro and Protobuf and show how they can be used in the context of a data lake and what advantages can be achieved. The support on Avro and Protobuf by Big Data and Fast Data platforms is also a topic.
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaGuido Schmutz
Apache Kafka is a popular distributed streaming data platform and more and more is the architectural backbone for integrating streaming data with a Data Lake, Microservices and Stream Processing. A lot of data necessary in stream processing is stored in traditional systems backed by relational databases. This session will present different approaches for integrating relational databases with Kafka, such as Kafka Connect, Oracle GoldenGate, ORDS APIs and bridging Kafka with Oracle AQ.
Stream Processing in the Cloud With Data Microservicesmarius_bogoevici
The future of scalable data processing is event-driven microservices! They provide a powerful paradigm that solves issues typically associated with distributed applications such as availability, data consistency, or communication complexity, and allows the creation of sophisticated and extensible data processing pipelines.
Building on the ease of development and deployment provided by Spring Boot and the cloud native capabilities of Spring Cloud, the Spring Cloud Stream project provides a simple and powerful framework for creating event-driven microservices. They make it easy to develop data-processing Spring Boot applications that build upon the capabilities of Spring Integration. At a higher level of abstraction, Spring Cloud Data Flow is an integrated orchestration layer that provides a highly productive experience for deploying and managing sophisticated data pipelines consisting of standalone microservices. Streams are defined using a DSL abstraction and can be managed via shell and a web UI. Furthermore, a pluggable runtime SPI allows Spring Cloud Data Flow to coordinate these applications across a variety of distributed runtime platforms such as Apache YARN, Cloud Foundry, Kubernetes, or Apache Mesos.
The big data platforms of many organisations are underpinned by a technology that is soon to celebrate its 45th birthday: SQL. This industry stalwart is applied in a multitude of critical points in business data flows; the results that these processes generate may significantly influence business and financial decision making. However, the SQL ecosystem has been overlooked and ignored by more recent innovations in the field of software engineering best practices such as fine grained automated testing and code quality metrics. This exposes organisations to poor application maintainability, high bug rates, and ultimately corporate risk.
We present the work we’ve been doing at Hotels.com to address these issues by bringing some advanced software engineering practices and open source tools to the realm of Apache Hive SQL. We first define the relevance of such approaches and demonstrate how automated testing can be applied to Hive SQL using HiveRunner, a JUnit based testing framework. We next consider how best to structure Hive queries to yield meaningful test scenarios that are maintainable and performant. Finally, we demonstrate how test coverage reports can highlight areas of risk in SQL codebases and weaknesses in the testing process. We do this using Mutant Swarm, an open source mutation testing tool for SQL languages developed by Hotels.com that can deliver insights similar to those produced by Java focused tools such as Jacoco and PIT.
This document discusses how to deliver a multi-tenant and PCI compliant Exalogic platform. It outlines the challenges of a shared platform including compliance with regulations like PCI-DSS. It then explains how Exalogic addresses these challenges through automation, isolation across storage, network and virtualization layers, and other security controls. Key aspects covered include per-tenant provisioning, encryption, firewalling, patching, and auditing to ensure isolation and compliance for different tenants including those processing credit card data.
Java EE 8 Presentation given at NYC Java SIG on May 4, 2017. This presentation provides the latest information on the forthcoming release of Java EE 8 in June.
How web works and browser works ? (behind the scenes)Vibhor Grover
how web and browser works, this presentation can help you in understanding what happens when you enter a URL in your browser and how the page is displayed by the browser, and how we can improve the performance of our applications.
This document provides an overview of Sybase BAM (Business Activity Monitoring). It discusses the technology background of BAM, CEP, and RTBI. It then describes Sybase BAM's analytic model, architecture, main features, and a demo. The analytic model uses fields, rules, actions and timers to process events. The architecture includes components like the BAM engine and tools. Main features include support for complex event processing, real-time BI, alerts, visualization, metadata-driven design, and high volume processing.
Enterprise Java Web Application Frameworks Sample Stack ImplementationMert Çalışkan
This document provides an overview of enterprise Java web application frameworks and sample stack implementations. It discusses choosing between various UI, controller, model, and integration frameworks like JSF, Spring, Hibernate, and Apache CXF. It then demonstrates a sample stack using these technologies along with Maven, Eclipse, and other tools. The aim is to provide a scalable and high-performance MVC architecture using proven open source solutions.
Enterprise Application Architectures by Dr. Indika KumaraThejan Wijesinghe
Enterprise Applications/Computing
Architecture Styles for Enterprise Applications
Method-oriented
Message-oriented
Resource-oriented
REST (representational state transfer)
Event-oriented
SOA (service-oriented architecture)
Basic and extended SOA
Implementing SOA
RESTful
WS-* (web services stack)
ESB (enterprise service bus)
Business processes and service compositions
Migrating ETL Workflow to Apache Spark at Scale in PinterestDatabricks
The document summarizes Pinterest's migration of ETL workflows from Cascading and Scalding to Spark. Key points:
- Pinterest runs Spark on AWS but manages its own clusters to avoid vendor lock-in. They have multiple Spark clusters with hundreds to thousands of nodes.
- The migration plan is to move remaining workloads from Hive, Cascading/Scalding, and Hadoop streaming to SparkSQL, PySpark, and native Spark over time. An automatic migration service helps with the process.
- Technical challenges included secondary sorting, accumulators behaving differently between frameworks, and output committer issues. Performance profiling and tuning was also important.
- Results of migrating so far include
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaGuido Schmutz
Apache Kafka is a popular distributed streaming data platform. A Kafka cluster stores streams of records (messages) in categories called topics. It is the architectural backbone for integrating streaming data with a Data Lake, Microservices and Stream Processing. Data sources flowing into Kafka are often native data streams such as social media streams, telemetry data, financial transactions and many others. But these data stream only contain part of the information. A lot of data necessary in stream processing is stored in traditional systems backed by relational databases. To implement new and modern, real-time solutions, an up-to-date view of that information is needed. So how do we make sure that information can flow between the RDBMS and Kafka, so that changes are available in Kafka as soon as possible in near-real-time? This session will present different approaches for integrating relational databases with Kafka, such as Kafka Connect, Oracle GoldenGate and bridging Kafka with Oracle Advanced Queuing (AQ).
Stream and Batch Processing in the Cloud with Data Microservicesmarius_bogoevici
The future of scalable data processing is microservices! Building on the ease of development and deployment provided by Spring Boot and the cloud native capabilities of Spring Cloud, the Spring Cloud Stream and Spring Cloud Task projects provide a simple and powerful framework for creating microservices for stream and batch processing. They make it easy to develop data-processing Spring Boot applications that build upon the capabilities of Spring Integration and Spring Batch, respectively. At a higher level of abstraction, Spring Cloud Data Flow is an integrated orchestration layer that provides a highly productive experience for deploying and managing sophisticated data pipelines consisting of standalone microservices. Streams and tasks are defined using a DSL abstraction and can be managed via shell and a web UI. Furthermore, a pluggable runtime SPI allows Spring Cloud Data Flow to coordinate these applications across a variety of distributed runtime platforms such as Apache YARN, Cloud Foundry, or Apache Mesos. This session will provide an overview of these projects, including how they evolved out of Spring XD. Both streaming and batch-oriented applications will be deployed in live demos on different platforms ranging from local cluster to a remote Cloud to show the simplicity of the developer experience.
Druid is a high performance, column-oriented distributed data store that is widely used at Oath for big data analysis. Druid has a JSON schema as its query language, making it difficult for new users unfamiliar with the schema to start querying Druid quickly. The JSON schema is designed to work with the data ingestion methods of Druid, so it can provide high performance features such as data aggregations in JSON, but many are unable to utilize such features, because they not familiar with the specifics of how to optimize Druid queries. However, most new Druid users at Yahoo are already very familiar with SQL, and the queries they want to write for Druid can be converted to concise SQL.
We found that our data analysts wanted an easy way to issue ad-hoc Druid queries and view the results in a BI tool in a way that's presentable to nontechnical stakeholders. In order to achieve this, we had to bridge the gap between Druid, SQL, and our BI tools such as Apache Superset. In this talk, we will explore different ways to query a Druid datasource in SQL and discuss which methods were most appropriate for our use cases. We will also discuss our open source contributions so others can utilize our work. GURUGANESH KOTTA, Software Dev Eng, Oath and JUNXIAN WU, Software Engineer, Oath Inc.
Apache Druid Auto Scale-out/in for Streaming Data Ingestion on KubernetesDataWorks Summit
Apache Druid supports auto-scaling of Middle Manager nodes to handle changes in data ingestion load. On Kubernetes, this can be implemented using Horizontal Pod Autoscaling based on custom metrics exposed from the Druid Overlord process, such as the number of pending/running tasks and expected number of workers. The autoscaler scales the number of Middle Manager pods between minimum and maximum thresholds to maintain a target average load percentage.
Druid: Sub-Second OLAP queries over Petabytes of Streaming DataDataWorks Summit
When interacting with analytics dashboards in order to achieve a smooth user experience, two major key requirements are sub-second response time and data freshness. Cluster computing frameworks such as Hadoop or Hive/Hbase work well for storing large volumes of data, although they are not optimized for ingesting streaming data and making it available for queries in realtime. Also, long query latencies make these systems sub-optimal choices for powering interactive dashboards and BI use-cases.
In this talk we will present Druid as a complementary solution to existing hadoop based technologies. Druid is an open-source analytics data store, designed from scratch, for OLAP and business intelligence queries over massive data streams. It provides low latency realtime data ingestion and fast sub-second adhoc flexible data exploration queries.
Many large companies are switching to Druid for analytics, and we will cover how druid is able to handle massive data streams and why it is a good fit for BI use cases.
Agenda -
1) Introduction and Ideal Use cases for Druid
2) Data Architecture
3) Streaming Ingestion with Kafka
4) Demo using Druid, Kafka and Superset.
5) Recent Improvements in Druid moving from lambda architecture to Exactly once Ingestion
6) Future Work
NoSQL no more: SQL on Druid with Apache Calcitegianmerlino
Druid is an analytics-focused, distributed, scale-out data store. Existing Druid clusters have scaled to petabytes of data and trillions of events, ingesting millions of events every second. Up until version 0.10, Druid could only be queried in a JSON-based language that many users found unfamiliar.
Enter Apache Calcite. It includes an industry-standard SQL parser, validator, and JDBC driver, as well as a cost-based relational optimizer. Calcite bills itself as “the foundation for your next high-performance database” and is used by Hive, Drill, and a variety of other projects. Druid uses Calcite to power Druid SQL, a standards-based query API that vaults Druid out of the NoSQL world and into the SQL world.
Gian Merlino offers an overview of Druid SQL and explains how Druid and Calcite are integrated and why you should stop worrying and learn to love relational algebra in your own projects.
This document discusses migrating databases from Oracle to PostgreSQL using AWS Database Migration Service (DMS) and AWS Schema Conversion Tool (SCT). It provides an overview of DMS and SCT, the migration process which involves assessing the database with SCT and then using DMS to replicate the data, and resources available to customers for both services. It also provides background on PostgreSQL, describing it as an open-source, object-relational database management system.
Modularization With Project Jigsaw in JDK 9Simon Ritter
The document discusses Project Jigsaw and modularization in JDK 9. It introduces modularization and modules, explaining that modules group code and declare dependencies. It outlines changes in JDK 9 like encapsulating internal APIs and changing the binary structure. The goals of modularization are to make Java more scalable, flexible, secure and maintainable for large applications. Modules, compilation, execution and linking with modular JAR files are also summarized.
Enable SQL/JDBC Access to Apache Geode/GemFire Using Apache CalciteChristian Tzolov
https://springoneplatform.io/sessions/enable-sql-jdbc-access-to-apache-geode-gemfire-using-apache-calcite
When working with BigData & IoT systems we often feel the need for an established, Common Query Language.
To fill this gap some NoSql vendors are building SQL access to their systems. Building SQL engine from scratch is a daunting job and frameworks like Apache Calcite can help you with the heavy lifting. It allows you to integrate SQL parser, Cost-Based Optimizer, and JDBC with your NoSql system. Calcite has been used to empower many BigData platforms such as Hive, Spark, Flink, Drill, HBase/Phoenix to name some.
In this session I will walk you through the process of building a SQL access layer for Apache Geode (GemFire). I will share my experience, pitfalls and technical consideration like balancing between the SQL/RDBMS semantics and the design choices and limitations of In-Memory-Data-Grid systems like Geode.
Hopefully this will enable you to add SQL capabilities to your preferred NoSQL data system.
Cloud-Native Streaming and Event-Driven MicroservicesVMware Tanzu
MARIUS BOGOEVICI SPRING CLOUD STREAM LEAD
Join us for an introduction to Spring Cloud Stream, a framework for creating event-driven microservices that builds on on the ease of development and execution of Spring Boot, the cloud-native capabilities of Spring Cloud, and the message-driven programming model of Spring Integration. See how Spring Cloud Stream’s abstractions and opinionated primitives allow you to easily build applications that can interchangeably use RabbitMQ, Kafka or Google PubSub without changing the application logic. Finally, we will show how these applications can be orchestrated and deployed on different modern runtimes such as Cloud Foundry, Kubernetes or Mesos using Spring Cloud Data Flow.
SpringFramework 5에서 선보이는 Reactive와 같은 핵심기능이 2017 2017년 12월 샌프란시스코에서 열린 Spring One Platform행사에서 소개된 내용중 Spring Data, Spring Security, Spring WebFlux프로젝트에 녹아져 있는지 살펴봅니다. 또한 이러한 기능들이 어떻게 여러분의 시스템의 반응성을 높이고 효율적으로 동작하게 하는지 알아봅니다.
As our applications grow and need to deal with new big data challenges and global distribution we look to new data stores designed to deal with these new challenges. Cassandra is one great tool to deal with both of these problems, we will go over some of the different ways of dealing with Cassandra from Grails. I will cover the various Cassandra plugins for grails, including the Cassandra ORM, Astyanax, and Cassandra GORM. Also, I will talk about using the Cassandra Java driver directly. By the end of the talk you should have a overview of the data modeling that will working well in Cassandra paired with Grails, when to use or not use each plugin, and some of the basic connection configuration details needed.
Under the Hood of Reactive Data Access (2/2)VMware Tanzu
SpringOne Platform 2017
Christoph Strobl, Pivotal; Mark Paluch, Pivotal
"A huge theme in Spring Framework 5.0 and its ecosystem projects is the native reactive support that empowers you to build end-to-end reactive applications. Reactive data access especially requires a reactive infrastructure. But how is this one different from the ones used before? How does it deal with I/O?
In this session, we will demystify what happens inside the driver and give you a better understanding of their capabilities. You will learn about the inner mechanics of reactive data access by walking through reactive drivers that are used in Spring Data."
Kafka Summit NYC 2017 - Cloud Native Data Streaming Microservices with Spring...confluent
This document discusses building microservices for data streaming and processing using Spring Cloud and Kafka. It provides an overview of Spring Cloud Stream and how it can be used to build event-driven microservices that connect to Kafka. It also discusses how Spring Cloud Data Flow can be used to orchestrate and deploy streaming applications and topologies. The document includes code samples of building a basic Kafka Streams processor application using Spring Cloud Stream and deploying it as part of a streaming data flow. It concludes with proposing a demonstration of these techniques.
Building Highly Scalable Spring Applications using In-Memory Data GridsJohn Blum
Slides for Luke Shannon and I's presentation at SpringOne2GX-2015 in Washingon D.C. on Tuesday, September 15th from 10:30 am to 12:00 PM EDT.
Session details @ https://2015.event.springone2gx.com/schedule/sessions/building_highly_scalable_spring_applications_with_in_memory_distributed_data_grids.html.
Federated Queries with HAWQ - SQL on Hadoop and BeyondChristian Tzolov
In the space of Big Data, Pivotal offers two powerful data processing tools namely HAWQ and GemFire. HAWQ is a scalable OLAP SQL-on-Hadoop system, while GemFire is OLTP like, in-memory data grid and event processing system. This presentation will show different integration approaches that allow integration and data exchange between HAWQ and GemFire. The practical experience in applying Spring Boot and Spring XD for some of the use cases will be shared while walking you through the implementation of the different Integration strategies. Amongst other we will show an integration path that leverages SpringXD to ingest GemFire data and store it in HDFS as well as the benefits of using Spring Boot to implement REStful proxy for the HAWQ Web Table integration scenario.
Simple Data Movement Patterns: Legacy Application to Cloud-Native Environment...VMware Tanzu
SpringOne Platform 2019
Session Title: Simple Data Movement Patterns: Legacy Application to Cloud-Native Environment and Apache Geode
Speaker: James Bedenbaugh, Advisory Data Solutions Architect, Pivotal; Zachary Hansen, Data Transformation Solutions Architect, Pivotal
Youtube: https://youtu.be/7ds0YZNlhmE
SpringFramework 5에서 선보이는 Reactive와 같은 핵심기능이 Spring Data, Spring Security, Spring WebFlux프로젝트에 녹아져 있는지 살펴봅니다. 또한 이러한 기능들이 어떻게 여러분의 시스템의 반응성을 높이고 효율적으로 동작하게 하는지 알아봅니다.
Lattice: A Cloud-Native Platform for Your Spring ApplicationsMatt Stine
As presented at SpringOne2GX 2015 in Washington, DC.
Lattice is a cloud-native application platform that enables you to run your applications in containers like Docker, on your local machine via Vagrant. Lattice includes features like:
Cluster scheduling
HTTP load balancing
Log aggregation
Health management
Lattice does this by packaging a subset of the components found in the Cloud Foundry elastic runtime. The result is an open, single-tenant environment suitable for rapid application development, similar to Kubernetes and Mesos Applications developed using Lattice should migrate unchanged to full Cloud Foundry deployments.
Lattice can be used by Spring developers to spin up powerful micro-cloud environments on their desktops, and can be useful for developing and testing cloud-native application architectures. Lattice already has deep integration with Spring Cloud and Spring XD, and you’ll have the opportunity to see deep dives into both at this year’s SpringOne 2GX. This session will introduce the basics:
Installing Lattice
Lattice’s Architecture
How Lattice Differs from Cloud Foundry
How to Package and Run Your Spring Apps on Lattice
Quickly Build Spring Boot Applications to Consume Public Cloud ServicesVMware Tanzu
SpringOne Platform 2017
Prasad Bopardikar, Pivotal; Colin Stevenson, Pivotal
"We all know Cloud Foundry is a great platform for cloud-native applications. However, what happens when you’re building an app that leverages services from public cloud providers such as Microsoft, Google and Amazon? Service brokers make it easy to spin up service instances and bind to apps. What about the actual code itself?
Developers leverage the popular Spring Boot framework to quickly build Java apps to deploy to Cloud Foundry. The Spring Boot Starters and Auto-Configuration eliminate the need to write boilerplate code to consume some services, but not all.
We’ve decided to give you a head start. This session is about extending the Spring framework. We’ll use examples from our recent work with Microsoft Azure, Google Cloud Platform and Amazon AWS services. As more services become available, developers will want to consume these on Cloud Foundry. Extend Spring to make it easier for developers to consume those backing services!"
Connecting All Abstractions with IstioVMware Tanzu
SpringOne Platform 2017
Ramiro Salas, Pivotal
The concept of a service mesh represents a paradigm shift on application connectivity for distributed systems, with wide implications for analytics, policy and extensibility. In this talk, we will explain what a service mesh is, the power it brings to microservices, and its impact on Cloud Foundry and K8s, both separately and together. We will also discuss the implications for the traditional network infrastructure, and the shifting of responsibilities from L3/4 to L7, and our current thinking of using Istio to integrate all abstractions.
Caching for Microservives - Introduction to Pivotal Cloud CacheVMware Tanzu
SpringOne Platform 2017
Pulkit Chandra, Pivotal
"One of the most important factors in a microservices architecture is that application logic is separate from the data store. This design choice makes it easier for the application to scale. Providing a caching solution inside Pivotal Cloud Foundry makes it easy for these microservices to store data which can be retrieved 100x times faster than with a regular database. Pivotal Cloud Cache not only provides such a cache but takes a “use case”-based approach which gets an application from 0 to production fast.
This session will provide insights into how to use Pivotal Cloud Cache and its performance under load. We will demo a Spring Boot app which uses Spring Data Geode to talk to a Pivotal Cloud Cache cluster."
SpringOne Platform 2017
Ryan Baxter, Pivotal
You have heard and seen great things about Spring Cloud and you decide it is time to dive in and try it out yourself. You fire up your browser head to Google and land on the Spring Cloud homepage. Then it hits you, where do you begin? What do each of these projects do? Do you need to use all of them or can you be selective? The number of projects under the Spring Cloud umbrella has grown immensely over the past couple of years and if you are a newcomer to the Spring Cloud ecosystem it can be quite daunting to sift through the projects to find what you need. By the end of this talk you will leave with a solid understanding of the Spring Cloud projects, how to use them to build cloud native apps, and the confidence to get started!
Tools to Slay the Fire Breathing Monoliths in Your EnterpriseVMware Tanzu
SpringOne Platform 2017
Rohit Kelapure, Pivotal; Joe Szodfridt, Pivotal; Shaun Anderson, Pivotal
Are fire-breathing monoliths lurking throughout your Enterprise? Many of these ancient behemoths can be millions of lines long and can wreak havoc when trying to evolve and transform your business. Unfortunately, your business depends on services they provide, so they can’t just be eliminated without a battle plan. The Pivotal App Transformation practice has continuously refined approaches and techniques to slay your monoliths. In this session, we will discuss how to carve up your legacy dragons into manageable pieces using techniques and patterns such as Event Storming, Strangling, Starving, Slice Analysis and Domain Driven Decomposition. Monolith slaying is not easy, but with the right tools and weapons at your disposal, your journey to the Cloud can be as easy as a stroll through the forest.
Developing Real-Time Data Pipelines with Apache KafkaJoe Stein
Developing Real-Time Data Pipelines with Apache Kafka http://kafka.apache.org/ is an introduction for developers about why and how to use Apache Kafka. Apache Kafka is a publish-subscribe messaging system rethought of as a distributed commit log. Kafka is designed to allow a single cluster to serve as the central data backbone. A single Kafka broker can handle hundreds of megabytes of reads and writes per second from thousands of clients. It can be elastically and transparently expanded without downtime. Data streams are partitioned and spread over a cluster of machines to allow data streams larger than the capability of any single machine and to allow clusters of coordinated consumers. Messages are persisted on disk and replicated within the cluster to prevent data loss. Each broker can handle terabytes of messages. For the Spring user, Spring Integration Kafka and Spring XD provide integration with Apache Kafka.
Running Java Applications on Cloud FoundryVMware Tanzu
SpringOne Platform 2017
Ben Hale, Pivotal
From a developer's perspective, running a Java application on Cloud Foundry appears to consist of pushing a compiled artifact and getting a running process. From the platform's perspective though, there's a whole lot more going on. In this talk, the lead developer of the Java Buildpack will walk you through what goes on during application staging and what the buildpack can do for you. It will cover everything from dependency resolution to memory calculation and will even discuss how to integrate with marketplace services with no application configuration.
Similar to Enable SQL/JDBC Access to Apache Geode/GemFire Using Apache Calcite (20)
What AI Means For Your Product Strategy And What To Do About ItVMware Tanzu
The document summarizes Matthew Quinn's presentation on "What AI Means For Your Product Strategy And What To Do About It" at Denver Startup Week 2023. The presentation discusses how generative AI could impact product strategies by potentially solving problems companies have ignored or allowing competitors to create new solutions. Quinn advises product teams to evaluate their strategies and roadmaps, ensure they understand user needs, and consider how AI may change the problems being addressed. He provides examples of how AI could influence product development for apps in home organization and solar sales. Quinn concludes by urging attendees not to ignore AI's potential impacts and to have hard conversations about emerging threats and opportunities.
Make the Right Thing the Obvious Thing at Cardinal Health 2023VMware Tanzu
This document discusses the evolution of internal developer platforms and defines what they are. It provides a timeline of how technologies like infrastructure as a service, public clouds, containers and Kubernetes have shaped developer platforms. The key aspects of an internal developer platform are described as providing application-centric abstractions, service level agreements, automated processes from code to production, consolidated monitoring and feedback. The document advocates that internal platforms should make the right choices obvious and easy for developers. It also introduces Backstage as an open source solution for building internal developer portals.
Enhancing DevEx and Simplifying Operations at ScaleVMware Tanzu
Cardinal Health introduced Tanzu Application Service in 2016 and set up foundations for cloud native applications in AWS and later migrated to GCP in 2018. TAS has provided Cardinal Health with benefits like faster development of applications, zero downtime for critical applications, hosting over 5,000 application instances, quicker patching for security vulnerabilities, and savings through reduced lead times and staffing needs.
Dan Vega discussed upcoming changes and improvements in Spring including Spring Boot 3, which will have support for JDK 17, Jakarta EE 9/10, ahead-of-time compilation, improved observability with Micrometer, and Project Loom's virtual threads. Spring Boot 3.1 additions were also highlighted such as Docker Compose integration and Spring Authorization Server 1.0. Spring Boot 3.2 will focus on embracing virtual threads from Project Loom to improve scalability of web applications.
Platforms, Platform Engineering, & Platform as a ProductVMware Tanzu
This document discusses building platforms as products and reducing developer toil. It notes that platform engineering now encompasses PaaS and developer tools. A quote from Mercedes-Benz emphasizes building platforms for developers, not for the company itself. The document contrasts reactive, ticket-driven approaches with automated, self-service platforms and products. It discusses moving from considering platforms as a cost center to experts that drive business results. Finally, it provides questions to identify sources of developer toil, such as issues with workstation setup, running software locally, integration testing, committing changes, and release processes.
This document provides an overview of building cloud-ready applications in .NET. It defines what makes an application cloud-ready, discusses common issues with legacy applications, and recommends design patterns and practices to address these issues, including loose coupling, high cohesion, messaging, service discovery, API gateways, and resiliency policies. It includes code examples and links to additional resources.
Dan Vega discussed new features and capabilities in Spring Boot 3 and beyond, including support for JDK 17, Jakarta EE 9, ahead-of-time compilation, observability with Micrometer, Docker Compose integration, and initial support for Project Loom's virtual threads in Spring Boot 3.2 to improve scalability. He provided an overview of each new feature and explained how they can help Spring applications.
Spring Cloud Gateway - SpringOne Tour 2023 Charles Schwab.pdfVMware Tanzu
Spring Cloud Gateway is a gateway that provides routing, security, monitoring, and resiliency capabilities for microservices. It acts as an API gateway and sits in front of microservices, routing requests to the appropriate microservice. The gateway uses predicates and filters to route requests and modify requests and responses. It is lightweight and built on reactive principles to enable it to scale to thousands of routes.
This document appears to be from a VMware Tanzu Developer Connect presentation. It discusses Tanzu Application Platform (TAP), which provides a developer experience on Kubernetes across multiple clouds. TAP aims to unlock developer productivity, build rapid paths to production, and coordinate the work of development, security and operations teams. It offers features like pre-configured templates, integrated developer tools, centralized visibility and workload status, role-based access control, automated pipelines and built-in security. The presentation provides examples of how these capabilities improve experiences for developers, operations teams and security teams.
The document provides information about a Tanzu Developer Connect Workshop on Tanzu Application Platform. The agenda includes welcome and introductions on Tanzu Application Platform, followed by interactive hands-on workshops on the developer experience and operator experience. It will conclude with a quiz, prizes and giveaways. The document discusses challenges with developing on Kubernetes and how Tanzu Application Platform aims to improve the developer experience with features like pre-configured templates, developer tools integration, rapid iteration and centralized management.
The Tanzu Developer Connect is a hands-on workshop that dives deep into TAP. Attendees receive a hands on experience. This is a great program to leverage accounts with current TAP opportunities.
The Tanzu Developer Connect is a hands-on workshop that dives deep into TAP. Attendees receive a hands on experience. This is a great program to leverage accounts with current TAP opportunities.
Simplify and Scale Enterprise Apps in the Cloud | Dallas 2023VMware Tanzu
This document discusses simplifying and scaling enterprise Spring applications in the cloud. It provides an overview of Azure Spring Apps, which is a fully managed platform for running Spring applications on Azure. Azure Spring Apps handles infrastructure management and application lifecycle management, allowing developers to focus on code. It is jointly built, operated, and supported by Microsoft and VMware. The document demonstrates how to create an Azure Spring Apps service, create an application, and deploy code to the application using three simple commands. It also discusses features of Azure Spring Apps Enterprise, which includes additional capabilities from VMware Tanzu components.
SpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring BootVMware Tanzu
The document discusses 15 factors for building cloud native applications with Kubernetes based on the 12 factor app methodology. It covers factors such as treating code as immutable, externalizing configuration, building stateless and disposable processes, implementing authentication and authorization securely, and monitoring applications like space probes. The presentation aims to provide an overview of the 15 factors and demonstrate how to build cloud native applications using Kubernetes based on these principles.
SpringOne Tour: The Influential Software EngineerVMware Tanzu
The document discusses the importance of culture in software projects and how to influence culture. It notes that software projects involve people and personalities, not just technology. It emphasizes that culture informs everything a company does and is very difficult to change. It provides advice on being aware of your company's culture, finding ways to inculcate good cultural values like writing high-quality code, and approaches for influencing decision makers to prioritize culture.
SpringOne Tour: Domain-Driven Design: Theory vs PracticeVMware Tanzu
This document discusses domain-driven design, clean architecture, bounded contexts, and various modeling concepts. It provides examples of an e-scooter reservation system to illustrate domain modeling techniques. Key topics covered include identifying aggregates, bounded contexts, ensuring single sources of truth, avoiding anemic domain models, and focusing on observable domain behaviors rather than implementation details.
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on automated letter generation for Bonterra Impact Management using Google Workspace or Microsoft 365.
Interested in deploying letter generation automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
Generating privacy-protected synthetic data using Secludy and MilvusZilliz
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Tatiana Kojar
Skybuffer AI, built on the robust SAP Business Technology Platform (SAP BTP), is the latest and most advanced version of our AI development, reaffirming our commitment to delivering top-tier AI solutions. Skybuffer AI harnesses all the innovative capabilities of the SAP BTP in the AI domain, from Conversational AI to cutting-edge Generative AI and Retrieval-Augmented Generation (RAG). It also helps SAP customers safeguard their investments into SAP Conversational AI and ensure a seamless, one-click transition to SAP Business AI.
With Skybuffer AI, various AI models can be integrated into a single communication channel such as Microsoft Teams. This integration empowers business users with insights drawn from SAP backend systems, enterprise documents, and the expansive knowledge of Generative AI. And the best part of it is that it is all managed through our intuitive no-code Action Server interface, requiring no extensive coding knowledge and making the advanced AI accessible to more users.
GraphRAG for Life Science to increase LLM accuracyTomaz Bratanic
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxSitimaJohn
Ocean Lotus cyber threat actors represent a sophisticated, persistent, and politically motivated group that poses a significant risk to organizations and individuals in the Southeast Asian region. Their continuous evolution and adaptability underscore the need for robust cybersecurity measures and international cooperation to identify and mitigate the threats posed by such advanced persistent threat groups.
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
Taking AI to the Next Level in Manufacturing.pdfssuserfac0301
Read Taking AI to the Next Level in Manufacturing to gain insights on AI adoption in the manufacturing industry, such as:
1. How quickly AI is being implemented in manufacturing.
2. Which barriers stand in the way of AI adoption.
3. How data quality and governance form the backbone of AI.
4. Organizational processes and structures that may inhibit effective AI adoption.
6. Ideas and approaches to help build your organization's AI strategy.
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on integration of Salesforce with Bonterra Impact Management.
Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Your One-Stop Shop for Python Success: Top 10 US Python Development Providersakankshawande
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
Skybuffer SAM4U tool for SAP license adoptionTatiana Kojar
Manage and optimize your license adoption and consumption with SAM4U, an SAP free customer software asset management tool.
SAM4U, an SAP complimentary software asset management tool for customers, delivers a detailed and well-structured overview of license inventory and usage with a user-friendly interface. We offer a hosted, cost-effective, and performance-optimized SAM4U setup in the Skybuffer Cloud environment. You retain ownership of the system and data, while we manage the ABAP 7.58 infrastructure, ensuring fixed Total Cost of Ownership (TCO) and exceptional services through the SAP Fiori interface.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
40. Learn More. Stay Connected.
Vote for [CALCITE-2059] and don’t miss:
Simplifying Apache Geode with Spring Data
Exploring Data-Driven, Cognitive Capabilities in Pivotal Cloud Foundry
Orchestrating Data Microservices with Spring Cloud Data Flow
40
#springone@s1p