This document discusses building Java applications with Apache Cassandra. It provides an overview of building and deploying a sample auction application using Cassandra for storage. It covers designing Cassandra schemas for users, auctions, and bids, and using the DataStax Java driver mapping API to interact with Cassandra from Java code. The document also compares Spring Boot and Dropwizard frameworks for building Cassandra-backed applications.
Azure Table Storage: The Good, the Bad, the Ugly (15 min. lightning talk)Sirar Salih
Azure Table storage, a NoSQL data service in the cloud. Schemaless and with JSON compatibility, it’s simple and it does its job well. But everything great has its pitfalls.
Join in this lightning talk to look at and investigate the wonders and the mysteries, the shocks and the no-nos of using Azure Table storage. We will look at sample code, setting up and using the storage in action. Most notably, the program also looks at performance metrics, comparing Azure Table storage to other data services. Is this the thing for you? Find out!
Progressive Web Apps are one of the hottest things to come to the web platform in years, but how much of it is just hot air? When can you actually start shipping these things? Decades ago! In a hands on presentation, I'll show how PWAs are truly meant to be progressive - building on an evolution of web technologies nearly as old as the web itself, and still let you ship one of the most performant and cutting edge web apps around.
Kief Morris - Infrastructure is terribleThoughtworks
Why is nearly every infrastructure project I've run across a big ball of mud? We're still in the early days of infrastructure as code tooling, so we're struggling with messy glue code, configuration files, and weird custom scripts and tools. What can you do on your project to cope with the current state of tooling? And what should we, as an industry, do to level up?
Raquel Guimaraes- Third party infrastructure as codeThoughtworks
While implementing cloud Infrastructure as Code you might have come across the problem of dealing with third-party resources. This is most common in complex environments where most of the resources live in a cloud provider (GCP or AWS for example) and there are some SaaS solutions to integrate with (Datadog and Pingdom for example). In this talk we will expose the problem and explain a solution that is currently being used by one of our key clients in Spain.
SenchaCon 2016: The Once and Future Grid - Nige WhiteSencha
The Ext JS Grid has been a powerhouse for years, used by thousands of enterprises to deliver robust, data-rich applications to the web. The Ext JS Grid for the Modern Toolkit builds on these years of experience and leverages the full power of HTML5 and CSS3 to provide an extremely flexible and efficient grid for the modern era. In this session, we'll explore some of the key architectural advantages of the Modern Grid. Come and see how you can take advantage of these capabilities to tame mountains of data and give your users the world-class experience they demand.
MongoDB .local Toronto 2019: Using Change Streams to Keep Up with Your DataMongoDB
Immediate feedback is an essential part of modern application development where developers want to sync across platforms, systems, and users to provide better end-user experiences. Change streams empower developers to easily leverage the power of MongoDB's internal real-time functionality to react to relevant data changes immediately. Change streams also provide the backbone of MongoDB Atlas triggers. This session introduces change streams and walks you through developing with them. We will discuss use cases, integrating with Kafka, and explore how to make good architectural decisions around this new functionality.
TSAR (TimeSeries AggregatoR) Tech TalkAnirudh Todi
Twitter's 250 million users generate tens of billions of tweet views per day. Aggregating these events in real time - in a robust enough way to incorporate into our products - presents a massive scaling challenge. In this presentation I introduce TSAR (the TimeSeries AggregatoR), a robust, flexible, and scalable service for real-time event aggregation designed to solve this problem and a range of similar ones. I discuss how we built TSAR using Python and Scala from the ground up, almost entirely on open-source technologies (Storm, Summingbird, Kafka, Aurora, and others), and describe some of the challenges we faced in scaling it to process tens of billions of events per day.
Azure Table Storage: The Good, the Bad, the Ugly (15 min. lightning talk)Sirar Salih
Azure Table storage, a NoSQL data service in the cloud. Schemaless and with JSON compatibility, it’s simple and it does its job well. But everything great has its pitfalls.
Join in this lightning talk to look at and investigate the wonders and the mysteries, the shocks and the no-nos of using Azure Table storage. We will look at sample code, setting up and using the storage in action. Most notably, the program also looks at performance metrics, comparing Azure Table storage to other data services. Is this the thing for you? Find out!
Progressive Web Apps are one of the hottest things to come to the web platform in years, but how much of it is just hot air? When can you actually start shipping these things? Decades ago! In a hands on presentation, I'll show how PWAs are truly meant to be progressive - building on an evolution of web technologies nearly as old as the web itself, and still let you ship one of the most performant and cutting edge web apps around.
Kief Morris - Infrastructure is terribleThoughtworks
Why is nearly every infrastructure project I've run across a big ball of mud? We're still in the early days of infrastructure as code tooling, so we're struggling with messy glue code, configuration files, and weird custom scripts and tools. What can you do on your project to cope with the current state of tooling? And what should we, as an industry, do to level up?
Raquel Guimaraes- Third party infrastructure as codeThoughtworks
While implementing cloud Infrastructure as Code you might have come across the problem of dealing with third-party resources. This is most common in complex environments where most of the resources live in a cloud provider (GCP or AWS for example) and there are some SaaS solutions to integrate with (Datadog and Pingdom for example). In this talk we will expose the problem and explain a solution that is currently being used by one of our key clients in Spain.
SenchaCon 2016: The Once and Future Grid - Nige WhiteSencha
The Ext JS Grid has been a powerhouse for years, used by thousands of enterprises to deliver robust, data-rich applications to the web. The Ext JS Grid for the Modern Toolkit builds on these years of experience and leverages the full power of HTML5 and CSS3 to provide an extremely flexible and efficient grid for the modern era. In this session, we'll explore some of the key architectural advantages of the Modern Grid. Come and see how you can take advantage of these capabilities to tame mountains of data and give your users the world-class experience they demand.
MongoDB .local Toronto 2019: Using Change Streams to Keep Up with Your DataMongoDB
Immediate feedback is an essential part of modern application development where developers want to sync across platforms, systems, and users to provide better end-user experiences. Change streams empower developers to easily leverage the power of MongoDB's internal real-time functionality to react to relevant data changes immediately. Change streams also provide the backbone of MongoDB Atlas triggers. This session introduces change streams and walks you through developing with them. We will discuss use cases, integrating with Kafka, and explore how to make good architectural decisions around this new functionality.
TSAR (TimeSeries AggregatoR) Tech TalkAnirudh Todi
Twitter's 250 million users generate tens of billions of tweet views per day. Aggregating these events in real time - in a robust enough way to incorporate into our products - presents a massive scaling challenge. In this presentation I introduce TSAR (the TimeSeries AggregatoR), a robust, flexible, and scalable service for real-time event aggregation designed to solve this problem and a range of similar ones. I discuss how we built TSAR using Python and Scala from the ground up, almost entirely on open-source technologies (Storm, Summingbird, Kafka, Aurora, and others), and describe some of the challenges we faced in scaling it to process tens of billions of events per day.
The OpenStack Horizon project provides a web-based User Interface to OpenStack services. It is constructed in two parts: (1) a core set of libraries for implementing a Dashboard; (2) the dashboard implementation that uses the core set of libraries.
Horizon uses python django — server side technology
Django is a wonderful framework, but a little dated. Pre-dates the rise in client-side and single page applications.
Javascript is used for enhancing the user experience
In the time since Horizon was first architected, there have been major advances in the design, and best practices for web applications. In particular, the use of more sophisticated and robust client-side javascript frameworks like BackboneJS, AngularJS, MeteorJS, have come to the fore.
These applications provide a much more responsive user experience, much cleaner separation between the client and server, enable configuration driven interfaces, and facilitate more modular testing.
This in turn, results in shorter development cycles, more testable software, and above all, a better user experience.
In this presentation, we share some of our recent work in re-architecting parts of Horizon to take advantage of these new technologies. We discuss some of the technologies we use, our application architecture, and some of the pitfalls to avoid.
SenchaCon 2016: Add Magic to Your Ext JS Apps with D3 Visualizations - Vitaly...Sencha
Ext JS provides easy-to-use charting components that satisfy common needs, but sometimes you want to deliver an exceptional, unique user experience. This presentation will discuss how Ext JS leverages the popular and extremely powerful D3 library to create sophisticated, data-driven visualizations. This functionality helps your users understand the story behind their data, so they can make informed decisions.
Closing the Loop in Extended Reality with Kafka Streams and Machine Learning ...confluent
We’ve built a real-time streaming platform that enables prediction based on user behavior, with events occurring in virtual and augmented reality environments. The solution enables organizations to train people in an extended reality environment, where real-life training may be costly and dangerous. Kafka Streams enables analyzing spatial and event data to detect gestural feature and analyze user behavior in real-time to be able to predict any future mistake the user might make. Kafka is the backbone of our real-time analytics and extended reality communication platform with our cluster and applications being deployed on Kubernetes.
In this talk, we will mainly focus on the following: 1. Why Extended Reality with Kafka is a step in the right direction. 2. Architecture & Power of Schema Registry in building a generic platform for pluggable XR apps and analytics models 3. How KSQL and Kafka Streams fits in Kafka Ecosystem to help analyze human motion data and detect features for real-time prediction. 4. Demo of a VR application with real-time analytics feedback, which assists people to be trained in how to work with chemical laboratory equipment.
Neolane API Custom SOAP request handlerDavid Garcia
Concept that extends Adobe Campaign Classic API (soaprouter.jsp) to process and store SOAP request data in custom schemas.
By David Garcia MarkTech Ltd
The OpenStack project was launched by Rackspace and NASA in July 2010. Since then it has gained considerable momentum, with over 200 companies joining the project.
Horizon is OpenStack’s web-based user interface. It is constructed in two parts: 1) a core set of libraries for implementing a dashboard; 2) a reference dashboard implementation that uses the core set of libraries. Customization is key to the Horizon framework. It allows developers to construct their own dashboards, panel groups, and panels, and assemble them via a common navigation/presentation framework.
In this presentation, David will provide a brief introduction to OpenStack and Horizon. Then he will review Horizon’s architecture, explain how it integrates with other OpenStack services, examine its most interesting features, and describe how to start developing with it.
This is a presentation given on October 24 by Michael Uzquiano of Cloud CMS (http://www.cloudcms.com) at the MongoDB Boston conference.
In this presentation, we cover Hazelcast - an in-memory data grid that provides distributed object persistence across multiple nodes in a cluster. When backed by MongoDB, objects are naturally written to Mongo by Hazelcast. The integration points are clean and easy to implement.
We cover a few simple cases along with code samples to provide the MongoDB community with some ideas of how to integrate Hazelcast into their own MongoDB Java applications.
MongoDB .local Paris 2020: Adéo @MongoDB : MongoDB Atlas & Leroy Merlin : et ...MongoDB
Adeo et en particulier Leroy Merlin utilisent massivement MongoDB pour propulser de nombreuses applications et en particulier son site web leroymerlin.fr.
Emmanuel Dieval Ingénieur Software chez ADEO, présentera le nouveau système au coeur de la publication de l'offre Leroy Merlin: OPUS.
OPUS s'appuie particulièrement sur MongoDB pour la construction des pages de famille de produits tout en supportant un important flux de données journalier.
Après un rappel sur les pipelines d'agrégation et une présentation de MongoDB Atlas par Maxime Beugnet, Developer Advocate chez MongoDB, Emmanuel parlera de l'utilisation des pipelines d'agrégation pour la construction des pages de famille de produits, mais aussi de Google Cloud Platform et des avantages à utiliser MongoDB Atlas.
The OpenStack Horizon project provides a web-based User Interface to OpenStack services. It is constructed in two parts: (1) a core set of libraries for implementing a Dashboard; (2) the dashboard implementation that uses the core set of libraries.
Horizon uses python django — server side technology
Django is a wonderful framework, but a little dated. Pre-dates the rise in client-side and single page applications.
Javascript is used for enhancing the user experience
In the time since Horizon was first architected, there have been major advances in the design, and best practices for web applications. In particular, the use of more sophisticated and robust client-side javascript frameworks like BackboneJS, AngularJS, MeteorJS, have come to the fore.
These applications provide a much more responsive user experience, much cleaner separation between the client and server, enable configuration driven interfaces, and facilitate more modular testing.
This in turn, results in shorter development cycles, more testable software, and above all, a better user experience.
In this presentation, we share some of our recent work in re-architecting parts of Horizon to take advantage of these new technologies. We discuss some of the technologies we use, our application architecture, and some of the pitfalls to avoid.
SenchaCon 2016: Add Magic to Your Ext JS Apps with D3 Visualizations - Vitaly...Sencha
Ext JS provides easy-to-use charting components that satisfy common needs, but sometimes you want to deliver an exceptional, unique user experience. This presentation will discuss how Ext JS leverages the popular and extremely powerful D3 library to create sophisticated, data-driven visualizations. This functionality helps your users understand the story behind their data, so they can make informed decisions.
Closing the Loop in Extended Reality with Kafka Streams and Machine Learning ...confluent
We’ve built a real-time streaming platform that enables prediction based on user behavior, with events occurring in virtual and augmented reality environments. The solution enables organizations to train people in an extended reality environment, where real-life training may be costly and dangerous. Kafka Streams enables analyzing spatial and event data to detect gestural feature and analyze user behavior in real-time to be able to predict any future mistake the user might make. Kafka is the backbone of our real-time analytics and extended reality communication platform with our cluster and applications being deployed on Kubernetes.
In this talk, we will mainly focus on the following: 1. Why Extended Reality with Kafka is a step in the right direction. 2. Architecture & Power of Schema Registry in building a generic platform for pluggable XR apps and analytics models 3. How KSQL and Kafka Streams fits in Kafka Ecosystem to help analyze human motion data and detect features for real-time prediction. 4. Demo of a VR application with real-time analytics feedback, which assists people to be trained in how to work with chemical laboratory equipment.
Neolane API Custom SOAP request handlerDavid Garcia
Concept that extends Adobe Campaign Classic API (soaprouter.jsp) to process and store SOAP request data in custom schemas.
By David Garcia MarkTech Ltd
The OpenStack project was launched by Rackspace and NASA in July 2010. Since then it has gained considerable momentum, with over 200 companies joining the project.
Horizon is OpenStack’s web-based user interface. It is constructed in two parts: 1) a core set of libraries for implementing a dashboard; 2) a reference dashboard implementation that uses the core set of libraries. Customization is key to the Horizon framework. It allows developers to construct their own dashboards, panel groups, and panels, and assemble them via a common navigation/presentation framework.
In this presentation, David will provide a brief introduction to OpenStack and Horizon. Then he will review Horizon’s architecture, explain how it integrates with other OpenStack services, examine its most interesting features, and describe how to start developing with it.
This is a presentation given on October 24 by Michael Uzquiano of Cloud CMS (http://www.cloudcms.com) at the MongoDB Boston conference.
In this presentation, we cover Hazelcast - an in-memory data grid that provides distributed object persistence across multiple nodes in a cluster. When backed by MongoDB, objects are naturally written to Mongo by Hazelcast. The integration points are clean and easy to implement.
We cover a few simple cases along with code samples to provide the MongoDB community with some ideas of how to integrate Hazelcast into their own MongoDB Java applications.
MongoDB .local Paris 2020: Adéo @MongoDB : MongoDB Atlas & Leroy Merlin : et ...MongoDB
Adeo et en particulier Leroy Merlin utilisent massivement MongoDB pour propulser de nombreuses applications et en particulier son site web leroymerlin.fr.
Emmanuel Dieval Ingénieur Software chez ADEO, présentera le nouveau système au coeur de la publication de l'offre Leroy Merlin: OPUS.
OPUS s'appuie particulièrement sur MongoDB pour la construction des pages de famille de produits tout en supportant un important flux de données journalier.
Après un rappel sur les pipelines d'agrégation et une présentation de MongoDB Atlas par Maxime Beugnet, Developer Advocate chez MongoDB, Emmanuel parlera de l'utilisation des pipelines d'agrégation pour la construction des pages de famille de produits, mais aussi de Google Cloud Platform et des avantages à utiliser MongoDB Atlas.
Al Tobey (@AlTobey) is an Open Source Mechanic at DataStax. Prior to working at DataStax, Al was a Tech Lead of Compute and Data Services at Ooyala, which has been using Apache Cassandra since version 0.4 and these days uses Go in production.
Al will be presenting a brief introduction to Go (#golang) and Cassandra, and how they are a great fit for each other. This talk will include code samples and a live demo.
AWS user group Serverless in September - Chris Johnson Bidler "Go Serverless ...AWS Chicago
September 19th joint meetup with Serverless Chicago user group at RedShelf - Serverless in AWS.
"Go Serverless from your iPad: Building a Data-driven REST API with AWS CodeStar, Lambda, and Cognitect’s Datomic and Vase" - Chris Johnson Bidler, CTO at Centriq Technology, Inc
KSQL is a stream processing SQL engine, which allows stream processing on top of Apache Kafka. KSQL is based on Kafka Stream and provides capabilities for consuming messages from Kafka, analysing these messages in near-realtime with a SQL like language and produce results again to a Kafka topic. By that, no single line of Java code has to be written and you can reuse your SQL knowhow. This lowers the bar for starting with stream processing significantly.
KSQL offers powerful capabilities of stream processing, such as joins, aggregations, time windows and support for event time. In this talk I will present how KSQL integrates with the Kafka ecosystem and demonstrate how easy it is to implement a solution using KSQL for most part. This will be done in a live demo on a fictitious IoT sample.
Importing Data into Neo4j quickly and easily - StackOverflowNeo4j
In this GraphConnect presentation Mark and Michael show several ways to import large amounts of highly connected data from different formats into Neo4j. Both Cypher's LOAD CSV as well as the bulk importer is demonstrated along with many tips.
We use the well know StackOverflow Q&A site data which is interestingly very graphy.
Hashicorp’s Terraform provides a declarative notation (like Puppet) to describe various Cloud resources. It is an open-source tool, provider-independent, and thus able to combine resources from multiple cloud platforms and to be extended through plugins. The talk demonstrates how to describe web application infrastructure with Terraform, showing how easily all related components can be started, updated, and stopped.
DataStax: Making Cassandra Fail (for effective testing)DataStax Academy
Interacting with a distributed database is inherently more complex than with a single server database. Add in a few datacenters and some network issues and things get even more hairy.
I'm a firm believer in automated tests, functionality of software that is not covered by an automated test is nothing more than a rumor.
If we want to build fault tolerant applications with Cassandra we better start testing the faults. The first step is to understand them, so we'll go through in detail:
- Read timeouts
- Write timeouts
- Unavailable
- Connectivity issues between the driver and the coordinator
Understanding these scenarios lets you decide when you should be re-trying your queries. Then how do we test our application under these scenarios? I'll go through three approaches you can use:
- Inserting a proxy between the driver and the cluster to drop traffic
- Getting into the Cassandra code and overriding the QueryHandler to inject faults (see https://github.com/chbatey/cassandra-killr)
- Stubbing Cassandra out at the protocol level (http://www.scassandra.org/)
You should leave this talk with an appreciation of the failures you can get from the driver and what you should do about them, and hopefully you'll be inspired to test all these scenarios.
Forrester CXNYC 2017 - Delivering great real-time cx is a true craftDataStax Academy
Companies today are innovating with real-time data to deliver truly amazing customer experiences in the moment. Real-time data management for real-time customer experience is core to staying ahead of competition and driving revenue growth. Join Trays to learn how Comcast is differentiating itself from it's own historical reputation with Customer Experience strategies.
Introduction to DataStax Enterprise Graph DatabaseDataStax Academy
DataStax Enterprise (DSE) Graph is a built to manage, analyze, and search highly connected data. DSE Graph, built on NoSQL Apache Cassandra delivers continuous uptime along with predictable performance and scales for modern systems dealing with complex and constantly changing data.
Download DataStax Enterprise: Academy.DataStax.com/Download
Start free training for DataStax Enterprise Graph: Academy.DataStax.com/courses/ds332-datastax-enterprise-graph
Introduction to DataStax Enterprise Advanced Replication with Apache CassandraDataStax Academy
DataStax Enterprise Advanced Replication supports one-way distributed data replication from remote database clusters that might experience periods of network or internet downtime. Benefiting use cases that require a 'hub and spoke' architecture.
Learn more at http://www.datastax.com/2016/07/stay-100-connected-with-dse-advanced-replication
Advanced Replication docs – https://docs.datastax.com/en/latest-dse/datastax_enterprise/advRep/advRepTOC.html
Data Modeling is the one of the first things to sink your teeth into when trying out a new database. That's why we are going to cover this foundational topic in enough detail for you to get dangerous. Data Modeling for relational databases is more than a touch different than the way it's approached with Cassandra. We will address the quintessential query-driven methodology through a couple of different use cases, including working with time series data for IoT. We will also demo a new tool to get you bootstrapped quickly with MovieLens sample data. This talk should give you the basics you need to get serious with Apache Cassandra.
Hear about how Coursera uses Cassandra as the core of its scalable online education platform. I'll discuss the strengths of Cassandra that we leverage, as well as some limitations that you might run into as well in practice.
In the second part of this talk, we'll dive into how best to effectively use the Datastax Java drivers. We'll dig into how the driver is architected, and use this understanding to develop best practices to follow. I'll also share a couple of interesting bug we've run into at Coursera.
Cassandra @ Sony: The good, the bad, and the ugly part 1DataStax Academy
This talk covers scaling Cassandra to a fast growing user base. Alex and Isaias will cover new best practices and how to work with the strengths and weaknesses of Cassandra at large scale. They will discuss how to adapt to bottlenecks while providing a rich feature set to the playstation community.
Cassandra @ Sony: The good, the bad, and the ugly part 2DataStax Academy
This talk covers scaling Cassandra to a fast growing user base. Alex and Isaias will cover new best practices and how to work with the strengths and weaknesses of Cassandra at large scale. They will discuss how to adapt to bottlenecks while providing a rich feature set to the playstation community.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Enhancing Performance with Globus and the Science DMZGlobus
ESnet has led the way in helping national facilities—and many other institutions in the research community—configure Science DMZs and troubleshoot network issues to maximize data transfer performance. In this talk we will present a summary of approaches and tips for getting the most out of your network infrastructure using Globus Connect Server.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
2. @chbatey
Who am I?
• Technical Evangelist for Apache Cassandra
•Founder of Stubbed Cassandra
•Help out Apache Cassandra users
• DataStax
•Builds enterprise ready version of Apache
Cassandra
• Previous: Cassandra backed apps at BSkyB
3. @chbatey
Agenda
• Everything I wish I knew before I started
• Example application: Auction service
- Build + Deploy
- Tech stack: Spring vs Dropwizard
- Configuration
- Designing schemas
- Example Driver code including LWTs
- Continuous Integration + Testing strategies
23. @chbatey
Languages
• DataStax (open source)
- C#, Java, C++, Python, Node, Ruby
- Very similar programming API
• Other open source
- Go
- Clojure
- Erlang
- Haskell
- Many more Java/Python drivers
- Perl
33. How it is stored on disk
customer
_id
time event_type store_type tags
charles 2014-11-18 16:52:04 basket_add online {'item': 'coffee'}
charles 2014-11-18 16:53:00 basket_add online {'item': ‘wine'}
charles 2014-11-18 16:53:09 logout online {}
chbatey 2014-11-18 16:52:21 login online {}
chbatey 2014-11-18 16:53:21 basket_add online {'item': 'coffee'}
chbatey 2014-11-18 16:54:00 basket_add online {'item': 'cheese'}
charles
event_type
basket_add
staff_id
n/a
store_type
online
tags:item
coffee
event_type
basket_add
staff_id
n/a
store_type
online
tags:item
wine
event_type
logout
staff_id
n/a
store_type
online
chbatey
event_type
login
staff_id
n/a
store_type
online
event_type
basket_add
staff_id
n/a
store_type
online
tags:item
coffee
event_type
basket_add
staff_id
n/a
store_type
online
tags:item
cheese
34. @chbatey
Requirements
• Store users
• Store auction metadata
• Store auction bids
Lowish throughput
High throughput / reads of many rows
Get top N bids quickly
35. @chbatey
Designing a schema for users
• Fields
- Username
- First name
- Last name
- Email addresses?
• Unique user names
39. @chbatey
AuctionDao
CREATE TABLE IF NOT EXISTS killrauction.auctions (
name text primary key,
owner text,
ends bigint);
@PostConstruct
public void prepareStatements() {
createAuction = session.prepare("insert INTO auctions (name, owner, ends) VALUES (?, ?, ?)");
getAuction = session.prepare("select * from auctions where name = ?");
getAllAuctionSparse = session.prepare("select * from auctions");
}
UUID for auction or use the name? Raise a PR :)
40. @chbatey
Auction Bids
CREATE TABLE IF NOT EXISTS killrauction.auction_bids (
name TEXT,
bid_time TIMEUUID,
bid_user TEXT,
bid_amount BIGINT,
PRIMARY KEY (name, bid_amount, bid_time ) )
WITH CLUSTERING ORDER BY (bid_amount DESC )
All bids for the same auction on the same node
Bids stored on disk in order of amount
Descending so we can always get the top N bids
47. @chbatey
Mapping API
@Accessor
public interface CustomerEventDao {
@Query("select * from customers.customer_events where customer_id = :customerId")
Result<CustomerEvent> getCustomerEvents(String customerId);
@Query("select * from customers.customer_events")
Result<CustomerEvent> getAllCustomerEvents();
}
@Bean
public CustomerEventDao customerEventDao() {
MappingManager mappingManager = new MappingManager(session);
return mappingManager.createAccessor(CustomerEventDao.class);
}
48. @chbatey
Adding some type safety
public enum StoreType {
ONLINE, RETAIL, FRANCHISE, MOBILE
}
@Table(keyspace = "customers", name = "customer_events")
public class CustomerEvent {
@PartitionKey
@Column(name = "customer_id")
private String customerId;
@ClusteringColumn()
private UUID time;
@Column(name = "staff_id")
private String staffId;
@Column(name = "store_type")
@Enumerated(EnumType.STRING) // could be EnumType.ORDINAL
private StoreType storeType;
49. @chbatey
User defined types
create TYPE store (name text, type text, postcode text) ;
CREATE TABLE customer_events_type(
customer_id text,
staff_id text,
time timeuuid,
store frozen<store>,
event_type text,
tags map<text, text>,
PRIMARY KEY ((customer_id), time));
50. @chbatey
Mapping user defined types
@UDT(keyspace = "customers", name = "store")
public class Store {
private String name;
private StoreType type;
private String postcode;
// getters etc
}
@Table(keyspace = "customers", name = "customer_events_type")
public class CustomerEventType {
@PartitionKey
@Column(name = "customer_id")
private String customerId;
@ClusteringColumn()
private UUID time;
@Column(name = "staff_id")
private String staffId;
@Frozen
private Store store;
@Column(name = "event_type")
private String eventType;
private Map<String, String> tags;
51. @chbatey
Mapping user defined types
@UDT(keyspace = "customers", name = "store")
public class Store {
private String name;
private StoreType type;
private String postcode;
// getters etc
}
@Table(keyspace = "customers", name = "customer_events_type")
public class CustomerEventType {
@PartitionKey
@Column(name = "customer_id")
private String customerId;
@ClusteringColumn()
private UUID time;
@Column(name = "staff_id")
private String staffId;
@Frozen
private Store store;
@Column(name = "event_type")
private String eventType;
private Map<String, String> tags;
53. @chbatey
Development Env + Testing
• Cassandra Unit
- Embeds a single Cassandra node in the same JVM for unit
testing
• https://github.com/jsevellec/cassandra-unit
• Cassandra Cluster Manager
- Spins up small clusters locally
• https://github.com/pcmanus/ccm
• Stubbed Cassandra
- Pretends to be a real Cassandra
56. @chbatey
Example Cassandra Unit with JUnit rule
@Rule
public CassandraUnit cassandraUnitRule = new CassandraUnit(new
ClassPathXmlDataSet("extendedDataSet.xml"));
@Test
public void shouldHaveLoadAnExtendDataSet() throws Exception {
// test dao
}
<dependency>
<groupId>org.cassandraunit</groupId>
<artifactId>cassandra-unit</artifactId>
<version>2.1.3.1</version>
<scope>test</scope>
</dependency>
57. @chbatey
Cassandra cluster manager
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID Rack
UN 127.0.0.1 102.27 KB 256 ? 15ad7694-3e76-4b74-aea0-fa3c0fa59532 rack1
UN 127.0.0.2 102.18 KB 256 ? cca7d0bb-e884-49f9-b098-e38fbe895cbc rack1
UN 127.0.0.3 93.16 KB 256 ? 1f9737d3-c1b8-4df1-be4c-d3b1cced8e30 rack1
UN 127.0.0.4 102.1 KB 256 ? fe27b958-5d3a-4f78-9880-76cb7c9bead1 rack1
UN 127.0.0.5 93.18 KB 256 ? 66eb3f23-8889-44d6-a9e7-ecdd57ed61d0 rack1
UN 127.0.0.6 102.12 KB 256 ? e2e99a7b-c1fb-4f2a-9e4f-7a4666f8245e rack1
ccm create test -v 2.0.5 -n 6 -s
60. @chbatey
Starting / Stopping
@ClassRule
public static final ScassandraServerRule SCASSANDRA = new
ScassandraServerRule();
Scassandra scassandra = ScassandraFactory.createServer();
scassandra.start();
PrimingClient primingClient = scassandra.primingClient();
ActivityClient activityClient = scassandra.activityClient();
61. @chbatey
Activity Client
• Query
‣ Query text
‣ Consistency
• PrepreparedStatementExecution
‣ Prepared statement text
‣ Bound variables
public List<Query> retrieveQueries();
public List<PreparedStatementExecution> retrievePreparedStatementExecutions()
public List<Connection> retrieveConnections();
public void clearAllRecordedActivity()
public void clearConnections();
public void clearQueries();
public void clearPreparedStatementExecutions();
62. @chbatey
Priming Client
• PrimingRequest
‣ Either a Query or PreparedStatement
‣ Query text or QueryPattern (regex)
‣ Consistency (default all)
‣ Result (success, read timeout, unavailable etc)
‣ Rows for successful response
‣ Column types for rows
‣ Variable types for prepared statements
public void prime(PrimingRequest prime)
public List<PrimingRequest> retrievePreparedPrimes()
public List<PrimingRequest> retrieveQueryPrimes()
public void clearAllPrimes()
public void clearQueryPrimes()
public void clearPreparedPrimes()
64. @chbatey
Testing slow connection
@Test(expected = UnableToSavePersonException.class)
public void testThatSlowQueriesTimeout() throws Exception {
// given
PrimingRequest preparedStatementPrime = PrimingRequest.preparedStatementBuilder()
.withQueryPattern("insert into person.*")
.withVariableTypes(VARCHAR, INT, list(TIMESTAMP))
.withFixedDelay(1000)
.build();
primingClient.prime(preparedStatementPrime);
underTest.connect();
underTest.storePerson(new Person("Christopher", 29, Collections.emptyList()));
}
Delays the response by 1000ms
Expect a custom exception
65. @chbatey
Continuous integration
Unit tests
cassandra unit
stubbed
cassandra
Deploy application to
isolated test
environment with
scaled down
cassandra cluster
Deploy to stage
environment with
same hardware as
production for
capacity testing
Deploy to production
Deploy to stage
integrated
environment +
run E2E tests
68. @chbatey
Summary
• Know the eco-system
- Community: Get on IRC / JIRA
- Tools: DataStax drivers, CCM, Cassandra-unit, DevCenter
• Get used to looking at trace
69. @chbatey
Thanks for listening
• Follow me on twitter @chbatey
• Cassandra + Fault tolerance posts a plenty:
• http://christopher-batey.blogspot.co.uk/
• Cassandra resources: http://planetcassandra.org/