This is an introduction to relational and non-relational databases and how their performance affects scaling a web application.
This is a recording of a guest Lecture I gave at the University of Texas school of Information.
In this talk I address the technologies and tools Gowalla (gowalla.com) uses including memcache, redis and cassandra.
Find more on my blog:
http://schneems.com
The rise of NoSQL is characterized with confusion and ambiguity; very much like any fast-emerging organic movement in the absence of well-defined standards and adequate software solutions. Whether you are a developer or an architect, many questions come to mind when faced with the decision of where your data should be stored and how it should be managed. The following are some of these questions: What does the rise of all these NoSQL technologies mean to my enterprise? What is NoSQL to begin with? Does it mean "No SQL"? Could this be just another fad? Is it a good idea to bet the future of my enterprise on these new exotic technologies and simply abandon proven mature Relational DataBase Management Systems (RDBMS)? How scalable is scalable? Assuming that I am sold, how do I choose the one that fit my needs best? Is there a middle ground somewhere? What is this Polyglot Persistence I hear about? The answers to these questions and many more is the subject of this talk along with a survey of the most popular of NoSQL technologies. Be there or be square.
Introduction to CosmosDB - Azure Bootcamp 2018Josh Carlisle
[Session Abstract] Cosmos DB is a globally distributed, multi-model, Serverless, NoSQL database solution that runs on Microsoft Azure. With guaranteed SLAs, various consistency models, and support for multiple APIs, Cosmos DB can have many advantages over common relational database solutions. However, the shift to NoSQL in addition to the numerous configuration options available in Cosmos DB can be a challenge for traditional relational database developers. In this talk, we will take an existing application built on a traditional relational database and update the solution to take advantage of Cosmos DB. Along the way, we will have many decisions to make including which API we should use, how best to model our data, which consistency model to use, and how our data should be indexed, partitioned, and organized. By the end of the talk you should have familiarity with the decisions you will need to make to successfully implement your own solutions on Cosmos DB.
Polyglot Persistence - Two Great Tastes That Taste Great TogetherJohn Wood
The days of the relational database being a one-stop-shop for all of your persistence needs are over. Although NoSQL databases address some issues that can’t be addressed by relational databases, the opposite is true as well. The relational database offers an unparalleled feature set and rock solid stability. One cannot underestimate the importance of using the right tool for the job, and for some jobs, one tool is not enough. This talk focuses on the strength and weaknesses of both relational and NoSQL databases, the benefits and challenges of polyglot persistence, and examples of polyglot persistence in the wild.
These slides were presented at WindyCityDB 2010.
Slides from my talk at ACCU2011 in Oxford on 16th April 2011. A whirlwind tour of the non-relational database families, with a little more detail on Redis, MongoDB, Neo4j and HBase.
The rise of NoSQL is characterized with confusion and ambiguity; very much like any fast-emerging organic movement in the absence of well-defined standards and adequate software solutions. Whether you are a developer or an architect, many questions come to mind when faced with the decision of where your data should be stored and how it should be managed. The following are some of these questions: What does the rise of all these NoSQL technologies mean to my enterprise? What is NoSQL to begin with? Does it mean "No SQL"? Could this be just another fad? Is it a good idea to bet the future of my enterprise on these new exotic technologies and simply abandon proven mature Relational DataBase Management Systems (RDBMS)? How scalable is scalable? Assuming that I am sold, how do I choose the one that fit my needs best? Is there a middle ground somewhere? What is this Polyglot Persistence I hear about? The answers to these questions and many more is the subject of this talk along with a survey of the most popular of NoSQL technologies. Be there or be square.
Introduction to CosmosDB - Azure Bootcamp 2018Josh Carlisle
[Session Abstract] Cosmos DB is a globally distributed, multi-model, Serverless, NoSQL database solution that runs on Microsoft Azure. With guaranteed SLAs, various consistency models, and support for multiple APIs, Cosmos DB can have many advantages over common relational database solutions. However, the shift to NoSQL in addition to the numerous configuration options available in Cosmos DB can be a challenge for traditional relational database developers. In this talk, we will take an existing application built on a traditional relational database and update the solution to take advantage of Cosmos DB. Along the way, we will have many decisions to make including which API we should use, how best to model our data, which consistency model to use, and how our data should be indexed, partitioned, and organized. By the end of the talk you should have familiarity with the decisions you will need to make to successfully implement your own solutions on Cosmos DB.
Polyglot Persistence - Two Great Tastes That Taste Great TogetherJohn Wood
The days of the relational database being a one-stop-shop for all of your persistence needs are over. Although NoSQL databases address some issues that can’t be addressed by relational databases, the opposite is true as well. The relational database offers an unparalleled feature set and rock solid stability. One cannot underestimate the importance of using the right tool for the job, and for some jobs, one tool is not enough. This talk focuses on the strength and weaknesses of both relational and NoSQL databases, the benefits and challenges of polyglot persistence, and examples of polyglot persistence in the wild.
These slides were presented at WindyCityDB 2010.
Slides from my talk at ACCU2011 in Oxford on 16th April 2011. A whirlwind tour of the non-relational database families, with a little more detail on Redis, MongoDB, Neo4j and HBase.
For our eReader development project, we had to find a persistent storage for our JSON documents. After initial scanning we zeroed into two products DynamoDB and MongoDB. These slides take a deeper dive in the selection of our JSON data store.
NoSQL, as many of you may already know, is basically a database used to manage huge sets of unstructured data, where in the data is not stored in tabular relations like relational databases. Most of the currently existing Relational Databases have failed in solving some of the complex modern problems like:
• Continuously changing nature of data - structured, semi-structured, unstructured and polymorphic data.
• Applications now serve millions of users in different geo-locations, in different timezones and have to be up and running all the time, with data integrity maintained
• Applications are becoming more distributed with many moving towards cloud computing.
NoSQL plays a vital role in an enterprise application which needs to access and analyze a massive set of data that is being made available on multiple virtual servers (remote based) in the cloud infrastructure and mainly when the data set is not structured. Hence, the NoSQL database is designed to overcome the Performance, Scalability, Data Modelling and Distribution limitations that are seen in the Relational Databases.
In this talk from the Dublin Websummit 2014 AWS Technical Evangelist Danilo Poccia discusses NoSQL technology.
Includes an introduction to NoSQL DB and a discussion of when it is time to consider NoSQL.
Danilo also introduces Amazon DynamoDB as a NoSQL solution and talks through several case studies of customers that are using Amazon DynamoDB today.
SQL vs. NoSQL. It's always a hard choice.Denis Reznik
This will be an interesting and sometimes fun session with a small demo. This session will answer some of your questions and force you to think about new questions. It will not be very technical, so it's ok for choose another more technical session from the schedule :) But if will decide to come, I can assure you, that you will not be disappointed. We will do a thought experiment with one famous public high-loaded website, will look at advantages and disadvantages of SQL and NoSQL databases, and will choose the best database engine for it.
In 2013:
- 1.4 Trillion digital interactions happen per month.
- 2.9 million emails are sent every second.
- 72.9 products are ordered on Amazon per second.
That is a lot of connected data, graphs are truly everywhere. Companies are finding that graph database technology is helping them make sense of their big data.
Objectivity’s Nick Quinn, Chief Architect of InfiniteGraph, shows us just how popular graph databases have become and where they are being used, as well as showing us the ins and outs.
Do you want to build technology that does great things with big data? You might want to find out what your colleagues are Tweeting about, make recommendations for apps, music or other retail that result in higher purchase rates, discover hidden connections between new and recorded medical research data, or maybe even leverage intel across government agencies to catch the bad guys.
All this is possible with a graph database.
Benchmarking, Load Testing, and Preventing Terrible DisastersMongoDB
"Have you ever crossed your fingers before performing an upgrade or switching storage engines, because you weren't quite sure what would happen? Have you ever been bitten by a slight change in behavior that turned out to be unexpectedly significant for your workload? At Parse we have developed a workflow that lets us repeatedly capture and replay real production workloads offline. This has allowed us to confidently perform upgrades across a large fleet with a minimum amount of canarying, and has helped us load test a variety of storage engines with real workloads so we can compare and understand the performance tradeoffs.
In this talk we will cover best practices for upgrades and migrations, and we will walk through how to use our open-sourced tooling to demonstrate how you can do the same. We will also share some fun war stories about various disasters found and averted *before* putting them into production thanks to offline benchmarking."
For our eReader development project, we had to find a persistent storage for our JSON documents. After initial scanning we zeroed into two products DynamoDB and MongoDB. These slides take a deeper dive in the selection of our JSON data store.
NoSQL, as many of you may already know, is basically a database used to manage huge sets of unstructured data, where in the data is not stored in tabular relations like relational databases. Most of the currently existing Relational Databases have failed in solving some of the complex modern problems like:
• Continuously changing nature of data - structured, semi-structured, unstructured and polymorphic data.
• Applications now serve millions of users in different geo-locations, in different timezones and have to be up and running all the time, with data integrity maintained
• Applications are becoming more distributed with many moving towards cloud computing.
NoSQL plays a vital role in an enterprise application which needs to access and analyze a massive set of data that is being made available on multiple virtual servers (remote based) in the cloud infrastructure and mainly when the data set is not structured. Hence, the NoSQL database is designed to overcome the Performance, Scalability, Data Modelling and Distribution limitations that are seen in the Relational Databases.
In this talk from the Dublin Websummit 2014 AWS Technical Evangelist Danilo Poccia discusses NoSQL technology.
Includes an introduction to NoSQL DB and a discussion of when it is time to consider NoSQL.
Danilo also introduces Amazon DynamoDB as a NoSQL solution and talks through several case studies of customers that are using Amazon DynamoDB today.
SQL vs. NoSQL. It's always a hard choice.Denis Reznik
This will be an interesting and sometimes fun session with a small demo. This session will answer some of your questions and force you to think about new questions. It will not be very technical, so it's ok for choose another more technical session from the schedule :) But if will decide to come, I can assure you, that you will not be disappointed. We will do a thought experiment with one famous public high-loaded website, will look at advantages and disadvantages of SQL and NoSQL databases, and will choose the best database engine for it.
In 2013:
- 1.4 Trillion digital interactions happen per month.
- 2.9 million emails are sent every second.
- 72.9 products are ordered on Amazon per second.
That is a lot of connected data, graphs are truly everywhere. Companies are finding that graph database technology is helping them make sense of their big data.
Objectivity’s Nick Quinn, Chief Architect of InfiniteGraph, shows us just how popular graph databases have become and where they are being used, as well as showing us the ins and outs.
Do you want to build technology that does great things with big data? You might want to find out what your colleagues are Tweeting about, make recommendations for apps, music or other retail that result in higher purchase rates, discover hidden connections between new and recorded medical research data, or maybe even leverage intel across government agencies to catch the bad guys.
All this is possible with a graph database.
Benchmarking, Load Testing, and Preventing Terrible DisastersMongoDB
"Have you ever crossed your fingers before performing an upgrade or switching storage engines, because you weren't quite sure what would happen? Have you ever been bitten by a slight change in behavior that turned out to be unexpectedly significant for your workload? At Parse we have developed a workflow that lets us repeatedly capture and replay real production workloads offline. This has allowed us to confidently perform upgrades across a large fleet with a minimum amount of canarying, and has helped us load test a variety of storage engines with real workloads so we can compare and understand the performance tradeoffs.
In this talk we will cover best practices for upgrades and migrations, and we will walk through how to use our open-sourced tooling to demonstrate how you can do the same. We will also share some fun war stories about various disasters found and averted *before* putting them into production thanks to offline benchmarking."
This presentation explains the major differences between SQL and NoSQL databases in terms of Scalability, Flexibility and Performance. It also talks about MongoDB which is a document-based NoSQL database and explains the database strutre for my mouse-human research classifier project.
The relational database has been the dominant database model for many years. However, a new model called NoSQL is gaining significant attention. NoSQL DBs are non-relational data stores that have been employed in various scenarios, where traditional RDBMS features matter less, and the improved performance of storing or retrieving relatively simple data sets matters most. The relational and the NoSQL database model are each good for specific applications. Depending on the problem to solve, a NoSQL or a relational model can be advantageous. In this session we present some typical use cases and how they can be solved with both NoSQL and the RDMBS databases. Will there be clear a winner or is there room for both NoSQL and RDMBS in the future?
A brief intro on the idea of what is Big Data and it's potential. This is primarily a basic study & I have quoted the source of infographics, stats & text at the end. If I have missed any reference due to human error & you recognize another source, please mention.
NoSQL databases get a lot of press coverage, but there seems to be a lot of confusion surrounding them, as in which situations they work better than a Relational Database, and how to choose one over another. This talk will give an overview of the NoSQL landscape and a classification for the different architectural categories, clarifying the base concepts and the terminology, and will provide a comparison of the features, the strengths and the drawbacks of the most popular projects (CouchDB, MongoDB, Riak, Redis, Membase, Neo4j, Cassandra, HBase, Hypertable).
Technical overview of three of the most representative KeyValue Stores: Cassandra, Redis and CouchDB. Focused on Ruby and Ruby on Rails developement, this talk shows how to solve common problems, the most popular libraries, benchmarking and the best use case for each one of them.
This talk was part of the Conferencia Rails 2009, Madrid, Spain.
http://app.conferenciarails.org/talks/43-key-value-stores-conviertete-en-un-jedi-master
SQL, NoSQL, Distributed SQL: Choose your DataStore carefullyMd Kamaruzzaman
In modern Software Development and Software Architecture, selecting the right DataStore is one of the most challenging and important task. In this presentation, I have summarized the major DataStores and the decision criteria to select the right DataStore according to the use case.
HBase hast established itself as the backend for many operational and interactive use-cases, powering well-known services that support millions of users and thousands of concurrent requests. In terms of features HBase has come a long way, overing advanced options such as multi-level caching on- and off-heap, pluggable request handling, fast recovery options such as region replicas, table snapshots for data governance, tuneable write-ahead logging and so on. This talk is based on the research for the an upcoming second release of the speakers HBase book, correlated with the practical experience in medium to large HBase projects around the world. You will learn how to plan for HBase, starting with the selection of the matching use-cases, to determining the number of servers needed, leading into performance tuning options. There is no reason to be afraid of using HBase, but knowing its basic premises and technical choices will make using it much more successful. You will also learn about many of the new features of HBase up to version 1.3, and where they are applicable.
From: DataWorks Summit 2017 - Munich - 20170406
HBase hast established itself as the backend for many operational and interactive use-cases, powering well-known services that support millions of users and thousands of concurrent requests. In terms of features HBase has come a long way, overing advanced options such as multi-level caching on- and off-heap, pluggable request handling, fast recovery options such as region replicas, table snapshots for data governance, tuneable write-ahead logging and so on. This talk is based on the research for the an upcoming second release of the speakers HBase book, correlated with the practical experience in medium to large HBase projects around the world. You will learn how to plan for HBase, starting with the selection of the matching use-cases, to determining the number of servers needed, leading into performance tuning options. There is no reason to be afraid of using HBase, but knowing its basic premises and technical choices will make using it much more successful. You will also learn about many of the new features of HBase up to version 1.3, and where they are applicable.
An overview of various database technologies and their underlying mechanisms over time.
Presentation delivered at Alliander internally to inspire the use of and forster the interest in new (NOSQL) technologies. 18 September 2012
Виталий Бондаренко "Fast Data Platform for Real-Time Analytics. Architecture ...Fwdays
We will start from understanding how Real-Time Analytics can be implemented on Enterprise Level Infrastructure and will go to details and discover how different cases of business intelligence be used in real-time on streaming data. We will cover different Stream Data Processing Architectures and discus their benefits and disadvantages. I'll show with live demos how to build Fast Data Platform in Azure Cloud using open source projects: Apache Kafka, Apache Cassandra, Mesos. Also I'll show examples and code from real projects.
Learn what you need to consider when moving from the world of relational databases to a NoSQL document store.
Hear from Developer Advocate Glynn Bird as he explains the key differences between relational databases and JSON document stores like Cloudant, as well as how to dodge the pitfalls of migrating from a relational database to NoSQL.
This is the last of 8 presentations given at University of Texas during my Beginner to Builder Rails 3 Class. For more info and the whole series including video presentations at my blog:
http://schneems.com/tagged/Rails-3-beginner-to-builder-2011
This is the 6th of 8 presentations given at University of Texas during my Beginner to Builder Rails 3 Class. For more info and the whole series including video presentations at my blog:
http://schneems.com/tagged/Rails-3-beginner-to-builder-2011
This is the 5th of 8 presentations given at University of Texas during my Beginner to Builder Rails 3 Class. For more info and the whole series including video presentations at my blog:
http://schneems.com/tagged/Rails-3-beginner-to-builder-2011
This is the 4th of 8 presentations given at University of Texas during my Beginner to Builder Rails 3 Class. For more info and the whole series including video presentations at my blog:
http://schneems.com/tagged/Rails-3-beginner-to-builder-2011
This is the 3rd of 8 presentations given at University of Texas during my Beginner to Builder Rails 3 Class. For more info and the whole series including video presentations at my blog:
http://schneems.tumblr.com/tagged/Rails-3-beginner-to-builder-2011
This is the 2nd of 8 presentations given at University of Texas during my Beginner to Builder Rails 3 Class. For more info and the whole series including video presentations at my blog:
http://schneems.tumblr.com/tagged/Rails-3-beginner-to-builder-2011
This is the first of 8 presentations given at University of Texas during my Beginner to Builder Rails 3 Class. For more info and the whole series including video presentations at my blog:
http://schneems.tumblr.com/tagged/Rails-3-beginner-to-builder-2011
This series introduces students to ruby on rails 3 through the book "Agile Web Development with Rails" with accompanying lecture videos found at http://www.thinkbohemian.com/tag/rails-summer-of-code/
This series introduces students to ruby on rails 3 through the book "Agile Web Development with Rails" with accompanying lecture videos found at http://www.thinkbohemian.com/tag/rails-summer-of-code/
This series introduces students to ruby on rails 3 through the book "Agile Web Development with Rails" with accompanying lecture videos found at http://www.thinkbohemian.com/tag/rails-summer-of-code/
This series introduces students to ruby on rails 3 through the book "Agile Web Development with Rails" with accompanying lecture videos found at http://www.thinkbohemian.com/tag/rails-summer-of-code/
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
10. The Web is Data
• Username => String
• Birthday => Int/ Int/ Int
• Blog Post => Text
• Image => Binary-file/blob
Data needs to be stored
to be useful
12. Gowalla Database
• PostgreSQL
• Relational (RDBMS)
• Open Source
• Competitor to MySQL
• ACID compliant
• Running on a Dedicated Managed Server
13. Need for Speed
• Throughput:
• The number of operations per minute that
can be performed
• Pure Speed:
• How long an individual operation takes.
14. Potential Problems
• Hardware
• Slow Network
• Slow hard-drive
• Insufficient CPU
• Insufficient Ram
• Software
• too many Reads
• too many Writes
15. Scaling Up versus Out
• Scale Up:
• More CPU, Bigger HD, More Ram etc.
• Scale Out:
• More machines
• More machines
• More machines
• ...
16. Scale Up
• Bigger faster machine
• More Ram
• More CPU
• Bigger ethernet bus
• ...
• Moores Law
• Diminishing returns
17. Scale Out
• Forget Moores law...
• Add more nodes
• Master/ Slave Database
• Sharding
18. Master/Slave
Write
Master DB
Copy
Slave DB Slave DB Slave DB Slave DB
Read
19. Master & Slave +/-
• Pro
• Increased read speed
• Takes read load off of master
• Allows us to Join across all tables
• Con
• Doesn’t buy increased write throughput
• Single Point of Failure in Master Node
20. Sharding
Write
Users in Users in Users in Users in
USA Europe Asia Africa
Read
21. Sharding +/-
• Pro
• Increased Write & Read throughput
• No Single Point of failure
• Individual features can fail
• Con
• Cannot Join queries between shards
22. What is a Database?
• Relational Database Managment System
(RDBMS)
• Stores Data Using Schema
• A.C.I.D. compliant
• Atomic
• Consistent
• Isolated
• Durable
23. RDBMS
• Relational
• Matches data on common characteristics
in data
• Enables “Join” & “Union” queries
• Makes data modular
24. Relational +/-
• Pros
• Data is modular
• Highly flexible data layout
• Cons
• Getting desired data can be tricky
• Over modularization leads to many join
queries
• Trade off performance for search-ability
25. Schema Storage
• Blueprint for data storage
• Break data into tables/columns/rows
• Give data types to your data
• Integer
• String
• Text
• Boolean
• ...
26. Schema +/-
• Pros
• Regularize our data
• Helps keep data consistent
• Converts to programming “types” easily
• Cons
• Must seperatly manage schema
• Adding columns & indexes to existing
large tables can be painful & slow
27. ACID
• Properties that guarante a reliably
transaction are processed
database
• Atomic
• Consistent
• Isolated
• Durable
28. ACID
• Atomic
• Any database Transaction is all or nothing.
• If one part of the transaction fails it all fails
“An Incomplete Transaction Cannot Exist”
29. ACID
• Consistent
• Any transaction will take the another
from one consistent state to
database
“Only Consistent data is allowed to be
written”
30. ACID
• Isolated
• No transaction should be able to interfere
with another transaction
“the same field cannot be updated by two
sources at the exact same time”
}
a = 0
a += 1 a = ??
a += 2
31. ACID
• Durable
• Onceway
that
a transaction Is committed it will stay
“Save it once, read it forever”
32. What is a Database?
• RDBMS
• Relational
• Flexible
• Has a schema
• Most likely ACID compliant
• Typically fast under low load or when
optimized
33. What is SQL?
• Structured Query Language
• The language databases speak
• Based on relational algebra
• Insert
• Query
• Update
• Delete
“SELECT Company, Country FROM Customers
WHERE Country = 'USA' ”
34. Why people <3 SQL
• Relational algebra is powerful
• SQL is proven
• well understood
• well documented
35. Why people </3 SQL
• Relational algebra Is hard
• Different databases support different SQL
syntax
• Yet another programming language to learn
36. SQL != Database
• SQL is used to talk to a RDBMS (database)
• SQL is not a RDBMS
42. Key Value Example
redis = Redis.new
Key Value
redis.set(“foo”, “bar”)
Key
redis.get(“foo”)
Value
>> “bar”
43. Key Value
• Like a databse that can only ever use
primary Key (id)
YES
select * from users where id = ‘3’;
NO
select * from users where name = ‘schneems’;
44. NoSQL @ Gowalla
• Redis (key-value store)
• Store “Likes” & Analytics
• Memcache (key-value store)
• Cache Database results
• Cassandra
• (eventually consistent, with-schema, key
value store)
• Store “feeds” or “timelines”
• Solr (search index)
45. Memcache
• Key-Value Store
• Open Source
• Distributed
• In memory (ram) only
• fast, but volatile
• Not ACID
• Memory object caching system
47. Memcache
• Can store whole objects
memcache = Memcache.new
user = User.where(:username => “schneems”)
memcache.set(“user:3”, user)
user_from_cache = memcache.get(“user:3”)
user_from_cache == user
>> true
user_from_cache.username
>> “Schneems”
48. Memcache @ Gowalla
• Cache Common Queries
• Decreases Load on DB (postgres)
• Enables higher throughput from DB
• Faster response than DB
• Users see quicker page load time
49. What to Cache?
• Objects that change infrequently
• users
• spots (places)
• etc.
• Expensive(ish) sql queries
• Friend ids for users
• User ids for people visiting spots
• etc.
52. Memcache <3’s DB
• We use them Together
• If memcache doesn’t have a value
• Fetch from the database
• Set the key from database
• Hard
• Cache Invalidation : (
53. Redis
• Key Value Store
• Open Source
• Not Distributed (yet)
• Extremely Quick
• “Data structure server”
65. Tradeoffs
• Every Data store has them
• Know your data store
• Strengths
• Weaknesses
66. NoSQL vs. RDBMS
• No Magic Bullet
• Use Both!!!
• Model data in a datastore you understand
• Switch to when/if you need to
• Understand Your Options