The document discusses MongoDB and Hadoop. It provides an overview of how MongoDB and Hadoop can be used together, including use cases in commerce, insurance and fraud detection. It describes the MongoDB Connector for Hadoop, which allows reading and writing to MongoDB from Hadoop tools like MapReduce, Pig and Hive. A demo is shown of a movie recommendation application that uses both MongoDB and Spark on Hadoop to power a web application.
MongoDB and Hadoop: Driving Business InsightsMongoDB
MongoDB and Hadoop can work together to solve big data problems facing today's enterprises. We will take an in-depth look at how the two technologies complement and enrich each other with complex analyses and greater intelligence. We will take a deep dive into the MongoDB Connector for Hadoop and how it can be applied to enable new business insights with MapReduce, Pig, and Hive, and demo a Spark application to drive product recommendations.
MongoDB and Hadoop: Driving Business InsightsMongoDB
MongoDB and Hadoop can work together to solve big data problems facing today's enterprises. We will take an in-depth look at how the two technologies complement and enrich each other with complex analyses and greater intelligence. We will take a deep dive into the MongoDB Connector for Hadoop and how it can be applied to enable new business insights with MapReduce, Pig, and Hive, and demo a Spark application to drive product recommendations.
Mongo db and hadoop driving business insights - finalMongoDB
MongoDB and Hadoop can work together to solve big data problems facing today's enterprises. We will take an in-depth look at how the two technologies complement and enrich each other with complex analyses and greater intelligence. We will take a deep dive into the MongoDB Connector for Hadoop and how it can be applied to enable new business insights with MapReduce, Pig, and Hive, and demo a Spark application to drive product recommendations.
Modern architectures are moving away from a "one size fits all" approach. We are well aware that we need to use the best tools for the job. Given the large selection of options available today, chances are that you will end up managing data in MongoDB for your operational workload and with Spark for your high speed data processing needs.
Description: When we model documents or data structures there are some key aspects that need to be examined not only for functional and architectural purposes but also to take into consideration the distribution of data nodes, streaming capabilities, aggregation and queryability options and how we can integrate the different data processing software, like Spark, that can benefit from subtle but substantial model changes. A clear example is when embedding or referencing documents and their implications on high speed processing.
Over the course of this talk we will detail the benefits of a good document model for the operational workload. As well as what type of transformations we should incorporate in our document model to adjust for the high speed processing capabilities of Spark.
We will look into the different options that we have to connect these two different systems, how to model according to different workloads, what kind of operators we need to be aware of for top performance and what kind of design and architectures we should put in place to make sure that all of these systems work well together.
Over the course of the talk we will showcase different libraries that enable the integration between spark and MongoDB, such as MongoDB Hadoop Connector, Stratio Connector and MongoDB Spark Native Connector.
By the end of the talk I expect the attendees to have an understanding of:
How they connect their MongoDB clusters with Spark
Which use cases show a net benefit for connecting these two systems
What kind of architecture design should be considered for making the most of Spark + MongoDB
How documents can be modeled for better performance and operational process, while processing these data sets stored in MongoDB.
The talk is suitable for:
Developers that want to understand how to leverage Spark
Architects that want to integrate their existing MongoDB cluster and have real time high speed processing needs
Data scientists that know about Spark, are playing with Spark and want to integrate with MongoDB for their persistency layer
MongoDB and Hadoop: Driving Business InsightsMongoDB
MongoDB and Hadoop can work together to solve big data problems facing today's enterprises. We will take an in-depth look at how the two technologies complement and enrich each other with complex analyses and greater intelligence. We will take a deep dive into the MongoDB Connector for Hadoop and how it can be applied to enable new business insights with MapReduce, Pig, and Hive, and demo a Spark application to drive product recommendations.
MongoDB and Hadoop: Driving Business InsightsMongoDB
MongoDB and Hadoop can work together to solve big data problems facing today's enterprises. We will take an in-depth look at how the two technologies complement and enrich each other with complex analyses and greater intelligence. We will take a deep dive into the MongoDB Connector for Hadoop and how it can be applied to enable new business insights with MapReduce, Pig, and Hive, and demo a Spark application to drive product recommendations.
Mongo db and hadoop driving business insights - finalMongoDB
MongoDB and Hadoop can work together to solve big data problems facing today's enterprises. We will take an in-depth look at how the two technologies complement and enrich each other with complex analyses and greater intelligence. We will take a deep dive into the MongoDB Connector for Hadoop and how it can be applied to enable new business insights with MapReduce, Pig, and Hive, and demo a Spark application to drive product recommendations.
Modern architectures are moving away from a "one size fits all" approach. We are well aware that we need to use the best tools for the job. Given the large selection of options available today, chances are that you will end up managing data in MongoDB for your operational workload and with Spark for your high speed data processing needs.
Description: When we model documents or data structures there are some key aspects that need to be examined not only for functional and architectural purposes but also to take into consideration the distribution of data nodes, streaming capabilities, aggregation and queryability options and how we can integrate the different data processing software, like Spark, that can benefit from subtle but substantial model changes. A clear example is when embedding or referencing documents and their implications on high speed processing.
Over the course of this talk we will detail the benefits of a good document model for the operational workload. As well as what type of transformations we should incorporate in our document model to adjust for the high speed processing capabilities of Spark.
We will look into the different options that we have to connect these two different systems, how to model according to different workloads, what kind of operators we need to be aware of for top performance and what kind of design and architectures we should put in place to make sure that all of these systems work well together.
Over the course of the talk we will showcase different libraries that enable the integration between spark and MongoDB, such as MongoDB Hadoop Connector, Stratio Connector and MongoDB Spark Native Connector.
By the end of the talk I expect the attendees to have an understanding of:
How they connect their MongoDB clusters with Spark
Which use cases show a net benefit for connecting these two systems
What kind of architecture design should be considered for making the most of Spark + MongoDB
How documents can be modeled for better performance and operational process, while processing these data sets stored in MongoDB.
The talk is suitable for:
Developers that want to understand how to leverage Spark
Architects that want to integrate their existing MongoDB cluster and have real time high speed processing needs
Data scientists that know about Spark, are playing with Spark and want to integrate with MongoDB for their persistency layer
The MongoDB Spark Connector integrates MongoDB and Apache Spark, providing users with the ability to process data in MongoDB with the massive parallelism of Spark. The connector gives users access to Spark's streaming capabilities, machine learning libraries, and interactive processing through the Spark shell, Dataframes and Datasets. We'll take a tour of the connector with a focus on practical use of the connector, and run a demo using both Spark and MongoDB for data processing.
MongoDB Evenings Dallas: What's the Scoop on MongoDB & HadoopMongoDB
What's the Scoop on MongoDB & Hadoop
Jake Angerman, Sr. Solutions Architect, MongoDB
MongoDB Evenings Dallas
March 30, 2016 at the Addison Treehouse, Dallas, TX
BM Cloudant is a NoSQL Database-as-a-Service. Discover how you can outsource the data layer of your mobile or web application to Cloudant to provide high availability, scalability and tools to take you to the next level.
How to get the best of both: MongoDB is great for low latency quick access of recent data; Treasure Data is great for infinitely growing store of historical data. In the latter case, one need not worry about scaling.
MongoDB Evenings DC: Get MEAN and Lean with Docker and KubernetesMongoDB
Get MEAN and Lean with Docker and Kubernetes
Vadim Polyakov, Director of Enterprise Application Architecture, Inovalon
MongoDB Evenings DC
April 12, 2016 at 1776
MongoDB has been conceived for the cloud age. Making sure that MongoDB is compatible and performant around cloud providers is mandatory to achieve complete integration with platforms and systems. Azure is one of biggest IaaS platforms available and very popular amongst developers that work on Microsoft Stack.
Data persistence using pouchdb and couchdbDimgba Kalu
A presentation on data persistence using pouchdb and couchdb. This is a basic way of building offline data repository and efficient data synchronization
Webinar: Elevate Your Enterprise Architecture with In-Memory ComputingMongoDB
The advantages of in-memory computing are well understood. Data can be accessed in RAM nearly 100,000 times faster than retrieving it from disk, delivering orders-of-magnitude higher performance for the most demanding applications. Examples include real-time re-scoring of personalized product recommendations as users are browsing a site, or trading stocks in immediate response to market events.
In this webinar, we’ll briefly explore the trends driving in-memory computing (IMC), the challenges that surround it, and how MongoDB fits into the big picture.
Topics covered in this session will include:
- IMC use cases and customer case studies
- Critical capabilities and components of IMC
- How MongoDB plays a role in an overall IMC strategy within your enterprise architecture
- Suggested architectures related to MongoDB’s in-memory capabilities:
-- Integration with Apache Spark
-- In-Memory Storage Engine
-- Integration with BI tools
This talk will provide a brief update on Microsoft’s recent history in Open Source with specific emphasis on Azure Databricks, a fast, easy and collaborative Apache Spark-based analytics service. Attendees will learn how to integrate MongoDB Atlas with Azure Databricks using the MongoDB Connector for Spark. This integration allows users to process data in MongoDB with the massive parallelism of Spark, its machine learning libraries, and streaming API.
The integration between Spring Framework and MongoDB tends to be somewhat unknown. This presentation shows the different projects that compose Spring ecosystem, Springdata, Springboot, SpringIO etc and how to merge between the pure JAVA projects to massive enterprise systems that require the interaction of these systems together.
MongoDB World 2019: Raiders of the Anti-patterns: A Journey Towards Fixing Sc...MongoDB
As a software adventurer, Charles “Indy” Sarrazin, has brought numerous customers through the MongoDB world, using his extensive knowledge to make sure they always got the most out of their databases.
Let us embark on a journey inside the Document Model, where we will identify, analyze and fix anti-patterns. I will also provide you with tools to ease migration strategies towards the Temple of Lost Performance!
Be warned, though! You might want to learn about design patterns before, in order to survive this exhilarating trial!
MongoDB World 2019: MongoDB in Data Science: How to Build a Scalable Product ...MongoDB
You have made a successful Proof of Concept by using Pandas for data manipulation and analysis. So, how are you going to productionize it? Come to learn how to transform your POC to a scalable product with MongoDB. Learn about pitfalls and drawbacks of Pandas and benefits of using MongoDB in the early stages.
Webinar: MongoDB and Hadoop - Working Together to provide Business InsightsMongoDB
Join us for a webinar on how MongoDB and Hadoop can work together to solve Big Data problems in today's enterprises. We will take an in depth look at how the two technologies make real business intelligence accessible to end users. After a brief introduction to both technologies, this webinar will dive deep into the MongoDB+Hadoop Connector and how it is applied to enable new business insights.
In this webinar you will learn:
What information problems are a good fit for MongoDB and Hadoop
How to integrate the two technologies using the MongoDB+Hadoop Connector
Programming paradigms for tackling common problems
Webinar: Data Streaming with Apache Kafka & MongoDBMongoDB
A new generation of technologies is needed to consume and exploit today's real time, fast moving data sources. Apache Kafka, originally developed at LinkedIn, has emerged as one of these key new technologies.
This webinar explores the use-cases and architecture for Kafka, and how it integrates with MongoDB to build sophisticated data-driven applications that exploit new sources of data.
Watch the webinar to learn:
- What MongoDB is and where it's used
- What data streaming is and where it fits into modern data architectures
- How Kafka works, what it delivers, and where it's used
- How to operationalize the Data Lake with MongoDB & Kafka
- How MongoDB integrates with Kafka – both as a producer and a consumer of event data
The webinar is co-presented with Confluent, the company founded by the creators of Apache Kafka.
The MongoDB Spark Connector integrates MongoDB and Apache Spark, providing users with the ability to process data in MongoDB with the massive parallelism of Spark. The connector gives users access to Spark's streaming capabilities, machine learning libraries, and interactive processing through the Spark shell, Dataframes and Datasets. We'll take a tour of the connector with a focus on practical use of the connector, and run a demo using both Spark and MongoDB for data processing.
MongoDB Evenings Dallas: What's the Scoop on MongoDB & HadoopMongoDB
What's the Scoop on MongoDB & Hadoop
Jake Angerman, Sr. Solutions Architect, MongoDB
MongoDB Evenings Dallas
March 30, 2016 at the Addison Treehouse, Dallas, TX
BM Cloudant is a NoSQL Database-as-a-Service. Discover how you can outsource the data layer of your mobile or web application to Cloudant to provide high availability, scalability and tools to take you to the next level.
How to get the best of both: MongoDB is great for low latency quick access of recent data; Treasure Data is great for infinitely growing store of historical data. In the latter case, one need not worry about scaling.
MongoDB Evenings DC: Get MEAN and Lean with Docker and KubernetesMongoDB
Get MEAN and Lean with Docker and Kubernetes
Vadim Polyakov, Director of Enterprise Application Architecture, Inovalon
MongoDB Evenings DC
April 12, 2016 at 1776
MongoDB has been conceived for the cloud age. Making sure that MongoDB is compatible and performant around cloud providers is mandatory to achieve complete integration with platforms and systems. Azure is one of biggest IaaS platforms available and very popular amongst developers that work on Microsoft Stack.
Data persistence using pouchdb and couchdbDimgba Kalu
A presentation on data persistence using pouchdb and couchdb. This is a basic way of building offline data repository and efficient data synchronization
Webinar: Elevate Your Enterprise Architecture with In-Memory ComputingMongoDB
The advantages of in-memory computing are well understood. Data can be accessed in RAM nearly 100,000 times faster than retrieving it from disk, delivering orders-of-magnitude higher performance for the most demanding applications. Examples include real-time re-scoring of personalized product recommendations as users are browsing a site, or trading stocks in immediate response to market events.
In this webinar, we’ll briefly explore the trends driving in-memory computing (IMC), the challenges that surround it, and how MongoDB fits into the big picture.
Topics covered in this session will include:
- IMC use cases and customer case studies
- Critical capabilities and components of IMC
- How MongoDB plays a role in an overall IMC strategy within your enterprise architecture
- Suggested architectures related to MongoDB’s in-memory capabilities:
-- Integration with Apache Spark
-- In-Memory Storage Engine
-- Integration with BI tools
This talk will provide a brief update on Microsoft’s recent history in Open Source with specific emphasis on Azure Databricks, a fast, easy and collaborative Apache Spark-based analytics service. Attendees will learn how to integrate MongoDB Atlas with Azure Databricks using the MongoDB Connector for Spark. This integration allows users to process data in MongoDB with the massive parallelism of Spark, its machine learning libraries, and streaming API.
The integration between Spring Framework and MongoDB tends to be somewhat unknown. This presentation shows the different projects that compose Spring ecosystem, Springdata, Springboot, SpringIO etc and how to merge between the pure JAVA projects to massive enterprise systems that require the interaction of these systems together.
MongoDB World 2019: Raiders of the Anti-patterns: A Journey Towards Fixing Sc...MongoDB
As a software adventurer, Charles “Indy” Sarrazin, has brought numerous customers through the MongoDB world, using his extensive knowledge to make sure they always got the most out of their databases.
Let us embark on a journey inside the Document Model, where we will identify, analyze and fix anti-patterns. I will also provide you with tools to ease migration strategies towards the Temple of Lost Performance!
Be warned, though! You might want to learn about design patterns before, in order to survive this exhilarating trial!
MongoDB World 2019: MongoDB in Data Science: How to Build a Scalable Product ...MongoDB
You have made a successful Proof of Concept by using Pandas for data manipulation and analysis. So, how are you going to productionize it? Come to learn how to transform your POC to a scalable product with MongoDB. Learn about pitfalls and drawbacks of Pandas and benefits of using MongoDB in the early stages.
Webinar: MongoDB and Hadoop - Working Together to provide Business InsightsMongoDB
Join us for a webinar on how MongoDB and Hadoop can work together to solve Big Data problems in today's enterprises. We will take an in depth look at how the two technologies make real business intelligence accessible to end users. After a brief introduction to both technologies, this webinar will dive deep into the MongoDB+Hadoop Connector and how it is applied to enable new business insights.
In this webinar you will learn:
What information problems are a good fit for MongoDB and Hadoop
How to integrate the two technologies using the MongoDB+Hadoop Connector
Programming paradigms for tackling common problems
Webinar: Data Streaming with Apache Kafka & MongoDBMongoDB
A new generation of technologies is needed to consume and exploit today's real time, fast moving data sources. Apache Kafka, originally developed at LinkedIn, has emerged as one of these key new technologies.
This webinar explores the use-cases and architecture for Kafka, and how it integrates with MongoDB to build sophisticated data-driven applications that exploit new sources of data.
Watch the webinar to learn:
- What MongoDB is and where it's used
- What data streaming is and where it fits into modern data architectures
- How Kafka works, what it delivers, and where it's used
- How to operationalize the Data Lake with MongoDB & Kafka
- How MongoDB integrates with Kafka – both as a producer and a consumer of event data
The webinar is co-presented with Confluent, the company founded by the creators of Apache Kafka.
MongoDB Days Germany: Data Processing with MongoDBMongoDB
Presented by Marc Schwering, Senior Solutions Architect, MongoDB
Modern architectures are moving away from "one size fits all" solutions. The best tools need to be put to the job and given the large amounts of options today, chances are that you’ll end up using MongoDB for your operational workload, as well as Data Processing Systems like Apache Flink or Spark for your high speed data processing needs. When documents or data structures are modeled, there are some key aspects that need to be attended. This takes into consideration the distribution of data nodes, streaming capabilities, performance, aggregation, and queryability options, and how we can integrate the different data processing software that can benefit from subtle but substantial model changes. This session will cover the way how you enhance your architecture using data processing technologies such as Apache Flink and Spark. It will take the audience through the evolution of an app from simple to complex with its architectural requirements . We´ll look into similarities and differences of available technologies and you will walk away with an understanding how to use MongoDB to fulfill more advanced tasks such as personalization through clustering algorithms.
Java Persistence Frameworks for MongoDBTobias Trelle
After a short introduction to the MongoDB Java driver we'll have a detailed look at higher level persistence frameworks like Morphia, Spring Data MongoDB and Hibernate OGM with lots of examples.
MongoDB SoCal 2020: Migrate Anything* to MongoDB AtlasMongoDB
During this talk we'll navigate through a customer's journey as they migrate an existing MongoDB deployment to MongoDB Atlas. While the migration itself can be as simple as a few clicks, the prep/post effort requires due diligence to ensure a smooth transfer. We'll cover these steps in detail and provide best practices. In addition, we’ll provide an overview of what to consider when migrating other cloud data stores, traditional databases and MongoDB imitations to MongoDB Atlas.
MongoDB .local Houston 2019: Wide Ranging Analytical Solutions on MongoDBMongoDB
MongoDB natively provides a rich analytics framework within the database. We will highlight the different tools, features and capabilities that MongoDB provides to enable various analytics scenarios ranging from AI, Machine Learning and applications. We will demonstrate a Machine Learning (ML) example using MongoDB and Spark.
How sitecore depends on mongo db for scalability and performance, and what it...Antonios Giannopoulos
Percona Live 2017 - How sitecore depends on mongo db for scalability and performance, and what it can teach you by Antonios Giannopoulos and Grant Killian
MongoDB is a scalable high-performance open-source document-orientated database which is built for speed, rich document based queries for easy readability, full index support for high performance, replication and failover for high availability, auto sharding for easy scalability and map/reduce for aggregation.
MongoDB.local Sydney: An Introduction to Document Databases with MongoDBMongoDB
This presentation will describe MongoDB's document database and what advantages it has over traditional databases. The presentation will explore MongoDB's server, query language, ecosystem and various tools. Brett will demonstrate using various MongoDB tools to assist in developing a Python application that utilises MongoDB as the database.
In the age of digital transformation and disruption, your ability to thrive depends on how you adapt to the constantly changing environment. MongoDB 3.4 is the latest release of the leading database for modern applications, a culmination of native database features and enhancements that will allow you to easily evolve your solutions to address emerging challenges and use cases.
In this webinar, we introduce you to what’s new, including:
- Multimodel Done Right. Native graph computation, faceted navigation, rich real-time analytics, and powerful connectors for BI and Apache Spark bring additional multimodel database support right into MongoDB.
- Mission-Critical Applications. Geo-distributed MongoDB zones, elastic clustering, tunable consistency, and enhanced security controls bring state-of-the-art database technology to your most mission-critical applications.
- Modernized Tooling. Enhanced DBA and DevOps tooling for schema management, fine-grained monitoring, and cloud-native integration allow engineering teams to ship applications faster, with less overhead and higher quality.
Machine Learning on Google Cloud with H2OSri Ambati
This meetup was held in San Francisco on July 23rd, 2018.
Video recording from the meetup can be viewed here: https://youtu.be/KZfRLGElQLE
Nicholas gave an overview of H2O, the leading open source machine learning platform for the enterprise, which integrates seamlessly with R and Python environments, as well as, Driverless AI, an enterprise automated machine learning solution. Nicholas also spoke about some of the integration points that H2O.ai has built with Google, including: Google Cloud Engine, Kubeflow, and more.
Speaker's Bio:
Nicholas Png is a Partnerships Software Engineer at H2O.ai. Prior to working at H2O, he worked as a Quality Assurance Software Engineer, developing software automation testing. Nicholas holds a degree in Mechanical Engineering, and has experience working with customers across multiple industries, identifying common problems, and designing robust, automated solutions.
Data Streaming with Apache Kafka & MongoDBconfluent
Explore the use-cases and architecture for Apache Kafka, and how it integrates with MongoDB to build sophisticated data-driven applications that exploit new sources of data.
Fast Cars, Big Data - How Streaming Can Help Formula 1Tugdual Grall
Modern cars produce data. Lots of data. And Formula 1 cars produce more than their share. I will present a working demonstration of how modern data streaming can be applied to the data acquisition and analysis problem posed by modern motorsports.
Instead of bringing multiple Formula 1 cars to the talk, I will show how we instrumented a high fidelity physics-based automotive simulator to produce realistic data from simulated cars running on the Spa-Francorchamps track. We move data from the cars, to the pits, to the engineers back at HQ.
The result is near real-time visualization and comparison of performance and a great exposition of how to move data using messaging systems like Kafka, and process data in real time with Apache Spark, then analyse data using SQL with Apache Drill.
Code available here: https://github.com/mapr-demos/racing-time-series
Lambda Architecture: The Best Way to Build Scalable and Reliable Applications!Tugdual Grall
Lambda Architecture is a useful framework to think about designing big data applications. This framework has been built initially at Twitter. In this presentation you will learn, based on concrete examples how to build deploy scalable and fault tolerant applications, with a focus on Big Data and Hadoop.
This presentation was delivered at the OOP conference, Munich, Feb 2016
Generic presentation about Big Data Architecture/Components. This presentation was delivered by David Pilato and Tugdual Grall during JUG Summer Camp 2015 in La Rochelle, France
Proud to be Polyglot - Riviera Dev 2015Tugdual Grall
New developers and teams are now polyglot :
- they use multiple programming languages (Java, Javascript, Ruby, ...)
- they use multiple persistence store (RDBMS, NoSQL, Hadoop)
In this talk you will learn about the benefits if being polyglot: use the good language or framework for the good cause, select the good persistence for specific constraints.
This presentation will show how developer could mix the Python, NodeJS, AngularJS, SQL with Drill for Hadoop and MongoDB.
Enabling Telco to Build and Run Modern Applications Tugdual Grall
See how new databases like MongoDB enable Telco Enterprises to Build and Run Modern Applications.
This presentations was delivered in Tel Aviv in Jan-2015 during a Telco round table organized by Matrix.
New developers and teams are now polyglot :
- they use multiple programming languages (Java, Javascript, Ruby, ...)
- they use multiple persistence store (RDBMS, NoSQL, Hadoop)
In this talk you will learn about the benefits if being polyglot: use the good language or framework for the good cause, select the good persistence for specific constraints.
This presentation will show how developer could mix the Java platform with other technologies such as NodeJS and AngularJS to build application in a more productive way. This is also the opportunity to talk about the new Command Query Responsibility Segregation (CQRS) pattern to allow developers to be more effective and deliver the proper application to the user quicker.
This presentation was delivered during Devfest Nantes 2014
Drop your table ! MongoDB Schema DesignTugdual Grall
MongoDB’s basic unit of storage is a document. Documents can represent rich, schema-free data structures, meaning that we have several viable alternatives to the normalized, relational model. In this talk, we’ll discuss the tradeoff of various data modeling strategies in MongoDB using various example. You will learn how to work with documents, evolve your schema, and common schema design patterns.
Delivered at Soft Shake '14 and Jug Summer Camp '14
Softshake 2013: Introduction to NoSQL with CouchbaseTugdual Grall
This presentation was delivered during Softshake 2013. Learn why RDBMS are not enought and why NoSQL help developers to scale their applications and provide agility.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
5. Hadoop
“The Apache Hadoop software library is a framework
that allows for the distributed processing of large
data sets across clusters of computers using simple
programming models.”
• Terabyte and Petabyte datasets
• Data warehousing
• Advanced analytics
http://hadoop.apache.org
7. ‹#›
Operational vs. Analytical: Enrichment
Applications, Interactions Warehouse, Analytics
8. Operational: MongoDB
First-Level
Analytics
Internet of Things
Mobile Apps
Social
Product/Asset
Catalog
Security & Fraud
Customer Data
Management
Single View
Churn Analysis
Risk Modeling
Trade
Surveillance
Sentiment
Analysis
Recommender
Warehouse & ETL
Predictive
Analytics
Ad Targeting
9. Analytical: Hadoop
First-Level
Analytics
Internet of Things
Mobile Apps
Social
Product/Asset
Catalog
Security & Fraud
Customer Data
Management
Single View
Churn Analysis
Risk Modeling
Trade
Surveillance
Sentiment
Analysis
Recommender
Warehouse & ETL
Predictive
Analytics
Ad Targeting
10. Operational & Analytical: Lifecycle
First-Level
Analytics
Internet of Things
Mobile Apps
Social
Product/Asset
Catalog
Security & Fraud
Customer Data
Management
Single View
Churn Analysis
Risk Modeling
Trade
Surveillance
Sentiment
Analysis
Recommender
Warehouse & ETL
Predictive
Analytics
Ad Targeting
12. Commerce
Applications
powered by
Analysis
powered by
Products & Inventory
Recommended products
Customer profile
Session management
Elastic pricing
Recommendation models
Predictive analytics
Clickstream history
MongoDB Connector
for Hadoop
13. Insurance
Applications
powered by
Analysis
powered by
Customer profiles
Insurance policies
Session data
Call center data
Customer action analysis
Churn analysis
Churn prediction
Policy rates
MongoDB Connector
for Hadoop
14. Fraud Detection
Payments Nightly Analysis
MongoDB Connector
for Hadoop
3rd Party
Data Sources
Results Cache
Fraud
Detection
Query Only
Query Only
17. ‹#›
Connector Features and Functionality
• Computes splits to read data
• Single Node, Replica Sets, Sharded Clusters
• Mappings for Pig and Hive
• MongoDB as a standard data source/destination
• Support for
• Filtering data with MongoDB queries
• Authentication
• Reading from Replica Set tags
• Appending to existing collections
19. ‹#›
Pig Mappings
• Input: BSONLoader and MongoLoader
data = LOAD ‘mongodb://mydb:27017/db.collection’
using com.mongodb.hadoop.pig.MongoLoader
• Output: BSONStorage and MongoInsertStorage
STORE records INTO ‘hdfs:///output.bson’
using com.mongodb.hadoop.pig.BSONStorage
20. ‹#›
Hive Support
• Access collections as Hive tables
• Use with MongoStorageHandler or BSONStorageHandler
CREATE TABLE mongo_users (id int, name string, age int)
STORED BY "com.mongodb.hadoop.hive.MongoStorageHandler"
WITH SERDEPROPERTIES("mongo.columns.mapping” = "_id,name,age”)
TBLPROPERTIES("mongo.uri" = "mongodb://host:27017/test.users”)
21. ‹#›
Spark
• Use with MapReduce input/output
formats
• Create Configuration objects with
input/output formats and data URI
• Load/save data using SparkContext
Hadoop file API
22. ‹#›
Data Movement
Dynamic queries to MongoDB vs. BSON snapshots in HDFS
Dynamic queries with most
recent data
Puts load on operational
database
Snapshots move load to
Hadoop
Snapshots add predictable
load to MongoDB
25. ‹#›
MovieWeb Web Application
• Browse
- Top movies by ratings count
- Top genres by movie count
• Log in to
- See My Ratings
- Rate movies
• Recommendations
- Movies You May Like
- Recommendations
26. ‹#›
MovieWeb Components
• MovieLens dataset
– 10M ratings, 10K movies, 70K users
– http://grouplens.org/datasets/movielens/
• Python web app to browse movies, recommendations
– Flask, PyMongo
• Spark app computes recommendations
– MLLib collaborative filter
• Predicted ratings are exposed in web app
– New predictions collection
27. ‹#›
Spark Recommender
• Apache Hadoop (2.3)
- HDFS & YARN
- Top genres by movie count
• Spark (1.0)
- Execute within YARN
- Assign executor resources
• Data
- From HDFS, MongoDB
- To MongoDB
28. ‹#›
MovieWeb Workflow
Snapshot db
as BSON
Predict ratings for
all pairings
Write Prediction to
MongoDB
collection
Store BSON
in HDFS
Read BSON
into Spark App
Create user movie
pairing
Web Application
exposes
recommendations
Train Model from
existing ratings
Repeat Process
31. ‹#›
Business First!
First-Level
Analytics
Internet of
Things
Mobile Apps
Social
What/Why How
Product/Asse
t Catalog
Security &
Fraud
Customer
Data
Management
Single View
Churn
Analysis
Risk
Modeling
Trade
Surveillance
Sentiment
Analysis
Recommend
er
Warehouse
& ETL
Predictive
Analytics
Ad Targeting
32. ‹#›
The good tool for the task
• Dataset size
• Data processing complexity
• Continuous improvement
V1.0
33. ‹#›
The good tool for the task
• Dataset size
• Data processing complexity
• Continuous improvement
V2.0
34. ‹#›
Resources / Questions
• MongoDB Connector for Hadoop
- http://github.com/mongodb/mongo-hadoop
• Getting Started with MongoDB and Hadoop
- http://docs.mongodb.org/ecosystem/tutorial/getting-started-
with-hadoop/
• MongoDB-Spark Demo
- https://github.com/crcsmnky/mongodb-hadoop-workshop
Apache def, a framework to enable many things
Distributed File system one of the core component is MapReduce
Now it is more YARN, that is resource manager, and MR is just one type of jobs you can manage
Mongo DB : GB and Terabytes
Hadoop : Tb and Pb
You have 2 places where you deal with data
You have to think about “enrichment”
MongoDB is here to enrich data that are in Hadoop
Hadoop is here to enrich data that are in Mongodb
Let’s look at the different uses cases between Operational and Analytics
First level could be done in MongoDB “what is your application is talking to?”
Hadoop will be there to analyze a bigger problem and do some treatment
We are talking of hadoop when it is Pb of data
We are trying to solve the bigger problem, by connecting the 2 technologies when it makes sense
Split the data when reading data (Mapper)
But also filtering queries, for example to take data from a specific timestamp
To reduce the load on your cluster read from replicaset tags
a new feature that people asked for, is adding result to the existing collection
Spark in a new data processing that happens most in memory
Take all the power of the connector
Open a new “hadoop file” that is loaded in RDD ( Resilient Distributed Dataset )
Load Data
Read Users from MongoDB (user collection)
Read Movies from BSON (HDFS)
Read Ratings from MongoDB (ratings collection)
Data Processing
Generate (user/movies) pairs
users.cartesian(movies)
Train : Collaborative Filter
ALS.train(ratings.rdd(), 10, 10, 0.01);
Predict/Recommend
Save data into MongoDB prediction collection