After three years of development Drupal 7 is finally here providing a powerful open source solution that is more flexible and more scalable than ever. Vast improvements in usability and functionality allow site developers to roll out their sites in less time. Now is the time to upgrade your website to Drupal 7.
Join Erik Webb and Jacob Singh for this one-hour webinar where they will show you just how new and improved Drupal 7 is. Attendees will learn:
-How to take advantage of all the new features
-Considerations when upgrading your site to Drupal 7
-Next steps for getting your upgrade started
Presentation delivered as part of the NISO "Back From the Endangered List: Using Authority Data to Enhance the Semantic Web" Webinar on February 9th 2011.
Better front-end development in Atlassian pluginsAtlassian
Traditionally the UI of Atlassian plugins have been based on a typical old-school MVC frameworks (webwork, Struts) with little dynamic behaviour. But no more! JIRA Team Lead Wociech Seliga will show you have to develop sexier and more user-friendly plugins based on a modern stack using AUI, Soy templates, Backbone.js and friends.
Django and Neo4j - Domain modeling that kicks assTobias Lindaaker
Presentation about using Neo4j from Django presented at OSCON 2010, Portland OR.
Sample code is available at: https://svn.neo4j.org/components/neo4j.py/trunk/src/examples/python/djangosites/blog/
After three years of development Drupal 7 is finally here providing a powerful open source solution that is more flexible and more scalable than ever. Vast improvements in usability and functionality allow site developers to roll out their sites in less time. Now is the time to upgrade your website to Drupal 7.
Join Erik Webb and Jacob Singh for this one-hour webinar where they will show you just how new and improved Drupal 7 is. Attendees will learn:
-How to take advantage of all the new features
-Considerations when upgrading your site to Drupal 7
-Next steps for getting your upgrade started
Presentation delivered as part of the NISO "Back From the Endangered List: Using Authority Data to Enhance the Semantic Web" Webinar on February 9th 2011.
Better front-end development in Atlassian pluginsAtlassian
Traditionally the UI of Atlassian plugins have been based on a typical old-school MVC frameworks (webwork, Struts) with little dynamic behaviour. But no more! JIRA Team Lead Wociech Seliga will show you have to develop sexier and more user-friendly plugins based on a modern stack using AUI, Soy templates, Backbone.js and friends.
Django and Neo4j - Domain modeling that kicks assTobias Lindaaker
Presentation about using Neo4j from Django presented at OSCON 2010, Portland OR.
Sample code is available at: https://svn.neo4j.org/components/neo4j.py/trunk/src/examples/python/djangosites/blog/
Craig Walker is Chief Technology Officer and co-founder of small business accounting software provider Xero. Xero has over 50,000 customers in over 100 countries and Craig has recently relocated to San Francisco to start the US based office.
Craig currently leads Xero’s software development technology strategy. He has a morbid obsession with JavaScript and not having to use any other language for development (much to the amusement of his colleagues).
Craig and Xero have been keen evangelists of Sencha products having utilized Sencha JavaScript libraries across Xero’s entire product line. Craig’s most recent fun project was building Xero Touch, a Sencha Touch-based mobile version of Xero released for the web and into the Apple App Store (and soon to be launched for Android).
Database Management in Different Applications of IOTijceronline
In the recent years, the Internet of Things (IoT) is considered as a part of the Internet of future and makes it possible for connecting various smart objects together through the Internet. The use of IoT technology in applications has spurred the increase of real-time data, which makes the information storage and accessing more difficult and challenging. This paper discusses the different Databases used for different applications in IOT.
Nuxeo introduction to ecr at the NYC Java meetup, April 2011Nuxeo
A quick look at the Eclipse Enterprise Content Repository Project, a project proposal from Nuxeo to the Eclipse Foundation that aims to provide a reference, vendor neutral, Java & OSGi based Content Repository to developers building content-driven applications.
Froscon2011: How i learned to use sql and then learned not to use itHenrik Ingo
Keynote for the Open DB Camp track (developer room) at Froscon2011. The point is to compare the history of MySQL with the evolution of new NoSQL systems.
An unprecedented amount of data is being created and is accessible. This presentation will instruct on using the new NoSQL technologies to make sense of all this data.
Given at QCon London 2012
Your systems are going to fail, it might not be today, it might not be tomorrow, but sometime soon, probably at 2am, your systems are going to fail in new and exciting ways. We've shared at QCon before about the core architecture of guardian.co.uk, and how we built the site. Now we are going to tell you what we've learnt since we built it, in what ways it went wrong, and how we are learning to architect for failure at the very beginning of each project.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Craig Walker is Chief Technology Officer and co-founder of small business accounting software provider Xero. Xero has over 50,000 customers in over 100 countries and Craig has recently relocated to San Francisco to start the US based office.
Craig currently leads Xero’s software development technology strategy. He has a morbid obsession with JavaScript and not having to use any other language for development (much to the amusement of his colleagues).
Craig and Xero have been keen evangelists of Sencha products having utilized Sencha JavaScript libraries across Xero’s entire product line. Craig’s most recent fun project was building Xero Touch, a Sencha Touch-based mobile version of Xero released for the web and into the Apple App Store (and soon to be launched for Android).
Database Management in Different Applications of IOTijceronline
In the recent years, the Internet of Things (IoT) is considered as a part of the Internet of future and makes it possible for connecting various smart objects together through the Internet. The use of IoT technology in applications has spurred the increase of real-time data, which makes the information storage and accessing more difficult and challenging. This paper discusses the different Databases used for different applications in IOT.
Nuxeo introduction to ecr at the NYC Java meetup, April 2011Nuxeo
A quick look at the Eclipse Enterprise Content Repository Project, a project proposal from Nuxeo to the Eclipse Foundation that aims to provide a reference, vendor neutral, Java & OSGi based Content Repository to developers building content-driven applications.
Froscon2011: How i learned to use sql and then learned not to use itHenrik Ingo
Keynote for the Open DB Camp track (developer room) at Froscon2011. The point is to compare the history of MySQL with the evolution of new NoSQL systems.
An unprecedented amount of data is being created and is accessible. This presentation will instruct on using the new NoSQL technologies to make sense of all this data.
Given at QCon London 2012
Your systems are going to fail, it might not be today, it might not be tomorrow, but sometime soon, probably at 2am, your systems are going to fail in new and exciting ways. We've shared at QCon before about the core architecture of guardian.co.uk, and how we built the site. Now we are going to tell you what we've learnt since we built it, in what ways it went wrong, and how we are learning to architect for failure at the very beginning of each project.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
NoSQL! is it for you?
1. NoSQL
What it is and is it for you?
Iraj Islam
Rubayeet Islam
Nurul Ferdous
NewsCred
Thursday, February 3, 2011
2. Agenda NewsCred
• Part 1. Why NoSQL?
• Part 2. NoSQL Use Cases
• Part 3. Choosing a NoSQL Solution
• Part 4. Understanding MongoDB
• Part 5. Building a MongoDB App
• Part 6. Scaling MongoDB
• Questions
Thursday, February 3, 2011
3. Who We Are NewsCred
Iraj Islam
CTO/Co-founder, NewsCred
Rubayeet Islam
Senior Software Engineer, NewsCred
Nurul Ferdous
Senior Software Engineer, NewsCred
Thursday, February 3, 2011
4. Our Story NewsCred
Launched 2008
Founded by two Bangladeshis 2008
Funded By Investors of Twitter
Floodgate Ventures (twitter), Bessemer Cap. (LinkedIn)
Top-tier Clients
Yahoo! Orange Telecom, Harvard U, The Daily Star etc.
Thursday, February 3, 2011
5. What We Do NewsCred
Domain Expertise
• Big Data
• Information Retrieval
• Machine Learning
• Semantic Web
Technologies
• Apache Solr
• MySQL/MongoDB
• Python/Java
Thursday, February 3, 2011
6. Part 1
Why NoSQL?
NewsCred
Thursday, February 3, 2011
7. What’s NoSQL? NewsCred
NoSQL
What’s with the weird name?
Thursday, February 3, 2011
9. Why NoSQL? NewsCred
Web 1.0
The read intensive web
Publishing Model
Thursday, February 3, 2011
10. Why NoSQL? NewsCred
Web 1.0
The read intensive web
Publishing Model
Textual Content
Thursday, February 3, 2011
11. Why NoSQL? NewsCred
Web 1.0
The read intensive web
Publishing Model Small Data
Textual Content
Thursday, February 3, 2011
12. Why NoSQL? NewsCred
Web 1.0
The read intensive web
Publishing Model Browsing Small Data
Textual Content
Thursday, February 3, 2011
13. Why NoSQL? NewsCred
Web 1.0
The read intensive web
Publishing Model Browsing Small Data
Textual Content Search
Thursday, February 3, 2011
14. Why NoSQL? NewsCred
Web 1.0
The read intensive web
Publishing Model Browsing Small Data
Textual Content Personal Computer Search
Thursday, February 3, 2011
15. Why NoSQL? NewsCred
The Age of Big Data
Exabytes (1018) of data stored per year
1000
750
500
250
2006
2007
2008 0
2009
2010
Thursday, February 3, 2011
16. Why NoSQL? NewsCred
Web 2.0+
The write intensive web
Thursday, February 3, 2011
17. Why NoSQL? NewsCred
Web 2.0+
The write intensive web
User-generated Content
Thursday, February 3, 2011
18. Why NoSQL? NewsCred
Web 2.0+
The write intensive web
Big Data
User-generated Content
Thursday, February 3, 2011
19. Why NoSQL? NewsCred
Web 2.0+
The write intensive web
Semi-structured Data Big Data
User-generated Content
Thursday, February 3, 2011
20. Why NoSQL? NewsCred
Web 2.0+
The write intensive web
Semi-structured Data Big Data
Semantic Web User-generated Content
Thursday, February 3, 2011
21. Why NoSQL? NewsCred
Web 2.0+
The write intensive web
Semi-structured Data Real-time Big Data
Semantic Web User-generated Content
Thursday, February 3, 2011
22. Why NoSQL? NewsCred
Web 2.0+
The write intensive web
Semi-structured Data Real-time Big Data
Semantic Web Ubiquity User-generated Content
Any device. Anywhere.
Thursday, February 3, 2011
23. Why NoSQL? NewsCred
The MySQL Problem
1. Default
Application
Data
Source
Writing
MySQL
User Reading
Thursday, February 3, 2011
24. Why NoSQL? NewsCred
The MySQL Problem
1. Default
Application
Bottleneck, too much load!
Data
Source
Writing
MySQL
User Reading
Thursday, February 3, 2011
25. Why NoSQL? NewsCred
The MySQL Problem
2. Replication
Application
Data
Source
Writing MySQL
Master
User Reading MySQL
Slaves
Thursday, February 3, 2011
26. Why NoSQL? NewsCred
The MySQL Problem
2. Replication
Application
Data
Source
Writing MySQL
Master
User Reading MySQL
Slaves
Scalable Reads!
Thursday, February 3, 2011
27. Why NoSQL? NewsCred
The MySQL Problem
2. Replication
Bottleneck, writes won’t scale!
Application
Data
Source
Writing MySQL
Master
User Reading MySQL
Slaves
Scalable Reads!
Thursday, February 3, 2011
28. Why NoSQL? NewsCred
The MySQL Problem
3. Sharding
Application
Data
Source
Writing S
MySQL
User Reading S
Thursday, February 3, 2011
29. Why NoSQL? NewsCred
The MySQL Problem
3. Sharding
Application
Great, scalable writes!
Data
Source
Writing S
MySQL
User Reading S
Thursday, February 3, 2011
30. Why NoSQL? NewsCred
The MySQL Problem
3. Sharding
Application
Great, scalable writes!
Data
Source
Writing S
MySQL
User Reading S
Development and maintenance
costs just skyrocketed!
Thursday, February 3, 2011
31. Why NoSQL? NewsCred
Web 2.0+
The write intensive web
Semi-structured Data Real-time Big Data
Semantic Web Ubiquity User-generated Content
Any device. Anywhere.
Thursday, February 3, 2011
32. Why NoSQL? NewsCred
The NoSQL Solution
Design Goals
Semi-structure >> Schema-free
Thursday, February 3, 2011
33. Why NoSQL? NewsCred
The NoSQL Solution
Design Goals
Semi-structure >> Schema-free
Big Data >> Scalable reads/writes
Thursday, February 3, 2011
34. Why NoSQL? NewsCred
The NoSQL Solution
Design Goals
Semi-structure >> Schema-free
Big Data >> Scalable reads/writes
Real-time >> High-performance
Thursday, February 3, 2011
35. Why NoSQL? NewsCred
The NoSQL Solution
Design Goals
Semi-structure >> Schema-free
Big Data >> Scalable reads/writes
Real-time >> High-performance
Ubiquity >> High-availability
Thursday, February 3, 2011
36. NoSQL vs RDMS NewsCred
NoSQL RDBMS
• Schema-free • Relational schema
• Scalable writes/reads • Scalable reads
vs
• Auto high-availability • Custom high-availability
Thursday, February 3, 2011
37. NoSQL vs RDMS NewsCred
NoSQL RDBMS
• Schema-free • Relational schema
• Scalable writes/reads • Scalable reads
vs
• Auto high-availability • Custom high-availability
• Limited queries • Flexible queries
• Eventual Consistency * • Consistency
• BASE • ACID
* Applies to most NoSQL systems
Thursday, February 3, 2011
38. Is NoSQL For You? NewsCred
NoSQL RDBMS
• Schema-free • Relational schema
• Scalable writes/reads • Scalable reads
vs
• Auto high-availability • Custom high-availability
• Limited queries • Flexible queries
• Eventual Consistency * • Consistency
• BASE • ACID
* Applies to most NoSQL systems
Thursday, February 3, 2011
39. Is NoSQL For You? NewsCred
NoSQL RDBMS
• Schema-free • Relational schema
• Scalable writes/reads • Scalable reads
vs
• Auto high-availability • Custom high-availability
• Limited queries • Flexible queries
• Eventual Consistency * • Consistency
• BASE • ACID
* Applies to most NoSQL systems
Thursday, February 3, 2011
40. Part 2
NoSQL Use Cases
NewsCred
Thursday, February 3, 2011
42. NoSQL Use Cases NewsCred
• Consumer Use Cases
• Facebook
• Twitter
• NetFlix
• Enterprise Use Cases
• Rackspace
• TrendMicro
• NewsCred
Thursday, February 3, 2011
43. NoSQL Use Cases NewsCred
• Facebook
• Hbase - Facebook messages
• Scribe - Real-time click logs
• Hive - SQL queries -> MapReduce jobs
• Hadoop
• Web analytics warehouse
• Distributed datastore
• MySQL backups
Thursday, February 3, 2011
44. NoSQL Use Cases NewsCred
• Twitter
• Hadoop - Analytics
• Hbase - People search
• Scribe - Log collection framework
• FlockDB - Social graph analysis
Thursday, February 3, 2011
45. NoSQL Use Cases NewsCred
• Rackspace
• Cassandra – stat collection, mail and apps
• TrendMicro
• Hbase & Hadoop – reputation databases
• NewsCred
• MongoDB
• API usage analytics
• Pixel tracking analytics
• Entity metadata storage
Thursday, February 3, 2011
46. Demo
NewsCred API Analytics
NewsCred
Thursday, February 3, 2011
47. Part 3
Choosing a NoSQL Solution
NewsCred
Thursday, February 3, 2011
48. Choosing a NoSQL Solution NewsCred
Availability
Each:client:can:always:read:and:write
A
RDBMSs Cassandra
MySQL: Voldemort
PostgreSQL CouchDB
Aster:Data CA AP Dynamo
GreenPlum SimpleDB
Vertica Tokyo:Cabinet
Riak
C P PartitionDtolerance:
Consistency CP
All:clients:have:the:same:view:of: The:system:works:well:despite:
the:data BigTable Scalaris physical:network:partitions
HyperTable Berkeley:DB
Hbase Memcache:DB
MongoDB Redis
Thursday, February 3, 2011
49. Consistent, Available (CA) NewsCred
CA-systems have trouble with partitions and
deal with it with replication.
• Examples
• MySQL (relational)
• Aster Data (relational)
• Greenplum (relational)
• Vertica (column)
Thursday, February 3, 2011
50. Availability, Partition-Tolerant (AP) NewsCred
AP-systems have trouble with consistency, achieve
“eventual consistency” through replication.
• Examples
• Cassandra (column/tabular)
• Dynamo (key-value)
• Voldemort (key-value)
• Tokyo Cabinet (key-value)
• CouchDB (document)
• SimpleDB (document)
• Riak (document)
Thursday, February 3, 2011
51. Consistent, Partition-Tolerant (CP) NewsCred
CP-systems have trouble with availability while
keeping data consistent across partitioned nodes.
• Examples
• MongoDB (document)
• BigTable (column/tabular)
• HyperTable (column/tabular)
• Hbase (column/tabular)
• Redis (key-value)
• Scalaris (key-value)
• MemcacheDB (key-value)
Thursday, February 3, 2011
52. Hbase NewsCred
Selling point: A
Billions of rows, millions of columns
Use when you need:
Random, real-time access to Big Data
C P
Written in: Java
License: Apache
Type: Column/Tabular
Protocol: HTTP/REST/Thrift Users:
Community Support: Good Yahoo!, Facebook, Microsoft, Adobe,
Learning Curve: High StumbleUpon etc.
Thursday, February 3, 2011
53. Cassandra NewsCred
Selling point: A
Best of Google BigTable and Amazon Dynamo
Use when you need:
To write more than you read (logging)
C P
Written in: Java
License: Apache
Type: Column/Tabular
Protocol: Custom, binary (Thrift) Users:
Community Support: Great Facebook, Twitter, Digg, Reddit,
Learning Curve: Medium Rackspace, Cisco, SimpleGeo, Cloudkick etc.
Thursday, February 3, 2011
54. Redis NewsCred
Selling point: A
Blazing fast, in-memory like memcached
Use when you need:
To manage rapidly changing data
C P
Written in: C/C++
License: BSD
Type: Key-value
Protocol: Telnet-like Users:
Community Support: Good Github, Craigslist, Stackoverflow,
Learning Curve: Low Disqus, The Guardian Uk etc.
Thursday, February 3, 2011
55. MongoDB NewsCred
Selling point: A
Best of NoSQL and RDBMS
Use when you need:
Dynamic queries and indexing on a Big DB
C P
Written in: C++
License: AGPL
Type: Document
Protocol: Custom, binary (BSON) Users:
Community Support: Great NewsCred, Foursquare, Github, Sourceforge,
Learning Curve: Low The New York Times, Etsy, Shutterfly etc.
Thursday, February 3, 2011
56. Part 4
Understanding MongoDB
NewsCred
Thursday, February 3, 2011
60. Understanding MongoDB NewsCred
• SELECT
SELECT * FROM users WHERE X = 3 AND Y = 'abc';
db.users.find({X:3, Y: ”abc”})
SELECT * FROM users WHERE X = 3 AND Y = 'abc' ORDER BY X ASC;
db.users.find({X:3, Y: ”abc”}).sort({X:1})
SELECT username, email FROM users WHERE X = 3 AND Y = 'abc';
db.users.find({X:3, Y: ”abc”}, {username:true, email:true})
Thursday, February 3, 2011
61. Understanding MongoDB NewsCred
• UPDATE
db.collection.update(criteria, modifier, upsert, multi)
criteria : Query which selects the record(s) to update
modifier : $set, $inc, $unset, $push, $pop...
upsert : Insert if not exists, update otherwise
multi : Update multiple docs matching the criteria
UPDATE users SET X = 4, Y = 'abc' WHERE username = 'joegunchy';
db.users.update({username:”joegunchy”}, {$set: {X:4, Y:'abc'}}, true, true)
Thursday, February 3, 2011
62. Understanding MongoDB NewsCred
• DELETE
db.articles.remove({}) /*remove all*/
db.articles.remove({tag:'sql'}) /*remove all articles with tag = 'sql'*/
db.articles.remove({tag:'sql'}) /*block other ops while removing*/
Thursday, February 3, 2011
64. Understanding MongoDB NewsCred
• Map/Reduce
• Algorithm introduced by Google for processing large
datasets on clusters
• MongoDB uses it for:
• Aggregation (Group By, Avg, Sum etc.)
• Batch processing jobs
Thursday, February 3, 2011
66. Understanding MongoDB NewsCred
• Map/Reduce Example
Document
We want to do something like...
Thursday, February 3, 2011
67. Understanding MongoDB NewsCred
• Map/Reduce Example
Map
Reduce
Thursday, February 3, 2011
68. Understanding MongoDB NewsCred
• Map/Reduce Example
Execute
Thursday, February 3, 2011
69. Understanding MongoDB NewsCred
• Map/Reduce Example
Result
Thursday, February 3, 2011
70. Part 5
Building a MongoDB App
NewsCred
Thursday, February 3, 2011
71. Part 6
Scaling with MongoDB
NewsCred
Thursday, February 3, 2011
72. Scaling with MongoDB NewsCred
• Scaling is a challenge
• No silver bullet
• Strategies
• Replication
• Replica Sets
• Auto-sharding
Thursday, February 3, 2011
73. Scaling with MongoDB NewsCred
Replication
Master
Slave Slave Slave
Thursday, February 3, 2011
74. Scaling with MongoDB NewsCred
Replica Sets
Secondary
User
Passive
Primary
Thursday, February 3, 2011
75. Scaling with MongoDB NewsCred
Replica Sets: Election
Synced,3ms,ago
C
Priority,1
A
Synced,1ms,ago
E
Priority,1
Priority 1
B
D
Priority,0
Thursday, February 3, 2011
76. Scaling with MongoDB NewsCred
• Replica Sets: Network Partition
• Election Process initiated
• When a node can’t reach primary
• When primary can’t reach majority of nodes in set
• New primary is elected by majority of nodes in set
• Node with the most recent data gets priority
• Arbiter node used to break ties
Thursday, February 3, 2011
77. Scaling with MongoDB NewsCred
• Auto-sharding
• Cluster handles sharding data and rebalancing
automatically
• No administrative headaches of manual sharding
• Application is oblivious to existence of shards
Thursday, February 3, 2011
78. Scaling with MongoDB NewsCred
Auto-sharding
Big$Collection
Thursday, February 3, 2011
79. Scaling with MongoDB NewsCred
Auto-sharding
User
Router)
Thursday, February 3, 2011
80. Scaling with MongoDB NewsCred
Auto-sharding
• Connect to a single server
• db = connect(‘localhost:27017’)
• Connect to a router
• db = connect(‘localhost:27017’)
User
Mongo)DB
Thursday, February 3, 2011
81. Scaling with MongoDB NewsCred
• When to shard?
• Running out of disk space
• Write intensive
• Need to keep large chunk of data in memory
• Don’t start out with a sharded collection!
• Shard “if and when” you need to
Thursday, February 3, 2011
82. Scaling with MongoDB NewsCred
• Choosing a Shard Key
• Incremental
• Example: timestamps i.e. ‘created_at’
• Queries on shard key is highly efficient
• Random
• Example: ‘username’
• Writes are distributed across multiple shards
Thursday, February 3, 2011
83. Scaling with MongoDB NewsCred
Sharding + Replica Sets
User
Router
P P
S S S S
Thursday, February 3, 2011
84. Questions? NewsCred
Iraj Islam
iraj@newscred.com, @irajislam
Rubayeet Islam
rubayeet@newscred.com, @rubayeet
Nurul Ferdous
nurul@newscred.com, @ferdous
Thursday, February 3, 2011