The document summarizes a presentation given by Ikai Lan on July 27, 2011 about Google App Engine's High Replication datastore. The presentation covered what App Engine is, the benefits of the High Replication datastore like strong consistency and high reliability, and how entity groups allow for transactional reads through ancestor queries. It also briefly discussed how Paxos is used underneath to provide strong consistency across datacenters.
webapp2 is a lightweight Python web framework compatible with Google App Engine’s webapp.
webapp2 is a single file that follows the simplicity of webapp, but improves it in some ways: it extends webapp to offer
better URI routing and exception handling, a full featured response object and a more flexible dispatching mechanism.
webapp2 also offers the package webapp2_extras with several optional utilities: sessions, localization, internationalization,
domain and subdomain routing, secure cookies and support for threaded environments.
webapp2 can also be used outside of Google App Engine, independently of the App Engine SDK.
For a complete description of how webapp2 improves webapp, see features.
Data Migrations in the App Engine DatastoreRyan Morlok
Data migration is a core problem when dealing with web frameworks. Rails and Django have their own built-in migration tools to help you manage data, but with Google Cloud Datastore, things are bit more manual. This presentation walks through several techniques and Python examples that leverage deferred tasks or map reduce to keep the data for your app consistent with the state of your code.
This talk describes how to build RESTful JSON services using Python on Google App engine. It looks at how this can be done with straight webapp2, Google Cloud Endpoints, and a library I wrote called Pytracts.
In recent years, we have seen an overwhelming number of TV commercials that promise that the Cloud can help with many problems, including some family issues. What stands behind the terms “Cloud” and “Cloud Computing,” and what we can actually expect from this phenomenon? A group of students of the Computer Systems Technology department and Dr. T. Malyuta, whom has been working with the Cloud technologies since its early days, will provide an overview of the business and technological aspects of the Cloud.
I promise that understand NoSQL is as easy as playing with LEGO bricks ! The Google Bigtable presented in 2006 is the inspiration for Apache HBase: let's take a deep dive into Bigtable to better understand Hbase.
webapp2 is a lightweight Python web framework compatible with Google App Engine’s webapp.
webapp2 is a single file that follows the simplicity of webapp, but improves it in some ways: it extends webapp to offer
better URI routing and exception handling, a full featured response object and a more flexible dispatching mechanism.
webapp2 also offers the package webapp2_extras with several optional utilities: sessions, localization, internationalization,
domain and subdomain routing, secure cookies and support for threaded environments.
webapp2 can also be used outside of Google App Engine, independently of the App Engine SDK.
For a complete description of how webapp2 improves webapp, see features.
Data Migrations in the App Engine DatastoreRyan Morlok
Data migration is a core problem when dealing with web frameworks. Rails and Django have their own built-in migration tools to help you manage data, but with Google Cloud Datastore, things are bit more manual. This presentation walks through several techniques and Python examples that leverage deferred tasks or map reduce to keep the data for your app consistent with the state of your code.
This talk describes how to build RESTful JSON services using Python on Google App engine. It looks at how this can be done with straight webapp2, Google Cloud Endpoints, and a library I wrote called Pytracts.
In recent years, we have seen an overwhelming number of TV commercials that promise that the Cloud can help with many problems, including some family issues. What stands behind the terms “Cloud” and “Cloud Computing,” and what we can actually expect from this phenomenon? A group of students of the Computer Systems Technology department and Dr. T. Malyuta, whom has been working with the Cloud technologies since its early days, will provide an overview of the business and technological aspects of the Cloud.
I promise that understand NoSQL is as easy as playing with LEGO bricks ! The Google Bigtable presented in 2006 is the inspiration for Apache HBase: let's take a deep dive into Bigtable to better understand Hbase.
Infrastructure and Workflow for the Formal Evaluation of Semantic Search Tech...Stuart Wrigley
This paper describes an infrastructure for the automated evaluation of semantic technologies and, in particular, semantic search technologies. For this purpose, we present an evaluation framework which follows a service-oriented approach for evaluating semantic technologies and uses the Business Process Execution Language (BPEL) to define evaluation workflows that can be executed by process engines. This framework supports a variety of evaluations, from different semantic areas, including search, and is extendible to new evaluations. We show how BPEL addresses this diversity as well as how it is used to solve specific challenges such as heterogeneity, error handling and reuse.
Presented at Data infrastructurEs for Supporting Information Retrieval Evaluation (DESIRE 2011) Workshop, Co-located with CIKM 2011, the 20th ACM Conference on Information and Knowledge Management
Friday 28th October 2011, Glasgow, UK
http://www.promise-noe.eu/events/desire-2011/
DealerTrack is the nation’s first and largest credit application network for the automotive industry, connecting 17,000 dealers and over 1000 lenders. Senior Director of Technology Architecture, and one of the founding members, Chris DeMeo detailed DealerTrack’s complex environment, spanning multiple geos and data centers and with the diverse architecture that comes with multiple acquisitions over the course of several years. As is common with our customers, they brought Splunk in for a unified view of their data.
“Our developers had full visibility into the anatomy of a problem to rectify quickly and avoid similar errors in the future. We’ve had so many ‘a-ha’ moments with Splunk, it’s become second nature to expect them,” Chris said.
DealerTrack has moved from reactive to proactive, and they have a much better understanding of their transaction volume, transaction mix, and watch dashboards for bunching or other challenges during load tests.
Slide deck for a presentation during a JavaScript meetup in Atlanta, GA.
This is an intro into titanium with a twist being that I focused on explaining some of the power titanium gives developers by allowing them to easily create their own UI versus using native graphics.
Skyline Innovations, a renewable energy company in Washington DC, uses MongoDB to store its time series data from its solar installations. This talk tells how, and why.
www.skylineinnovations.com
Given at MongoDC2011
Governing services, data, rules, processes and moreRandall Hauch
Randall and Kurt will present how Guvnor is being reborn so that it can manage artifacts from a variety of domains, including web services, data services, business rules and processes, and metadata management. Guvnor not only will storing these artifacts, but it will fully manage their lifecycle, enable search and discovery, and provide insight into how, when and where they can be used. They'll also describe Guvnor's architecture and use of JCR, REST, GWT, Atom, and S-RAMP.
PushToTest TestMaker 6.5 Open Source Test Design DocumentClever Moe
PushToTest TestMaker version 6.5 product design document for a major feature enhancement. Contains user interface definitions, product roadmap, and feature requirements. Please comment on this to improve TestMaker.
Open Source Test Workshop for CIOs, CTOs, ManagersClever Moe
This Open Source Test Workshop is for senior IT and business executives needing visibility and management tools and methodology into all the demand for IT. Shows how to bring Open Source Testing into your organization.
Slides for a talk at the Colorado Software Summit in 2008 that I did about growing Bumper Sticker, a Ruby on Rails Facebook app to over a billion pageviews.
Funny thing is ... I had to bail on the conference. Had to ship product.
More Related Content
Similar to 2011 july-gtug-high-replication-datastore
Infrastructure and Workflow for the Formal Evaluation of Semantic Search Tech...Stuart Wrigley
This paper describes an infrastructure for the automated evaluation of semantic technologies and, in particular, semantic search technologies. For this purpose, we present an evaluation framework which follows a service-oriented approach for evaluating semantic technologies and uses the Business Process Execution Language (BPEL) to define evaluation workflows that can be executed by process engines. This framework supports a variety of evaluations, from different semantic areas, including search, and is extendible to new evaluations. We show how BPEL addresses this diversity as well as how it is used to solve specific challenges such as heterogeneity, error handling and reuse.
Presented at Data infrastructurEs for Supporting Information Retrieval Evaluation (DESIRE 2011) Workshop, Co-located with CIKM 2011, the 20th ACM Conference on Information and Knowledge Management
Friday 28th October 2011, Glasgow, UK
http://www.promise-noe.eu/events/desire-2011/
DealerTrack is the nation’s first and largest credit application network for the automotive industry, connecting 17,000 dealers and over 1000 lenders. Senior Director of Technology Architecture, and one of the founding members, Chris DeMeo detailed DealerTrack’s complex environment, spanning multiple geos and data centers and with the diverse architecture that comes with multiple acquisitions over the course of several years. As is common with our customers, they brought Splunk in for a unified view of their data.
“Our developers had full visibility into the anatomy of a problem to rectify quickly and avoid similar errors in the future. We’ve had so many ‘a-ha’ moments with Splunk, it’s become second nature to expect them,” Chris said.
DealerTrack has moved from reactive to proactive, and they have a much better understanding of their transaction volume, transaction mix, and watch dashboards for bunching or other challenges during load tests.
Slide deck for a presentation during a JavaScript meetup in Atlanta, GA.
This is an intro into titanium with a twist being that I focused on explaining some of the power titanium gives developers by allowing them to easily create their own UI versus using native graphics.
Skyline Innovations, a renewable energy company in Washington DC, uses MongoDB to store its time series data from its solar installations. This talk tells how, and why.
www.skylineinnovations.com
Given at MongoDC2011
Governing services, data, rules, processes and moreRandall Hauch
Randall and Kurt will present how Guvnor is being reborn so that it can manage artifacts from a variety of domains, including web services, data services, business rules and processes, and metadata management. Guvnor not only will storing these artifacts, but it will fully manage their lifecycle, enable search and discovery, and provide insight into how, when and where they can be used. They'll also describe Guvnor's architecture and use of JCR, REST, GWT, Atom, and S-RAMP.
PushToTest TestMaker 6.5 Open Source Test Design DocumentClever Moe
PushToTest TestMaker version 6.5 product design document for a major feature enhancement. Contains user interface definitions, product roadmap, and feature requirements. Please comment on this to improve TestMaker.
Open Source Test Workshop for CIOs, CTOs, ManagersClever Moe
This Open Source Test Workshop is for senior IT and business executives needing visibility and management tools and methodology into all the demand for IT. Shows how to bring Open Source Testing into your organization.
Slides for a talk at the Colorado Software Summit in 2008 that I did about growing Bumper Sticker, a Ruby on Rails Facebook app to over a billion pageviews.
Funny thing is ... I had to bail on the conference. Had to ship product.
OSCON Google App Engine Codelab - July 2010ikailan
Slides for the App Engine codelab given on July 20, 2010. Note that a more verbose version of this codelab is available at:
https://sites.google.com/site/gdevelopercodelabs/app-engine/python-codelab
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
1. High Replication
Datastore
Ikai Lan
plus.ikailan.com
NYC GTUG
July 27, 2011
Wednesday, July 27, 2011
2. About the speaker
• Ikai Lan
• Developer Relations at Google based out
of San Francisco, CA
• Twitter: @ikai
• Google+: plus.ikailan.com
Wednesday, July 27, 2011
3. Agenda
• What is App Engine?
• What is High Replication datastore?
• Underneath the hood
Wednesday, July 27, 2011
15. App Engine
Datastore
Schemaless, non-relational
datastore built on top of
Google’s Bigtable technology
Enables rapid development
and scalability
Wednesday, July 27, 2011
16. High Replication
• strongly consistent
• multi datacenter
• High reliability
• consistent
performance
• no data loss
Wednesday, July 27, 2011
17. How do I use HR?
• Create a new application! Just remember
the rules
• Fetch by key and ancestor queries exhibit
strongly consistent behavior
• Queries without an ancestor exhibit
eventually consistent behavior
Wednesday, July 27, 2011
18. Strong vs. Eventual
• Strong consistency means immediately after
the datastore tells us the data has been
committed, a subsequent read will return
the data written
• Eventual consistency means that some time
after the datastore tells us data has been
committed, a read will return written data -
immediate read may or may not
Wednesday, July 27, 2011
19. This is strongly
consistent
DatastoreService datastore = DatastoreServiceFactory
.getDatastoreService();
Entity item = new Entity("Item");
item.setProperty("data", 123);
Key key = datastore.put(item);
// This exhibits strong consistency.
// It should return the item we just saved.
Entity result = datastore.get(key);
Wednesday, July 27, 2011
20. This is strongly
consistent
// Save the entity root
Entity root = new Entity("Root");
Key rootKey = datastore.put(root);
// Save the child
Entity childItem = new Entity("Item", rootKey);
childItem.setProperty("data", 123);
datastore.put(childItem);
Query strongConsistencyQuery = new Query("Item");
strongConsistencyQuery.setAncestor(rootKey);
strongConsistencyQuery.addFilter("data", FilterOperator.EQUAL, 123);
FetchOptions opts = FetchOptions.Builder.withDefaults();
// This query exhibits strong consistency.
// It will return the item we just saved.
List<Entity> results = datastore.prepare(strongConsistencyQuery)
.asList(opts);
Wednesday, July 27, 2011
21. This is eventually
consistent
Entity item = new Entity("Item");
item.setProperty("data", 123);
datastore.put(item);
// Not an ancestor query
Query eventuallyConsistentQuery = new Query("Item");
eventuallyConsistentQuery.addFilter("data", FilterOperator.EQUAL, 123);
FetchOptions opts = FetchOptions.Builder.withDefaults();
// This query exhibits eventual consistency.
// It will likely return an empty list.
List<Entity> results = datastore.prepare(eventuallyConsistentQuery)
.asList(opts);
Wednesday, July 27, 2011
22. Why?
• Reads are transactional
• On a read, we try to determine if we have
the latest version of some data
• If not, we catch up the data on the node to
the latest version
Wednesday, July 27, 2011
23. To understand this ...
• We need some understanding of Paxos ...
• ... which necessitates some understanding
of transactions
• ... which necessitates some understanding
of entity groups
Wednesday, July 27, 2011
24. Entity Groups
Entity
User
group root
Blog Blog
Entry Entry Entry
Comment
Comment Comment
Wednesday, July 27, 2011
25. Entity groups
// Save the entity root
Entity root = new Entity("Root");
Key rootKey = datastore.put(root);
// Save the child
Entity childItem = new Entity("Item", rootKey);
childItem.setProperty("data", 123);
datastore.put(childItem);
Query strongConsistencyQuery = new Query("Item");
strongConsistencyQuery.setAncestor(rootKey);
strongConsistencyQuery.addFilter("data", FilterOperator.EQUAL, 123);
FetchOptions opts = FetchOptions.Builder.withDefaults();
// This query exhibits strong consistency.
// It will return the item we just saved.
List<Entity> results = datastore.prepare(strongConsistencyQuery)
.asList(opts);
Wednesday, July 27, 2011
26. Entity groups
// Save the entity root
Entity root = new Entity("Root");
Key rootKey = datastore.put(root);
// Save the child
Entity childItem = new Entity("Item", rootKey);
childItem.setProperty("data", 123);
datastore.put(childItem);
Query strongConsistencyQuery = new Query("Item");
strongConsistencyQuery.setAncestor(rootKey);
strongConsistencyQuery.addFilter("data", FilterOperator.EQUAL, 123);
FetchOptions opts = FetchOptions.Builder.withDefaults();
// This query exhibits strong consistency.
// It will return the item we just saved.
List<Entity> results = datastore.prepare(strongConsistencyQuery)
.asList(opts);
Wednesday, July 27, 2011
27. Optimistic locking
Client A reads Client B
data. It's reads data.
current It's current
version is 11 version is 11
Modify data. Modify data.
Increment version Increment version
to 12 Datastore to 12
Client B tries
Client ! tries to to save data.
save data. Success!
Datastore
version is
higher or equal
than my
version - FAIL
Wednesday, July 27, 2011
28. Transactional reads
// Save the entity root
Entity root = new Entity("Root");
Key rootKey = datastore.put(root);
// Save the child
Entity childItem = new Entity("Item", rootKey);
childItem.setProperty("data", 123);
datastore.put(childItem);
Query strongConsistencyQuery = new Query("Item");
strongConsistencyQuery.setAncestor(rootKey);
strongConsistencyQuery.addFilter("data", FilterOperator.EQUAL, 123);
FetchOptions opts = FetchOptions.Builder.withDefaults();
// This query exhibits strong consistency.
// It will return the item we just saved.
List<Entity> results = datastore.prepare(strongConsistencyQuery)
.asList(opts);
Wednesday, July 27, 2011
29. Transactional reads
Still being committed
Blog Entry
Version 11
Comment Comment
Parent: Entry Parent: Entry
Version 11 Version 12
Client B
Client A reads
Datastore transactionally
data
writing data
Version 12 has not finished committing -
Datastore returns version 11
Wednesday, July 27, 2011
30. Paxos simplified
Give me the
newest data Node A Node B
Datastore
Client
Is my data
up to date?
Node C Node D
1. If the data is up to date, return it
2. if the data is NOT up to date, "catch up" the data
by applying the jobs in the journal and return the latest
data
Wednesday, July 27, 2011
31. More reading
• My example was grossly oversimplified
• More details can be found here:
http://www.cidrdb.org/cidr2011/Papers/
CIDR11_Paper32.pdf
Wednesday, July 27, 2011
32. Contradictory advice
• Entity groups must be as big as possible to
cover as much related data as you can
• Entity groups must be small enough such
that your write rate per entity group never
goes above one write/second
Wednesday, July 27, 2011
33. Summary
• Remember the rules of strong consistency
and eventual consistency
• Group your data into entity groups when
possible and use ancestor queries
Wednesday, July 27, 2011
