This presentation will cover the history of creation, implementation details and various challenges related to embedded documents querying in MongoDB, along with examples of how to properly create and utilize the extension on top of official MongoDB Scala Driver. This newly introduced extension allows to fully utilize Spray JSON and represents bidirectional serialization for case classes in BSON, as well as flexible DSL for MongoDB query operators, documents and collections.
This document provides biographical information about the speaker Matt Stine and outlines the topics he will cover in his presentation. Stine will discuss how software designs tend to degrade over time due to various factors, and how following the SOLID principles of object-oriented design can help address this problem and make designs more functional in nature. He will also cover trends in how software systems evolve, different programming paradigms, and the ongoing quest for software design "best practices."
Functional Programming Patterns (NDC London 2014)Scott Wlaschin
(video of these slides available here http://fsharpforfunandprofit.com/fppatterns/)
In object-oriented development, we are all familiar with design patterns such as the Strategy pattern and Decorator pattern, and design principles such as SOLID.
The functional programming community has design patterns and principles as well.
This talk will provide an overview of some of these, and present some demonstrations of FP design in practice.
The document discusses principles and best practices for writing clean code, including using meaningful names, separating commands and queries, avoiding repetition, using exceptions instead of return codes, and following object-oriented principles like polymorphism instead of switch statements on objects. It provides examples of good and bad code for concepts like single responsibility, primitive obsession, and refused bequest. The overall goal is to write code that is readable, maintainable, and extendable.
Download for better quality - Learn about the composition of effectful functions through the work of Bartosz Milewski, Rob Norris, Rúnar Bjarnason and Debasish Ghosh.
The Functional Programming Triad of Map, Filter and FoldPhilip Schwarz
This slide deck is my homage to SICP, the book which first introduced me to the Functional Programming triad of map, filter and fold.
It was during my Computer Science degree that a fellow student gave me a copy of the first edition, not long after the book came out.
I have not yet come across a better introduction to these three functions.
The upcoming slides are closely based on the second edition of the book, a free online copy of which can be found here:
https://mitpress.mit.edu/sites/default/files/sicp/full-text/book/book.html.
Download for original image quality.
Errata:
slide 20: the Clojure map function is in fact the Scheme one repeated - see code below for correction.
Scheme code: https://github.com/philipschwarz/the-fp-triad-of-map-filter-and-fold-scheme
Clojure code: https://github.com/philipschwarz/the-fp-triad-of-map-filter-and-fold-clojure
(Video and code at http://fsharpforfunandprofit.com/composition)
Composition is a fundamental principle of functional programming, but how is it different from an object-oriented approach, and how do you use it in practice?
In this talk for beginners, we'll start by going over the basic concepts of functional programming, and then look at some different ways that composition can be used to build large things from small things.
After that, we'll see how composition is used in practice, beginning with a simple FizzBuzz example, and ending with a complete (object-free!) web application.
Left and Right Folds- Comparison of a mathematical definition and a programm...Philip Schwarz
We compare typical definitions of the left and right fold functions, with their mathematical definitions in Sergei Winitzki’s upcoming book: The Science of Functional Programming.
Errata:
Slide 13: "The way 𝑓𝑜𝑙𝑑𝑙 does it is by associating to the right" - should, of course ,end in "to the left".
This document provides biographical information about the speaker Matt Stine and outlines the topics he will cover in his presentation. Stine will discuss how software designs tend to degrade over time due to various factors, and how following the SOLID principles of object-oriented design can help address this problem and make designs more functional in nature. He will also cover trends in how software systems evolve, different programming paradigms, and the ongoing quest for software design "best practices."
Functional Programming Patterns (NDC London 2014)Scott Wlaschin
(video of these slides available here http://fsharpforfunandprofit.com/fppatterns/)
In object-oriented development, we are all familiar with design patterns such as the Strategy pattern and Decorator pattern, and design principles such as SOLID.
The functional programming community has design patterns and principles as well.
This talk will provide an overview of some of these, and present some demonstrations of FP design in practice.
The document discusses principles and best practices for writing clean code, including using meaningful names, separating commands and queries, avoiding repetition, using exceptions instead of return codes, and following object-oriented principles like polymorphism instead of switch statements on objects. It provides examples of good and bad code for concepts like single responsibility, primitive obsession, and refused bequest. The overall goal is to write code that is readable, maintainable, and extendable.
Download for better quality - Learn about the composition of effectful functions through the work of Bartosz Milewski, Rob Norris, Rúnar Bjarnason and Debasish Ghosh.
The Functional Programming Triad of Map, Filter and FoldPhilip Schwarz
This slide deck is my homage to SICP, the book which first introduced me to the Functional Programming triad of map, filter and fold.
It was during my Computer Science degree that a fellow student gave me a copy of the first edition, not long after the book came out.
I have not yet come across a better introduction to these three functions.
The upcoming slides are closely based on the second edition of the book, a free online copy of which can be found here:
https://mitpress.mit.edu/sites/default/files/sicp/full-text/book/book.html.
Download for original image quality.
Errata:
slide 20: the Clojure map function is in fact the Scheme one repeated - see code below for correction.
Scheme code: https://github.com/philipschwarz/the-fp-triad-of-map-filter-and-fold-scheme
Clojure code: https://github.com/philipschwarz/the-fp-triad-of-map-filter-and-fold-clojure
(Video and code at http://fsharpforfunandprofit.com/composition)
Composition is a fundamental principle of functional programming, but how is it different from an object-oriented approach, and how do you use it in practice?
In this talk for beginners, we'll start by going over the basic concepts of functional programming, and then look at some different ways that composition can be used to build large things from small things.
After that, we'll see how composition is used in practice, beginning with a simple FizzBuzz example, and ending with a complete (object-free!) web application.
Left and Right Folds- Comparison of a mathematical definition and a programm...Philip Schwarz
We compare typical definitions of the left and right fold functions, with their mathematical definitions in Sergei Winitzki’s upcoming book: The Science of Functional Programming.
Errata:
Slide 13: "The way 𝑓𝑜𝑙𝑑𝑙 does it is by associating to the right" - should, of course ,end in "to the left".
Nat, List and Option Monoids -from scratch -Combining and Folding -an examplePhilip Schwarz
Nat, List and Option Monoids, from scratch. Combining and Folding: an example.
Code: https://github.com/philipschwarz/nat-list-and-option-monoids-from-scratch-combining-and-folding-an-example
Arriving at monads by going from pure-function composition to effectful-funct...Philip Schwarz
This document discusses functional programming principles and how to deal with effects like mutability, exceptions, and logging in a pure functional style. It introduces the idea of effects, distinguishing them from side effects. Six common effects are discussed: partiality, nondeterminism, dependency injection, logging, mutability, and exceptions. The document explains how the Kleisli category and monads can be used to define functions with effects and allow their composition while preserving referential transparency. This provides techniques for handling effects normally found in imperative programs using functional programming.
This year ECMA International will be ratifying the biggest update to the JavaScript language in its history. In this talk we'll look at key features already appearing in browsers as well as those coming in the near future. We'll also explore how you can begin leveraging the power of ES6 across all browsers today. If you haven't looked at JavaScript recently, you soon realize that a bigger, better world awaits.
Designing an extensible, flexible schema that supports user customization is a common requirement, but it's easy to paint yourself into a corner.
Examples of extensible database requirements:
- A database that allows users to declare new fields on demand.
- Or an e-commerce catalog with many products, each with distinct attributes.
- Or a content management platform that supports extensions for custom data.
The solutions we use to meet these requirements is overly complex and the performance is terrible. How should we find the right balance between schema and schemaless database design?
I'll briefly cover the disadvantages of Entity-Attribute-Value (EAV), a problematic design that's an example of the antipattern called the Inner-Platform Effect, That is, modeling an attribute-management system on top of the RDBMS architecture, which already provides attributes through columns, data types, and constraints.
Then we'll discuss the pros and cons of alternative data modeling patterns, with respect to developer productivity, data integrity, storage efficiency and query performance, and ease of extensibility.
- Class Table Inheritance
- Serialized BLOB
- Inverted Indexing
Finally we'll show tools like pt-online-schema-change and new features of MySQL 5.6 that take the pain out of schema modifications.
Node.js is an exciting new platform developed by Ryan Dahl that allows JavaScript developers to create extremely high performance servers by leveraging Google's V8 JavaScript engine and asynchronous I/O. It uses a non-blocking I/O model that makes it lightweight and efficient, as well as features like CommonJS modules and the npm package manager. The document provides examples of building simple HTTP servers and modules in Node.js to demonstrate its capabilities.
Regular expressions (regex) allow defining patterns to match in text strings using special characters like ., *, + to match types of characters or ranges. They have components like literal characters, character classes, shorthand classes, quantifiers and modifiers that define what is matched. PHP supports both PCRE and POSIX regex with functions like preg_match() and ereg_replace() and there are online tools to test and learn regular expressions.
The session conducted by DSC DYPCOE Pune. It's based on the book "Clean Code" by Robert C. Martin.
Any fool can write a code which machine can understand, good programmer write a code which humans can understand.
(video and more at http://fsharpforfunandprofit.com/fppatterns)
In object-oriented development, we are all familiar with design patterns such as the Strategy pattern and Decorator pattern, and design principles such as SOLID. The functional programming community has design patterns and principles as well. This talk will provide an overview of some of these patterns (such as currying, monads), and present some demonstrations of FP design in practice. We'll also look at some of the ways you can use these patterns as part of a domain driven design process, with some simple real world examples in F#. No jargon, no maths, and no prior F# experience necessary.
Domain Driven Design with the F# type System -- NDC London 2013Scott Wlaschin
(Video of these slides here http://fsharpforfunandprofit.com/ddd)
Statically typed functional programming languages like F# encourage a very different way of thinking about types. The type system is your friend, not an annoyance, and can be used in many ways that might not be familiar to OO programmers.
Types can be used to represent the domain in a fine-grained, self documenting way. And in many cases, types can even be used to encode business rules so that you literally cannot create incorrect code. You can then use the static type checking almost as an instant unit test — making sure that your code is correct at compile time.
In this talk, we'll look at some of the ways you can use types as part of a domain driven design process, with some simple real world examples in F#. No jargon, no maths, and no prior F# experience necessary.
Code, links to video, etc., at http://fsharpforfunandprofit.com/ddd
For more on DDD and F# see:
http://fsharpforfunandprofit.com/ddd/
http://tomasp.net/blog/type-first-development.aspx/
http://gorodinski.com/blog/2013/02/17/domain-driven-design-with-fsharp-and-eventstore/
The document discusses principles of clean code, beginning with definitions of clean code from various authors. It outlines key principles like the single responsibility principle and open/closed principle. It then covers sections on meaningful naming, functions, and classes. For functions, it recommends they be small, do one thing, avoid flag arguments, and have no side effects. For classes, it advocates for small size, encapsulation, high cohesion and low coupling following the single responsibility principle.
This document provides an overview of functional programming concepts including:
- Functional programming focuses on immutable data and treating computation as mathematical functions. It emphasizes eliminating side effects.
- Key concepts include first-class functions, higher-order functions, pure functions, recursion, currying, and pattern matching.
- Functional programming languages often use immutable data, recursion instead of looping, and avoid mutable state through techniques like immutable data structures and actor models.
Not so long ago Microsoft announced a new language trageting on front-end developers. Everybody's reaction was like: Why?!! Is it just Microsoft darting back to Google?!
So, why a new language? JavaScript has its bad parts. Mostly you can avoid them or workaraund. You can emulate class-based OOP style, modules, scoping and even run-time typing. But that is doomed to be clumsy. That's not in the language design. Google has pointed out these flaws, provided a new language and failed. Will the story of TypeScript be any different?
(Video available at https://fsharpforfunandprofit.com/composition/)
Composition is a fundamental building block of functional programming, but how is it different from an object-oriented approach, and how do you use it in practice?
In this talk for beginners, we'll start by going over the basic concepts of functional programming, and then look at some different ways that composition can be used to build large things from small things.
After that, we'll see how composition is used in practice, beginning with some basic examples, and ending with a complete (object-free!) web application.
The document discusses the differences between objects and data structures. Data structures expose data but usually have no behavior, while objects encapsulate data and expose functions. Procedural code is easier to add new functions to but harder to add new data structures, while object-oriented code is easier to add new classes but harder to add new functions. Combining data structures and objects has the disadvantages of both and is difficult to refactor. The document also discusses the Law of Demeter which limits how deeply methods can call methods of other objects.
This document discusses various techniques for optimizing MySQL queries, including queries for exclusion joins, random selection, and greatest per group. For a query seeking movies without directors, solutions using NOT EXISTS, NOT IN, and outer joins are examined. The outer join solution performed best by taking advantage of a "not exists" optimization. For random selection of a movie, an initial naive solution using ORDER BY RAND() is shown to be inefficient, prompting discussion of alternative approaches.
(Video available at http://fsharpforfunandprofit.com/monadster/)
You've got a pile of assorted functions lying around. Each one is useful and reliable, but they just don't fit together properly. How can you assemble them into a complete system that can stand on its own two feet and terrorize the local villagers?
In this session, I'll show how functional programming can transform all sorts of existing code into shapes that are plug-compatible and which can be bolted together effortlessly.
SAFETY NOTE: The techniques demonstrated are perfectly harmless and can even be used at your workplace -- no lightning bolts required.
This document provides an agenda for a LINQ training that covers:
- A brief history of data access technologies leading to LINQ
- An overview of what LINQ is and the C# language enhancements that support it
- Base concepts like LINQ to Objects, deferred and immediate operators
- More advanced concepts like LINQ to Datasets, XML, and SQL
- Examples and questions
Going Native: Leveraging the New JSON Native Datatype in Oracle 21cJim Czuprynski
Need to incorporate JSON documents into existing Oracle database applications? The new native JSON datatype introduced in Oracle 21c makes it simple to store, access, traverse, and filter the complex data often found within JSON documents, often without any application code changes.
Http4s, Doobie and Circe: The Functional Web StackGaryCoady
Http4s, Doobie and Circe together form a nice platform for building web services. This presentations provides an introduction to using them to build your own service.
Nat, List and Option Monoids -from scratch -Combining and Folding -an examplePhilip Schwarz
Nat, List and Option Monoids, from scratch. Combining and Folding: an example.
Code: https://github.com/philipschwarz/nat-list-and-option-monoids-from-scratch-combining-and-folding-an-example
Arriving at monads by going from pure-function composition to effectful-funct...Philip Schwarz
This document discusses functional programming principles and how to deal with effects like mutability, exceptions, and logging in a pure functional style. It introduces the idea of effects, distinguishing them from side effects. Six common effects are discussed: partiality, nondeterminism, dependency injection, logging, mutability, and exceptions. The document explains how the Kleisli category and monads can be used to define functions with effects and allow their composition while preserving referential transparency. This provides techniques for handling effects normally found in imperative programs using functional programming.
This year ECMA International will be ratifying the biggest update to the JavaScript language in its history. In this talk we'll look at key features already appearing in browsers as well as those coming in the near future. We'll also explore how you can begin leveraging the power of ES6 across all browsers today. If you haven't looked at JavaScript recently, you soon realize that a bigger, better world awaits.
Designing an extensible, flexible schema that supports user customization is a common requirement, but it's easy to paint yourself into a corner.
Examples of extensible database requirements:
- A database that allows users to declare new fields on demand.
- Or an e-commerce catalog with many products, each with distinct attributes.
- Or a content management platform that supports extensions for custom data.
The solutions we use to meet these requirements is overly complex and the performance is terrible. How should we find the right balance between schema and schemaless database design?
I'll briefly cover the disadvantages of Entity-Attribute-Value (EAV), a problematic design that's an example of the antipattern called the Inner-Platform Effect, That is, modeling an attribute-management system on top of the RDBMS architecture, which already provides attributes through columns, data types, and constraints.
Then we'll discuss the pros and cons of alternative data modeling patterns, with respect to developer productivity, data integrity, storage efficiency and query performance, and ease of extensibility.
- Class Table Inheritance
- Serialized BLOB
- Inverted Indexing
Finally we'll show tools like pt-online-schema-change and new features of MySQL 5.6 that take the pain out of schema modifications.
Node.js is an exciting new platform developed by Ryan Dahl that allows JavaScript developers to create extremely high performance servers by leveraging Google's V8 JavaScript engine and asynchronous I/O. It uses a non-blocking I/O model that makes it lightweight and efficient, as well as features like CommonJS modules and the npm package manager. The document provides examples of building simple HTTP servers and modules in Node.js to demonstrate its capabilities.
Regular expressions (regex) allow defining patterns to match in text strings using special characters like ., *, + to match types of characters or ranges. They have components like literal characters, character classes, shorthand classes, quantifiers and modifiers that define what is matched. PHP supports both PCRE and POSIX regex with functions like preg_match() and ereg_replace() and there are online tools to test and learn regular expressions.
The session conducted by DSC DYPCOE Pune. It's based on the book "Clean Code" by Robert C. Martin.
Any fool can write a code which machine can understand, good programmer write a code which humans can understand.
(video and more at http://fsharpforfunandprofit.com/fppatterns)
In object-oriented development, we are all familiar with design patterns such as the Strategy pattern and Decorator pattern, and design principles such as SOLID. The functional programming community has design patterns and principles as well. This talk will provide an overview of some of these patterns (such as currying, monads), and present some demonstrations of FP design in practice. We'll also look at some of the ways you can use these patterns as part of a domain driven design process, with some simple real world examples in F#. No jargon, no maths, and no prior F# experience necessary.
Domain Driven Design with the F# type System -- NDC London 2013Scott Wlaschin
(Video of these slides here http://fsharpforfunandprofit.com/ddd)
Statically typed functional programming languages like F# encourage a very different way of thinking about types. The type system is your friend, not an annoyance, and can be used in many ways that might not be familiar to OO programmers.
Types can be used to represent the domain in a fine-grained, self documenting way. And in many cases, types can even be used to encode business rules so that you literally cannot create incorrect code. You can then use the static type checking almost as an instant unit test — making sure that your code is correct at compile time.
In this talk, we'll look at some of the ways you can use types as part of a domain driven design process, with some simple real world examples in F#. No jargon, no maths, and no prior F# experience necessary.
Code, links to video, etc., at http://fsharpforfunandprofit.com/ddd
For more on DDD and F# see:
http://fsharpforfunandprofit.com/ddd/
http://tomasp.net/blog/type-first-development.aspx/
http://gorodinski.com/blog/2013/02/17/domain-driven-design-with-fsharp-and-eventstore/
The document discusses principles of clean code, beginning with definitions of clean code from various authors. It outlines key principles like the single responsibility principle and open/closed principle. It then covers sections on meaningful naming, functions, and classes. For functions, it recommends they be small, do one thing, avoid flag arguments, and have no side effects. For classes, it advocates for small size, encapsulation, high cohesion and low coupling following the single responsibility principle.
This document provides an overview of functional programming concepts including:
- Functional programming focuses on immutable data and treating computation as mathematical functions. It emphasizes eliminating side effects.
- Key concepts include first-class functions, higher-order functions, pure functions, recursion, currying, and pattern matching.
- Functional programming languages often use immutable data, recursion instead of looping, and avoid mutable state through techniques like immutable data structures and actor models.
Not so long ago Microsoft announced a new language trageting on front-end developers. Everybody's reaction was like: Why?!! Is it just Microsoft darting back to Google?!
So, why a new language? JavaScript has its bad parts. Mostly you can avoid them or workaraund. You can emulate class-based OOP style, modules, scoping and even run-time typing. But that is doomed to be clumsy. That's not in the language design. Google has pointed out these flaws, provided a new language and failed. Will the story of TypeScript be any different?
(Video available at https://fsharpforfunandprofit.com/composition/)
Composition is a fundamental building block of functional programming, but how is it different from an object-oriented approach, and how do you use it in practice?
In this talk for beginners, we'll start by going over the basic concepts of functional programming, and then look at some different ways that composition can be used to build large things from small things.
After that, we'll see how composition is used in practice, beginning with some basic examples, and ending with a complete (object-free!) web application.
The document discusses the differences between objects and data structures. Data structures expose data but usually have no behavior, while objects encapsulate data and expose functions. Procedural code is easier to add new functions to but harder to add new data structures, while object-oriented code is easier to add new classes but harder to add new functions. Combining data structures and objects has the disadvantages of both and is difficult to refactor. The document also discusses the Law of Demeter which limits how deeply methods can call methods of other objects.
This document discusses various techniques for optimizing MySQL queries, including queries for exclusion joins, random selection, and greatest per group. For a query seeking movies without directors, solutions using NOT EXISTS, NOT IN, and outer joins are examined. The outer join solution performed best by taking advantage of a "not exists" optimization. For random selection of a movie, an initial naive solution using ORDER BY RAND() is shown to be inefficient, prompting discussion of alternative approaches.
(Video available at http://fsharpforfunandprofit.com/monadster/)
You've got a pile of assorted functions lying around. Each one is useful and reliable, but they just don't fit together properly. How can you assemble them into a complete system that can stand on its own two feet and terrorize the local villagers?
In this session, I'll show how functional programming can transform all sorts of existing code into shapes that are plug-compatible and which can be bolted together effortlessly.
SAFETY NOTE: The techniques demonstrated are perfectly harmless and can even be used at your workplace -- no lightning bolts required.
This document provides an agenda for a LINQ training that covers:
- A brief history of data access technologies leading to LINQ
- An overview of what LINQ is and the C# language enhancements that support it
- Base concepts like LINQ to Objects, deferred and immediate operators
- More advanced concepts like LINQ to Datasets, XML, and SQL
- Examples and questions
Going Native: Leveraging the New JSON Native Datatype in Oracle 21cJim Czuprynski
Need to incorporate JSON documents into existing Oracle database applications? The new native JSON datatype introduced in Oracle 21c makes it simple to store, access, traverse, and filter the complex data often found within JSON documents, often without any application code changes.
Http4s, Doobie and Circe: The Functional Web StackGaryCoady
Http4s, Doobie and Circe together form a nice platform for building web services. This presentations provides an introduction to using them to build your own service.
These are slides from our Big Data Warehouse Meetup in April. We talked about NoSQL databases: What they are, how they’re used and where they fit in existing enterprise data ecosystems.
Mike O’Brian from 10gen, introduced the syntax and usage patterns for a new aggregation system in MongoDB and give some demonstrations of aggregation using the new system. The new MongoDB aggregation framework makes it simple to do tasks such as counting, averaging, and finding minima or maxima while grouping by keys in a collection, complementing MongoDB’s built-in map/reduce capabilities.
For more information, visit our website at http://casertaconcepts.com/ or email us at info@casertaconcepts.com.
The document provides an overview of MongoDB including key concepts like collections, documents, queries, indexes, aggregation, and more. It compares MongoDB to a traditional SQL database and shows examples of common operations like insert, update, find in MongoDB shell and Node.js driver. It also briefly covers more advanced topics like geospatial queries, optimization, replication and sharding.
This document summarizes MongoDB, an open-source document database. It discusses MongoDB's key features like schema-less document storage, rich queries, auto-sharding and replication. It provides examples of CRUD operations using the mongo shell and comparisons to SQL. The document also outlines some use cases, driver support, limitations and resources for learning more.
This document discusses using MongoDB as the database for a RESTful web API. Some key points:
- MongoDB's schema-less and JSON-like data structure makes it a natural fit for storing and retrieving JSON data from a REST API. The document formats can be directly mapped to and from the database.
- Common CRUD operations like getting, inserting, updating and deleting documents can be easily implemented with MongoDB's simple API. Queries can be expressed in JSON format.
- MongoDB is lightweight, allows flexible schemas, and has fast performance for common operations needed in a REST API backed by a database.
- The document uses examples of how a hypothetical REST API for a "contacts" resource might work,
Building DSLs with the Spoofax Language WorkbenchEelco Visser
This document discusses the Spoofax language workbench, which is a tool for building domain-specific languages (DSLs). It addresses key concerns in developing DSLs such as the tradeoffs between internal and external DSLs, textual vs graphical syntax, code generation vs interpretation, and static vs dynamic typing. The Spoofax workbench uses declarative syntax definitions and rewrite rules to provide features like syntax checking, error recovery, and incremental processing for integrated development environments.
The document discusses MongoDB, a non-relational database that uses documents with a flexible schema rather than tables, and is well-suited for applications that need to store data in complex, nested structures. It provides an overview of key MongoDB concepts like collections, queries, indexing, updating documents, and aggregation capabilities. Examples are given of how MongoDB can be used for applications involving user profiles, blogs, and logging.
Google apps script database abstraction exposed versionBruce McPherson
This document describes a database abstraction library for Google Apps Script that provides a consistent API for NoSQL databases. It allows code to be reused across different database backends by handling queries, authentication, caching, and more transparently. The library exposes the capabilities through a JSON REST API that can be accessed from other programming languages. It also includes a VBA client library that translates VBA calls into requests to the JSON API, allowing VBA code to access databases in the same way as Google Apps Script.
MongoDB + Java - Everything you need to know Norberto Leite
Learn everything you need to know to get started building a MongoDB-based app in Java. We'll explore the relationship between MongoDB and various languages on the Java Virtual Machine such as Java, Scala, and Clojure. From there, we'll examine the popular frameworks and integration points between MongoDB and the JVM including Spring Data and object-document mappers like Morphia.
Learn everything you need to know to get started building a MongoDB-based app in Java. We'll explore the relationship between MongoDB and various languages on the Java Virtual Machine such as Java, Scala, and Clojure. From there, we'll examine the popular frameworks and integration points between MongoDB and the JVM including Spring Data and object-document mappers like Morphia.
This document provides an introduction to NoSQL and MongoDB. It discusses key concepts like horizontal scalability, the CAP theorem, BASE consistency, and MongoDB's document-oriented data model using JSON-like documents and collections. It also covers common MongoDB operations like inserting, querying, updating, and deleting documents.
This document discusses using CouchDB on Android applications. It provides instructions on adding CouchDB functionality to an Android project using Couchbase Mobile for Android. It also describes how to access a CouchDB instance from an Android device using Futon and the Couchbase Service. The document contains code examples for creating, reading, updating and deleting CouchDB documents from an Android app.
Introduces the use of JSON-Like data structures within DivConq. Includes how to create and access the structures within Java, as well as how to declare structures in dcSchema for data validation.
This document provides an overview of MongoDB, including its key features such as document-oriented storage, full index support, replication and high availability, auto sharding, querying capabilities, and fast in-place updates. It also discusses MongoDB's architecture for replication, sharding, and configuration servers.
Dev Jumpstart: Build Your First App with MongoDBMongoDB
This document provides an overview of MongoDB and how to build a simple blogging application with it. It describes MongoDB as a document database where documents are analogous to JSON objects and collections are analogous to tables in a relational database. It demonstrates how to install and run MongoDB, insert sample data for users and blog posts, query the data, and update documents. The document also lists various MongoDB drivers and provides pointers for next steps in learning about data modeling, replication, performance, and scaling in MongoDB.
Recent releases of the .NET driver have added lots of cool new features. In this webinar we will highlight some of the most important ones. We will begin by discussing serialization. We will describe how serialization is normally handled, and how you can customize the process when you need to, including some tips on migration strategies when your class definitions change. We will continue with a discussion of the new Query builder, which now includes support for typed queries. A major new feature of recent releases is support for LINQ queries. We will show you how the .NET driver supports LINQ and discuss what kinds of LINQ queries are supported. Finally, we will discuss what you need to do differently in your application when authentication is enabled at the server.
What do you mean, Backwards Compatibility?Trisha Gee
Lessons learnt developing the new Java driver for MongoDB. This is a totally different version of my backwards compatibility talk, delivered at JFokus.
Similar to Spray Json and MongoDB Queries: Insights and Simple Tricks. (20)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Julian Hyde
SQL has attained widespread adoption, but Business Intelligence tools still use their own higher level languages based upon a multidimensional paradigm. Composable calculations are what is missing from SQL, and we propose a new kind of column, called a measure, that attaches a calculation to a table. Like regular tables, tables with measures are composable and closed when used in queries.
SQL-with-measures has the power, conciseness and reusability of multidimensional languages but retains SQL semantics. Measure invocations can be expanded in place to simple, clear SQL.
To define the evaluation semantics for measures, we introduce context-sensitive expressions (a way to evaluate multidimensional expressions that is consistent with existing SQL semantics), a concept called evaluation context, and several operations for setting and modifying the evaluation context.
A talk at SIGMOD, June 9–15, 2024, Santiago, Chile
Authors: Julian Hyde (Google) and John Fremlin (Google)
https://doi.org/10.1145/3626246.3653374
Enhanced Screen Flows UI/UX using SLDS with Tom KittPeter Caitens
Join us for an engaging session led by Flow Champion, Tom Kitt. This session will dive into a technique of enhancing the user interfaces and user experiences within Screen Flows using the Salesforce Lightning Design System (SLDS). This technique uses Native functionality, with No Apex Code, No Custom Components and No Managed Packages required.
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdfVALiNTRY360
Salesforce Healthcare CRM, implemented by VALiNTRY360, revolutionizes patient management by enhancing patient engagement, streamlining administrative processes, and improving care coordination. Its advanced analytics, robust security, and seamless integration with telehealth services ensure that healthcare providers can deliver personalized, efficient, and secure patient care. By automating routine tasks and providing actionable insights, Salesforce Healthcare CRM enables healthcare providers to focus on delivering high-quality care, leading to better patient outcomes and higher satisfaction. VALiNTRY360's expertise ensures a tailored solution that meets the unique needs of any healthcare practice, from small clinics to large hospital systems.
For more info visit us https://valintry360.com/solutions/health-life-sciences
Flutter is a popular open source, cross-platform framework developed by Google. In this webinar we'll explore Flutter and its architecture, delve into the Flutter Embedder and Flutter’s Dart language, discover how to leverage Flutter for embedded device development, learn about Automotive Grade Linux (AGL) and its consortium and understand the rationale behind AGL's choice of Flutter for next-gen IVI systems. Don’t miss this opportunity to discover whether Flutter is right for your project.
Most important New features of Oracle 23c for DBAs and Developers. You can get more idea from my youtube channel video from https://youtu.be/XvL5WtaC20A
A neural network is a machine learning program, or model, that makes decisions in a manner similar to the human brain, by using processes that mimic the way biological neurons work together to identify phenomena, weigh options and arrive at conclusions.
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...XfilesPro
Wondering how X-Sign gained popularity in a quick time span? This eSign functionality of XfilesPro DocuPrime has many advancements to offer for Salesforce users. Explore them now!
The Key to Digital Success_ A Comprehensive Guide to Continuous Testing Integ...kalichargn70th171
In today's business landscape, digital integration is ubiquitous, demanding swift innovation as a necessity rather than a luxury. In a fiercely competitive market with heightened customer expectations, the timely launch of flawless digital products is crucial for both acquisition and retention—any delay risks ceding market share to competitors.
Malibou Pitch Deck For Its €3M Seed Roundsjcobrien
French start-up Malibou raised a €3 million Seed Round to develop its payroll and human resources
management platform for VSEs and SMEs. The financing round was led by investors Breega, Y Combinator, and FCVC.
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdfBaha Majid
IBM watsonx Code Assistant for Z, our latest Generative AI-assisted mainframe application modernization solution. Mainframe (IBM Z) application modernization is a topic that every mainframe client is addressing to various degrees today, driven largely from digital transformation. With generative AI comes the opportunity to reimagine the mainframe application modernization experience. Infusing generative AI will enable speed and trust, help de-risk, and lower total costs associated with heavy-lifting application modernization initiatives. This document provides an overview of the IBM watsonx Code Assistant for Z which uses the power of generative AI to make it easier for developers to selectively modernize COBOL business services while maintaining mainframe qualities of service.
WWDC 2024 Keynote Review: For CocoaCoders AustinPatrick Weigel
Overview of WWDC 2024 Keynote Address.
Covers: Apple Intelligence, iOS18, macOS Sequoia, iPadOS, watchOS, visionOS, and Apple TV+.
Understandable dialogue on Apple TV+
On-device app controlling AI.
Access to ChatGPT with a guest appearance by Chief Data Thief Sam Altman!
App Locking! iPhone Mirroring! And a Calculator!!
Odoo releases a new update every year. The latest version, Odoo 17, came out in October 2023. It brought many improvements to the user interface and user experience, along with new features in modules like accounting, marketing, manufacturing, websites, and more.
The Odoo 17 update has been a hot topic among startups, mid-sized businesses, large enterprises, and Odoo developers aiming to grow their businesses. Since it is now already the first quarter of 2024, you must have a clear idea of what Odoo 17 entails and what it can offer your business if you are still not aware of it.
This blog covers the features and functionalities. Explore the entire blog and get in touch with expert Odoo ERP consultants to leverage Odoo 17 and its features for your business too.
An Overview of Odoo ERP
Odoo ERP was first released as OpenERP software in February 2005. It is a suite of business applications used for ERP, CRM, eCommerce, websites, and project management. Ten years ago, the Odoo Enterprise edition was launched to help fund the Odoo Community version.
When you compare Odoo Community and Enterprise, the Enterprise edition offers exclusive features like mobile app access, Odoo Studio customisation, Odoo hosting, and unlimited functional support.
Today, Odoo is a well-known name used by companies of all sizes across various industries, including manufacturing, retail, accounting, marketing, healthcare, IT consulting, and R&D.
The latest version, Odoo 17, has been available since October 2023. Key highlights of this update include:
Enhanced user experience with improvements to the command bar, faster backend page loading, and multiple dashboard views.
Instant report generation, credit limit alerts for sales and invoices, separate OCR settings for invoice creation, and an auto-complete feature for forms in the accounting module.
Improved image handling and global attribute changes for mailing lists in email marketing.
A default auto-signature option and a refuse-to-sign option in HR modules.
Options to divide and merge manufacturing orders, track the status of manufacturing orders, and more in the MRP module.
Dark mode in Odoo 17.
Now that the Odoo 17 announcement is official, let’s look at what’s new in Odoo 17!
What is Odoo ERP 17?
Odoo 17 is the latest version of one of the world’s leading open-source enterprise ERPs. This version has come up with significant improvements explained here in this blog. Also, this new version aims to introduce features that enhance time-saving, efficiency, and productivity for users across various organisations.
Odoo 17, released at the Odoo Experience 2023, brought notable improvements to the user interface and added new functionalities with enhancements in performance, accessibility, data analysis, and management, further expanding its reach in the market.
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesQuickdice ERP
Explore the seamless transition to e-invoicing with this comprehensive guide tailored for Saudi Arabian businesses. Navigate the process effortlessly with step-by-step instructions designed to streamline implementation and enhance efficiency.
UI5con 2024 - Keynote: Latest News about UI5 and it’s EcosystemPeter Muessig
Learn about the latest innovations in and around OpenUI5/SAPUI5: UI5 Tooling, UI5 linter, UI5 Web Components, Web Components Integration, UI5 2.x, UI5 GenAI.
Recording:
https://www.youtube.com/live/MSdGLG2zLy8?si=INxBHTqkwHhxV5Ta&t=0
Using Query Store in Azure PostgreSQL to Understand Query PerformanceGrant Fritchey
Microsoft has added an excellent new extension in PostgreSQL on their Azure Platform. This session, presented at Posette 2024, covers what Query Store is and the types of information you can get out of it.
Using Query Store in Azure PostgreSQL to Understand Query Performance
Spray Json and MongoDB Queries: Insights and Simple Tricks.
1. Spray Json and MongoDB Queries
Insights and Simple Tricks
2. Spray JSON
spray-json is a lightweight, clean and efficient JSON implementation in Scala.
It is currently maintained by the Akka team at Lightbend.
https://github.com/spray/spray-json
3. MongoDB / NoSQL / BSON
Introduction to MongoDB: https://docs.mongodb.com/manual/introduction/
MongoDB is an open-source document database that provides high performance, high
availability, and automatic scaling. MongoDB stores data records as BSON documents.
BSON is a binary representation of JSON documents, though it contains more data types
than JSON. https://docs.mongodb.com/manual/core/document
NoSQL Databases Explained: https://www.mongodb.com/nosql-explained
4. MongoDB Scala Driver
MongoDB Scala Driver introduced in Sep 2015.
http://rosslawley.co.uk/introducing-mongodb-scala-driver/
The driver requires to define BSON codecs to work with custom data types.
Case classes support added in Mar 2017.
http://mongodb.github.io/mongo-scala-driver/2.0/changelog/
https://github.com/mongodb/mongo-scala-driver/releases/tag/r2.0.0
5. Application components
Models, JSON formats for API and BSON codecs for MongoDB
INTERNAL
API / WEB
SERVICES MODELS
{ JSON }
FORMATS
MongoDB
API
EXTERNAL
API / WEB
SERVICES
{ BSON }
CODECS
6. Coding example of application components
Models, JSON formats for API and BSON codecs for MongoDB
// MODEL
case class Test(id: Long, number: Int, comment: String)
// JSON PROTOCOL
trait TestJsonProtocol extends DefaultJsonProtocol {
implicit def testJf = jsonFormat3(Test)
}
// BSON CODEC
class TestBsonCodec extends Codec[Test] {
override def getEncoderClass: Class[Test] = classOf[Test]
override def encode(writer: BsonWriter, value: Test, encoderContext: EncoderContext) = // ...
override def decode(reader: BsonReader, decoderContext: DecoderContext): Test = // ...
}
val registry = fromRegistries(fromCodecs(new TestBsonCodec()), DEFAULT_CODEC_REGISTRY)
val database: MongoDatabase = MongoClient().getDatabase("mydb").withCodecRegistry(registry)
val collection: MongoCollection[Test] = database.getCollection("test-collection")
8. Implementation of BSON codec decode
class TestBsonCodec extends Codec[Test] {
override def decode(reader: BsonReader, decoderContext: DecoderContext): Test = {
reader.readStartDocument()
reader.readObjectId("_id")
val number = reader.readInt32()
val id = reader.readInt64()
val comment = reader.readString()
reader.readEndDocument()
Test(id, number, comment)
}
}
9. Use the test codec to write and read test record
collection.insertOne(Test(123L, 123, "some string data"))
.toFuture().onComplete(res => println(s"INSERT: $res"))
INSERT: Success(The operation completed successfully)
collection.find(Filters.eq("id", 123L))
.toFuture().onComplete(res => println(s"FIND: $res"))
FIND: Failure(org.bson.BsonInvalidOperationException: readInt32 can only be
called when CurrentBSONType is INT32, not when CurrentBSONType is INT64.)
10. Implementation of BSON codec decode
read fields by name
class TestScalaCodec extends Codec[Test] {
override def decode(reader: BsonReader, decoderContext: DecoderContext): Test = {
reader.readStartDocument()
reader.readObjectId("_id")
val number = reader.readInt32("number")
val id = reader.readInt64("id")
val comment = reader.readString("comment")
reader.readEndDocument()
Test(id, number, comment)
}
}
11. Use the test codec to write and read test record
collection.insertOne(Test(123L, 123, "some string data"))
.toFuture().onComplete(res => println(s"INSERT: $res"))
INSERT: Success(The operation completed successfully)
collection.find(Filters.eq("id", 123L))
.toFuture().onComplete(res => println(s"FIND: $res"))
FIND: Failure(org.bson.BsonSerializationException: Expected element name to be
'number', not 'id'.)
12. MongoDB and BSON
BSON is a binary format in which zero or more ordered key/value pairs are stored as a single
entity.
http://bsonspec.org/spec.html
MongoDB represents JSON documents in binary-encoded format called BSON behind the
scenes. BSON extends the JSON model to provide additional data types, ordered fields, and
to be efficient for encoding and decoding within different languages.
https://www.mongodb.com/json-and-bson
13. Motivation: reuse existing JSON formats in MongoDB
INTERNAL
API / WEB
SERVICES
MODELS
{ JSON / BSON }
FORMATS
MongoDB
API
EXTERNAL
API / WEB
SERVICES
{ BSON }
CODECS
14. MongoDB Extended JSON
http://mongodb.github.io/mongo-scala-driver/2.6/bson/extended-json/
The Scala driver supports reading and writing BSON documents represented as MongoDB
Extended JSON.
Furthermore, the Document provides two sets of convenient methods for this purpose:
● Document.toJson(): a set of overloaded methods that convert a Document instance to a
JSON string
● Document(json): a set of overloaded static factory methods that convert a JSON string to a
Documentinstance
15. Motivation: reuse existing JSON formats in MongoDB
INTERNAL
API / WEB
SERVICES
MODELS
{ JSON / BSON }
FORMATS
MongoDB
API
EXTERNAL
API / WEB
SERVICES
{ BSON }
CODECS
16. Data transformation pipeline
[T]
// custom data types
{ JSON }
// related JSON
representation
Document(JSON)
// Document from JSON
BSON
// Binary data
in MongoDB
21. Override default LongJsonFormat in JsonProtocol
https://github.com/spray/spray-json/blob/v1.3.5/src/main/scala/spray/json/BasicFormats.scala#L33-L39
package spray.json
/**
* Provides the JsonFormats for the most important Scala types.
*/
trait BasicFormats {
implicit object LongJsonFormat extends JsonFormat[Long] {
def write(x: Long) = JsNumber(x)
def read(value: JsValue) = value match {
case JsNumber(x) => x.longValue
case x => deserializationError("Expected Long as JsNumber, but got " + x)
}
}
// …
object LongJsonFormat cannot override final member
22. Hide LongJsonFormat in JsonProtocol
Liskov substitution principle
Subtype Requirement: Let φ( 𝓍) be a property provable about objects 𝓍 of type T. Then
φ(y) should be true for objects y of type S where S is a subtype of T.
https://en.wikipedia.org/wiki/Liskov_substitution_principle
Error: overriding object LongJsonFormat in trait BasicFormats;
value LongJsonFormat has weaker access privileges; it should be public
23. Hide LongJsonFormat from import
Don’t extend DefaultJsonProtocol
trait TestJsonProtocol {
import DefaultJsonProtocol.{ LongJsonFormat => _, _ }
implicit val LongJsonFormat: JsonFormat[Long] = new JsonFormat[Long] {
def read(jsValue: JsValue): Long = jsValue match {
case JsObject(fields) => fields("$numberLong") match {
case JsString(v) => v.toLong
case _ => deserializationError("Long expected")
}
case JsNumber(v) => v.toLong
case _ => deserializationError("Long expected")
}
override def write(obj: Long): JsValue = JsObject("$numberLong" -> JsString(obj.toString))
}
implicit val testJf = jsonFormat3(Test)
}
27. LongJsonFormat is available in both JSON protocols
import DefaultJsonProtocol._
import TestJsonProtocol._
collection.find(Document(s"""{ "id": ${2147483648L.toJson} }""")).toFuture()
.map { bson => println(s"FIND BSON: $bson"); bson.map(_.toJson()) }
.map { json => println(s"FIND JSON: $json"); json.map(_.parseJson.convertTo[Test]) }
.map { obj => println(s"FIND OBJ: $obj") }
Error: Cannot find JsonWriter or JsonFormat type class for Long
collection.find(Document(s"""{ "id": ${2147483648L.toJson} }""")).toFuture()
Error: not enough arguments for method toJson: (implicit writer:
spray.json.JsonWriter[Long])spray.json.JsValue. Unspecified value parameter writer.
collection.find(Document(s"""{ "id": ${2147483648L.toJson} }""")).toFuture()
28. Hide LongJsonFormat from import
Both queries by long id and int number work well
import DefaultJsonProtocol.{ LongJsonFormat => _, _ }
import TestJsonProtocol._
collection.find(Document(s"""{ "id": ${2147483648L.toJson} }""")).toFuture()
.map { bson => bson.map(_.toJson().parseJson.convertTo[Test]) }
.map { obj => println(s"FIND BY ID (long): $obj") }
// will print
FIND BY ID (long): List(Test(2147483648,123,some string data))
collection.find(Document(s"""{ "number": ${123.toJson} }""")).toFuture()
.map { bson => bson.map(_.toJson().parseJson.convertTo[Test]) }
.map { obj => println(s"FIND BY NUMBER (int): $obj") }
// will print
FIND BY NUMBER (int): List(Test(2147483648,123,some string data))
29. GreenLeafJsonProtocol
trait GreenLeafJsonProtocol
extends StandardFormats
with CollectionFormats
with ProductFormats
with AdditionalFormats {
implicit val IntJsonFormat: JsonFormat[Int] = DefaultJsonProtocol.IntJsonFormat
implicit val LongJsonFormat: JsonFormat[Long] = DefaultJsonProtocol.LongJsonFormat
// …
}
object GreenLeafJsonProtocol extends GreenLeafJsonProtocol
New GreenLeafJsonProtocol based on DefaultJsonProtocol from Spray JSON. It allows to
override predefined JsonFormats to make possible use custom seriallization in BSON.
30. GreenLeafBsonProtocol
Override LongJsonFormat to serialize Long to BSON
trait GreenLeafBsonProtocol extends GreenLeafJsonProtocol {
override implicit val LongJsonFormat: JsonFormat[Long] = new JsonFormat[Long] {
def read(jsValue: JsValue): Long = jsValue match {
case JsObject(fields) => fields("$numberLong") match {
case JsString(v) => v.toLong
case _ => deserializationError("Long expected")
}
case JsNumber(v) => v.toLong
case _ => deserializationError("Long expected")
}
override def write(obj: Long): JsValue = JsObject("$numberLong" -> JsString(obj.toString))
}
}
object GreenLeafBsonProtocol extends GreenLeafBsonProtocol
31. Define JSON and BSON protocols for Test entity
// MODEL
case class Test(id: Long, number: Int, notes: String)
// JSON
trait TestJsonProtocol extends GreenLeafJsonProtocol {
implicit def testJf: RootJsonFormat[Test] = jsonFormat3(Test)
}
object TestJsonProtocol extends TestJsonProtocol
// BSON
object TestBsonProtocol extends TestJsonProtocol with GreenLeafBsonProtocol
32. Verify Test JSON protocol
import TestJsonProtocol._
val obj = Test(0xC0FFEE, 12648430, "Coffee")
println(obj.toJson.prettyPrint)
// will print
{
"id": 12648430,
"notes": "Coffee",
"number": 12648430
}
33. Verify Test BSON protocol
import TestBsonProtocol._
val obj = Test(0xC0FFEE, 12648430, "Coffee")
println(obj.toJson.prettyPrint)
// will print
{
"id": {
"$numberLong": "12648430"
},
"notes": "Coffee",
"number": 12648430
}
34. // import DefaultJsonProtocol.{ LongJsonFormat => _, _ }
// import TestJsonProtocol._
import TestBsonProtocol._
collection.find(Document(s"""{ "id": ${2147483648L.toJson} }""")).toFuture()
.map { bson => bson.map(_.toJson().parseJson.convertTo[Test]) }
.map { obj => println(s"FIND BY ID (long): $obj") }
// will print
FIND BY ID (long): List(Test(2147483648,123,some string data))
collection.find(Document(s"""{ "number": ${123.toJson} }""")).toFuture()
.map { bson => bson.map(_.toJson().parseJson.convertTo[Test]) }
.map { obj => println(s"FIND BY NUMBER (int): $obj") }
// will print
FIND BY NUMBER (int): List(Test(2147483648,123,some string data))
Use Test BSON protocol to write and to read BSON
35. Support for additional types
GreenLeaf JSON and BSON protocols provide formats for basic types such as Int and Long.
However, they also support various other common types such as Enumeration and
ZonedDateTime.
36. Scala Enumeration
Defines a set of values specific to the enumeration. Typically these values enumerate
all possible forms that can take and provide a lightweight alternative to case classes.
https://www.scala-lang.org/api/current/scala/Enumeration.html
// Define a new enumeration with a type alias and work with the full set of enumerated values
object WeekDay extends Enumeration {
type WeekDay = Value
val Mon, Tue, Wed, Thu, Fri, Sat, Sun = Value
}
WeekDay.values.foreach(x => println(s"${x.id}t$x"))
// output:
// 0 Mon
// 1 Tue
// 2 Wed
// 3 Thu
// 4 Fri
// 5 Sat
// 6 Sun
37. Support scala Enumeration in JSON/BSON protocols
trait GreenLeafJsonProtocol {
def enumToJsonFormatAsString(e: Enumeration): JsonFormat[e.Value] = new JsonFormat[e.Value] {
def write(v: e.Value): JsValue = JsString(v.toString)
def read(value: JsValue): e.Value = value match {
case JsString(v) => e.withName(v)
case x => deserializationError(s"Expected enum, but got $x")
}
}
def enumToJsonFormatAsInt(e: Enumeration): JsonFormat[e.Value] = new JsonFormat[e.Value] {
def write(v: e.Value): JsValue = JsNumber(v.id)
def read(value: JsValue): e.Value = value match {
case JsNumber(v) => e.apply(v.intValue())
case x => deserializationError(s"Expected enum, but got $x")
}
}
}
38. Verify scala Enumeration JSON/BSON formats
// MODEL
object StockSymbol extends Enumeration {
type StockSymbol = Value
val AAPL, GOOG, MSFT, AMZN, FB, BABA, JNJ, JPM = Value
}
case class Quote(symbol: StockSymbol, open: BigDecimal, high: BigDecimal, low: BigDecimal)
// JSON
trait StockJsonProtocol extends GreenLeafJsonProtocol {
implicit val stockJf: JsonFormat[StockSymbol] = enumToJsonFormatAsString(StockSymbol)
implicit val quoteJf: RootJsonFormat[Quote] = jsonFormat4(Quote)
}
object StockJsonProtocol extends StockJsonProtocol
// BSON
object StockBsonProtocol extends StockJsonProtocol with GreenLeafBsonProtocol
40. ObjectId type
ObjectIds are small, likely unique, fast to generate and ordered.
ObjectId values consist of 12 bytes, where the first four bytes represent timestamp that
reflect the ObjectId’s creation.
In MongoDB, each document stored in collection requires a unique _id field that acts as a
primary key. If an inserted document omits the _id field, the MongoDB driver automatically
generates an ObjectId for the _id field.
https://docs.mongodb.com/manual/reference/bson-types/#objectid
41. Add ObjectId JSON format
trait GreenLeafJsonProtocol {
implicit val ObjectIdJsonFormat: JsonFormat[ObjectId] = new JsonFormat[ObjectId] {
def write(obj: ObjectId): JsValue = JsString(obj.toString)
def read(jsValue: JsValue): ObjectId = jsValue match {
case JsString(value) => new ObjectId(value)
case x => deserializationError(s"Expected ObjectId, but got $x")
}
}
}
42. Add ObjectId BSON format
https://docs.mongodb.com/manual/reference/mongodb-extended-json/#oid
trait GreenLeafBsonProtocol {
override implicit val ObjectIdJsonFormat: JsonFormat[ObjectId] = new JsonFormat[ObjectId] {
def write(obj: ObjectId): JsValue = JsObject("$oid" -> JsString(obj.toString))
def read(jsValue: JsValue): ObjectId = jsValue match {
case JsObject(fields) => fields("$oid") match {
case JsString(oid) => new ObjectId(oid)
case x => deserializationError("Expected ObjectId, but got " + x)
}
case x => deserializationError("Expected ObjectId, but got " + x)
}
}
}
43. Sorting on ObjectId field
MongoDB clients should add an _id field with a unique ObjectId. Using ObjectIds for the _id
field provides the following additional benefits:
● in the mongo shell, users can access the creation time of the ObjectId by utilizing the
ObjectId.getTimestamp() method.
● sorting on an _id field that stores ObjectId values is roughly equivalent to sorting by
creation time.
https://docs.mongodb.com/manual/reference/bson-types/#objectid
44. // MODEL
case class Message(id: ObjectId = new ObjectId, text: String)
// JSON
trait MessageJsonProtocol extends GreenLeafJsonProtocol {
implicit val messageJf: RootJsonFormat[Message] = jsonFormat2(Message)
}
object MessageJsonProtocol extends MessageJsonProtocol
// BSON
object MessageBsonProtocol extends MessageJsonProtocol with GreenLeafBsonProtocol {
override implicit val messageJf: RootJsonFormat[Message] = jsonFormat(Message, "_id", "text")
}
Define model and JSON/BSON protocols
to verify ObjectId formats
46. Find test records and sort them by _id field
import MessageBsonProtocol._
collection.find(/* all messages */).sort(Document("""{"_id": 1}""")).toFuture()
.map { bson => bson.map(_.toJson().parseJson.convertTo[Message]) }
.map { messages => println(messages.mkString("n")) }
// output:
Message(5c8863e0d639e92a5f953273,A)
Message(5c8863e0d639e92a5f953274,B)
Message(5c8863e0d639e92a5f953275,C)
Message(5c8863e0d639e92a5f953276,D)
Message(5c8863e0d639e92a5f953277,E)
…
Message(5c8863e0d639e92a5f953288,V)
Message(5c8863e0d639e92a5f953289,W)
Message(5c8863e0d639e92a5f95328a,X)
Message(5c8863e0d639e92a5f95328b,Y)
Message(5c8863e0d639e92a5f95328c,Z)
47. Important note about sorting on ObjectId field
While ObjectId values should increase over time, they are not necessarily monotonic for the
following reasons:
● They only contain one second of temporal resolution, so ObjectId values created
within the same second do not have a guaranteed ordering
● They get generated by clients, which may have differing system clocks
https://docs.mongodb.com/manual/reference/bson-types/#objectid
48. UUID JSON format
Simple serialization to string by default which is possible to override
trait GreenLeafJsonProtocol {
implicit val UuidAsStrJsonFormat: JsonFormat[UUID] = new JsonFormat[UUID] {
def write(v: UUID): JsValue = JsString(v.toString)
def read(value: JsValue): UUID = value match {
case JsString(v) => UUID.fromString(v)
case x => deserializationError(s"Expected UUID, but got $x")
}
}
}
49. ZonedDateTime
ZonedDateTime is an immutable representation of a date-time with a time-zone.
This class stores all date and time fields to a precision of nanoseconds and a time-zone, with a
zone offset used to handle ambiguous local date-times.
https://docs.oracle.com/javase/8/docs/api/java/time/ZonedDateTime.html
50. ZonedDateTime JSON format
trait GreenLeafJsonProtocol {
implicit val ZdtJsonFormat: JsonFormat[ZonedDateTime] = new JsonFormat[ZonedDateTime] {
def write(obj: ZonedDateTime): JsValue = JsString(obj.format(DateTimePattern))
def read(jsValue: JsValue): ZonedDateTime = jsValue match {
// 1970-01-01T01:02:03+04:00
case JsString(zdt) if zdt.length >= 20 => parseDateTimeIso(zdt)
// 1970-01-01T00:00:00
case JsString(zdt) if zdt.length >= 19 && zdt.contains('T') => parseDateTimeIso(zdt)
// 1970-01-01 00:00:00
case JsString(zdt) if zdt.length == 19 => parseDateTime(zdt)
case JsString(zdt) => parseDate(zdt)
case x => deserializationError(s"Expected ZonedDateTime, but got $x")
}
}
}
51. BSON date type
BSON Date is a 64-bit integer that represents the number of milliseconds since the Unix
epoch.
This results in a representable date range of about 290 million years into the past and future.
The official BSON specification refers to the BSON Date type as the UTC datetime.
BSON Date type is signed. Negative values represent dates before 1970.
https://docs.mongodb.com/manual/reference/bson-types/#date
52. ZonedDateTime BSON format
trait GreenLeafBsonProtocol {
override implicit val ZdtJsonFormat: JsonFormat[ZonedDateTime] = new JsonFormat[ZonedDateTime] {
def write(obj: ZonedDateTime): JsValue = {
JsObject("$date" -> JsNumber(obj.toInstant.toEpochMilli))
}
def read(jsValue: JsValue): ZonedDateTime = jsValue match {
case JsObject(fields) => fields("$date") match {
case JsNumber(v) => Instant.ofEpochMilli(v.toLong).atZone(ZoneOffset.UTC)
case x => deserializationError("Expected ZonedDateTime, but got " + x)
}
case x => deserializationError("Expected ZonedDateTime, but got " + x)
}
}
}
53. Verify ZonedDateTime JSON/BSON formats
// MODEL
case class LogMessage(text: String, timestamp: ZonedDateTime)
// JSON
trait LogMessageJsonProtocol extends GreenLeafJsonProtocol {
implicit def logMessageJf = jsonFormat2(LogMessage)
}
object LogMessageJsonProtocol extends LogMessageJsonProtocol
// BSON
object LogMessageBsonProtocol extends LogMessageJsonProtocol with GreenLeafBsonProtocol
56. MongoDB Extended JSON and Date format
In Strict mode, <date> is an ISO-8601 date format with a mandatory time zone field following
the template YYYY-MM-DDTHH:mm:ss.mmm<+/-Offset>.
The MongoDB JSON parser currently DOESN’T support loading ISO-8601 strings representing
dates prior to the Unix epoch.
https://docs.mongodb.com/manual/reference/mongodb-extended-json/#date
58. BigDecimal type
BigDecimal represents decimal floating-point numbers of an arbitrary precision.
By default, the precision approximately matches that of IEEE 128-bit floating point numbers
(34 decimal digits, HALF_EVEN rounding mode).
https://www.scala-lang.org/api/current/scala/math/BigDecimal.html
59. Verify BigDecimal BSON format
// MODEL
case class MathematicalConstant(value: BigDecimal, symbol: String, description: String)
// JSON
trait MathematicalConstantJsonProtocol extends GreenLeafJsonProtocol {
implicit def logMessageJf = jsonFormat3(MathematicalConstant)
}
object MathematicalConstantJsonProtocol extends MathematicalConstantJsonProtocol
// BSON
object MathematicalConstantBsonProtocol extends MathematicalConstantJsonProtocol with GreenLeafBsonProtocol
import MathematicalConstantBsonProtocol._
val pi = MathematicalConstant(BigDecimal("3.141592653589793238462643383279"), "π", "Archimedes' constant")
val e = MathematicalConstant(BigDecimal("2.718281828459045235360287471352"), "e", "Euler number")
collection.insertOne(Document(pi.toJson.compactPrint)).toFuture().onComplete(x => println(x))
collection.insertOne(Document(e.toJson.compactPrint)).toFuture().onComplete(x => println(x))
// output:
Success(The operation completed successfully)
Success(The operation completed successfully)
61. MongoDB Extended JSON: NumberDecimal
The mongo shell treats all numbers as 64-bit floating-point double values by default.
The mongo shell provides the NumberDecimal() constructor to explicitly specify 128-bit decimal-based
floating-point values capable of emulating decimal rounding with exact precision.
This functionality is intended for applications that handle monetary data, such as financial, tax, and
scientific computations.
The decimal BSON type uses the IEEE 754 decimal128 floating-point numbering format which supports 34
decimal digits (i.e. significant digits) and an exponent range of −6143 to +6144.
https://docs.mongodb.com/manual/core/shell-types/#numberdecimal
https://docs.mongodb.com/manual/tutorial/model-monetary-data/#using-the-decimal-bson-type
https://docs.mongodb.com/manual/reference/mongodb-extended-json/#numberdecimal
62. BigDecimal BSON format
trait GreenLeafBsonProtocol {
override implicit val BigDecimalJsonFormat: JsonFormat[BigDecimal] = new JsonFormat[BigDecimal] {
override def read(jsValue: JsValue): BigDecimal = jsValue match {
case JsObject(fields) => fields("$numberDecimal") match {
case JsString(v) => BigDecimal(v)
case x => deserializationError("Expected BigDecimal/NumberDecimal, but got " + x)
}
case x => deserializationError("Expected BigDecimal/NumberDecimal, but got " + x)
}
override def write(obj: BigDecimal): JsValue = {
JsObject("$numberDecimal" -> JsString(obj.toString()))
}
}
}
64. Order of fields in JSON, BSON and MongoDB documents
JSON is built on two structures: a collection of name/value pairs and an ordered list of values.
An object is an unordered set of name/value pairs.
https://www.json.org/
BSON is a binary format in which zero or more ordered key/value pairs are stored as a single entity.
http://bsonspec.org/spec.html
MongoDB preserves the order of the document fields following write operations except for the following cases:
● The _id field is always the first field in the document.
● Updates that include renaming of field names may result in the reordering of fields in the document.
https://docs.mongodb.com/manual/core/document/#embedded-documents
65. Example of different order of JSON fields
case class Test(i: Int, l: Long)
object TestJsonProtocol extends GreenLeafJsonProtocol { implicit val testJf = jsonFormat2(Test) }
import TestJsonProtocol._
println(Test(1, 1024L).toJson)
// "spray-json" % "1.3.5" output:
// {"i":1,"l":1024}
// "spray-json" % "1.3.4" output:
// {"i":1,"l":1024}
case class Test(i: Int, l: Long, f: Float, d: Double)
object TestJsonProtocol extends GreenLeafJsonProtocol { implicit val testJf = jsonFormat4(Test) }
import TestJsonProtocol._
println(Test(1, 1024L, 2.0f, 20.48d).toJson)
// "spray-json" % "1.3.5" output:
// {"d":20.48,"f":2.0,"i":1,"l":1024}
// "spray-json" % "1.3.4" output:
// {"i":1,"l":1024,"f":2.0,"d":20.48}
66. JSON fields order and related MongoDB filter issue
// MODEL
object Currency extends Enumeration {
type Currency = Value
val USD, GBP, CAD, PLN, JPY, EUR = Value
}
import Currency._
// ID as object { "id": { "base": "USD", "date": "2019-01-18" }, "rates": ... }
case class ExchangeRateId(base: Currency, date: ZonedDateTime)
// In official driver macro codecs don't allow to use Enum value as key in Map data structure
case class ExchangeRate(id: ExchangeRateId, rates: Map[Currency, BigDecimal], updated: ZonedDateTime)
// BSON
object ExchangeRateJsonProtocol extends GreenLeafBsonProtocol {
implicit def ccyJf = enumToJsonFormatAsString(Currency)
implicit def erIdJf = jsonFormat2(ExchangeRateId)
implicit def erJf = jsonFormat(ExchangeRate, "_id", "rates", "updated")
}
69. JSON fields order and related MongoDB filter issue
plain query where filter fields order doesn’t match 2nd
record fields order
val filter = Document("""{ "_id": {"base": "USD", "date": { "$date": "2019-01-02T00:00:00.000Z" } } }""")
val filter = Filters.eq("_id", Document(ExchangeRateId(USD, "2019-01-02").toJson.compactPrint))
val filter = Document(s"""{ "_id": ${ExchangeRateId(USD, "2019-01-02").toJson} }""")
// FILTER: {"_id": {"base": "USD", "date": {"$date": 1546387200000}}}
// output:
BSON: List()
JSON: List()
OBJ: List()
70. MongoDB query operator ‘$eq’
Specifies equality condition. The $eq operator matches documents where the value of a field
equals the specified value. The $eq expression is equivalent to { field: <value> }.
If the specified <value> is a document, the order of the fields in the document matters.
https://docs.mongodb.com/manual/reference/operator/query/eq/#match-a-document-value
Equality matches on the whole embedded document require an exact match of the specified
<value> document, including the field order.
https://docs.mongodb.com/manual/tutorial/query-embedded-documents/#match-an-embedded-nested-document
71. JSON fields order and related MongoDB filter issue
plain query where filter fields order matches 2nd
record fields order
// explicit correct fields order as stored in MongoDB for this record
val filter = Document("""{"_id": { "date": { "$date": "2019-01-02T00:00:00.000Z" }, "base": "USD" } }""")
// wrong fields order
// val filter = Filters.eq("_id", Document(ExchangeRateId(USD, "2019-01-02").toJson.compactPrint))
// wrong fields order
// val filter = Document(s"""{"_id": ${ExchangeRateId(USD, "2019-01-02").toJson} }""")
// FILTER: {"_id": {"date": {"$date": 1546387200000}, "base": "USD"}}
// output:
BSON: List(Document((_id,{"date": {"$date": 1546387200000}, "base": "USD"}), (rates,{"PLN": 3.7671665351,
"CAD": 1.3442076646, "GBP": 0.7891607472, "JPY": 108.0417434009, "USD": 1.0, "EUR": 0.8769622029}),
(updated,BsonDateTime{value=1546387200000})))
JSON: List({"_id": {"date": {"$date": 1546387200000}, "base": "USD"}, "rates": {"PLN": 3.7671665351,
"CAD": 1.3442076646, "GBP": 0.7891607472, "JPY": 108.0417434009, "USD": 1.0, "EUR": 0.8769622029},
"updated": {"$date": 1546387200000}})
OBJ: List(ExchangeRate(ExchangeRateId(USD,2019-01-02T00:00Z),Map(USD -> 1.0, EUR -> 0.8769622029,
GBP -> 0.7891607472, CAD -> 1.3442076646, PLN -> 3.7671665351, JPY -> 108.0417434009),2019-01-02T00:00Z))
72. MongoDB Query on Nested Field
Dot notation is used to specify a query condition on fields in an embedded/nested
document.
https://docs.mongodb.com/manual/tutorial/query-embedded-documents/#query-on-nested-field
To specify or access a field of an embedded document with dot notation, concatenate the
embedded document name with the dot (.) and the field name, and enclose in quotes
https://docs.mongodb.com/manual/core/document/#embedded-documents
73. JSON fields order and related MongoDB filter issue
BSON document can’t be created from basic types
val filter = Filters.and(
Filters.eq("_id.base", Document(USD.toJson.compactPrint)),
org.bson.BsonInvalidOperationException: readStartDocument can only be called
when CurrentBSONType is DOCUMENT, not when CurrentBSONType is STRING.
Filters.eq("_id.date", Document(ZonedDateTime("2019-01-03").toJson.compactPrint))
org.bson.BsonInvalidOperationException: readStartDocument can only be called
when CurrentBSONType is DOCUMENT, not when CurrentBSONType is DATE_TIME.
)
74. JSON fields order and related MongoDB filter issue
Plain ZonedDateTime JSON can’t be used in Filter
val filter = Filters.and(
Filters.eq("_id.base", USD.toJson.compactPrint),
Filters.eq("_id.date", ZonedDateTime("2019-01-03").toJson.compactPrint)
)
// will return nothing because filter is incorrect - date is string, but
should be an object
// FILTER: {"_id.base": ""USD"", "_id.date": "{"$date":1546473600000}"}
79. MongoDB query on Nested fields limitations
upsert:true and dotted _id in insert operation
// ENTITY DOESN’T EXIST
val filter = "_id" $eq ExchangeRateId(USD, "2039-01-03")
val replacement = // ...
val options = FindOneAndReplaceOptions().upsert(true)
collection.findOneAndReplace(filter, replacement, options).toFuture()
com.mongodb.MongoCommandException: Command failed with error 111
(NotExactValueField): 'field at '_id' must be exactly specified, field at
sub-path '_id.base' found'.
The full response is {"ok": 0.0, "errmsg": "field at '_id' must be exactly
specified, field at sub-path '_id.base' found", "code": 111, "codeName":
"NotExactValueField"}
80. MongoDB query on Nested fields limitations
upsert:true and dotted _id
When users execute an update() with upsert: true and the query matches no existing
document, MongoDB will refuse to insert a new document if the query specifies conditions
on the _id field using dot notation.
This restriction ensures that the order of fields embedded in the _id document is well-defined
and not bound to the order specified in the query
If users attempted to insert a document in such way, MongoDB will raise an error.
https://docs.mongodb.com/manual/reference/method/db.collection.update/#upsert-true-with-a-dotted-id-query
81. Optional fields
// MODEL
case class GeoKey(country: String, state: Option[String] = None, city: Option[String] = None)
case class GeoRecord(key: GeoKey, name: String, population: Int)
// JSON
trait GeoModelJsonProtocol extends GreenLeafJsonProtocol {
implicit val GeoKeyFormat: RootJsonFormat[GeoKey] = jsonFormat3(GeoKey)
implicit val GeoRecordFormat: RootJsonFormat[GeoRecord] = jsonFormat3(GeoRecord)
}
object GeoModelJsonProtocol extends GeoModelJsonProtocol
// BSON
object GeoModelBsonProtocol extends GeoModelJsonProtocol with GreenLeafBsonProtocol {
override implicit val GeoRecordFormat: RootJsonFormat[GeoRecord] =
jsonFormat(GeoRecord, "_id", "name", "population")
}
82. Incorrect use of query with optional fields
import GeoModelBsonProtocol._
import GreenLeafMongoDsl._
val filter: Document = "_id" $eq GeoKey("6252001")
// filter will select all records with _id.country = USA includes states and cities:
// but this is look strange because we used pretty explicit filter by primary key
// FILTER: {"_id.country": {"$eq": "6252001"}}
// output:
GeoRecord(GeoKey(6252001,None,None),United States of America,310232863)
GeoRecord(GeoKey(6252001,Some(5128638),None),New York,19274244)
GeoRecord(GeoKey(6252001,Some(5128638),Some(5128581)),New York City,8175133)
GeoRecord(GeoKey(6252001,Some(5128638),Some(5133273)),Queens,2272771)
GeoRecord(GeoKey(6252001,Some(5128638),Some(5110302)),Brooklyn,2300664)
GeoRecord(GeoKey(6252001,Some(5101760),None),New Jersey,8751436)
GeoRecord(GeoKey(6252001,Some(5101760),Some(5099836)),Jersey City,264290)
GeoRecord(GeoKey(6252001,Some(5101760),Some(5099133)),Hoboken,53635)
GeoRecord(GeoKey(6252001,Some(5332921),None),California,37691912)
GeoRecord(GeoKey(6252001,Some(5332921),Some(5391959)),San Francisco,864816)
...
83. Optional fields, spray-json and MongoDB queries
Usually, optional members that are undefined (None) are not rendered at all.
The NullOptions trait supplies an alternative rendering mode for optional case class
members.
By mixing this trait into custom JsonProtocol users can enforce the rendering of undefined
members as null.
https://github.com/spray/spray-json#nulloptions
The { item : null } query matches documents that either contain item field with value null
or do not contain this field.
https://docs.mongodb.com/manual/tutorial/query-for-null-fields/
84. Verify BSON protocol with NullOptions
trait GreenLeafBsonProtocol extends GreenLeafJsonProtocol with NullOptions { // ...
// select country by primary key
val filter: Document = "_id" $eq GeoKey(country = "6252001")
// {"_id.city": {"$eq": null}, "_id.country": {"$eq": "6252001"}, "_id.state": {"$eq": null}}
// output:
GeoRecord(GeoKey(6252001,None,None),United States of America,310232863)
// select state by primary key
val filter: Document = "_id" $eq GeoKey(country = "6252001", state = "5128638")
// {"_id.city": {"$eq": null}, "_id.country": {"$eq": "6252001"}, "_id.state": {"$eq": "5128638"}}
// output:
GeoRecord(GeoKey(6252001,Some(5128638),None),New York,19274244)
// etc.
87. GreenLeafMongoDao
GreenLeafMongoDao extends GreenLeafMongoDsl and provides simple DSL to
transform Mongo's Observable[Document] instances to Future[Seq[T]],
Future[Option[T]] and Future[T].
This trait also provides many useful generic methods such as insert, getById, findById,
updateById and replaceById.
GreenLeafMongoDao uses _id field name for primary key by default, but it is possible to
override it to something different like “id” or “key” which will be reflected in all
predefined queries.
Methods such as insert or replace make preprocessing of the JSON to skip fields with
nullable values.
88. Example of GreenLeafMongoDao usage
// MODEL
case class Building(id: Long, name: String, height: Int, floors: Int, year: Int, address: String)
// JSON
trait BuildingModelJsonProtocol extends GreenLeafJsonProtocol {
implicit lazy val BuildingFormat: RootJsonFormat[Building] = jsonFormat6(Building)
}
object BuildingModelJsonProtocol extends BuildingModelJsonProtocol
// BSON
trait BuildingModelBsonProtocol extends BuildingModelJsonProtocol with GreenLeafBsonProtocol {
override implicit lazy val BuildingFormat: RootJsonFormat[Building] =
jsonFormat(Building, "_id", "name", "height", "floors", "year", "address")
}
object BuildingModelBsonProtocol extends BuildingModelBsonProtocol with DaoBsonProtocol[Long, Building] {
override implicit val idFormat: JsonFormat[Long] = LongJsonFormat
override implicit val entityFormat: JsonFormat[Building] = BuildingFormat
}
90. Conclusion:
1) GreenLeafJsonProtocol provides all JSON formats from Spray’s DefaultJsonProtocol.
JSON formats for additional types such as Zoned Date Time and Scala Enumeration also
defined. All these JSON formats can be overridden.
2) GreenLeafBsonProtocol extends GreenLeafJsonProtocol and overrides JSON formats for
some types as required for BSON serialization.
3) GreenLeafMongoDsl provides DSL that allows to write queries with a syntax that is more
close to real queries in MongoDB.
4) GreenLeafMongoDao extends GreenLeafMongoDsl, defines DSL for mongo’s observables
and provides typical methods such as insert, findById, getById, updateById, etc.