Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Taking advantage of the built-in developer tools – Couchbase Connect 2016

394 views

Published on

Couchbase offers various developer tools that ship with each version of Couchbase Server. These tools allow developers to quickly set up a Couchbase cluster, load custom data sets, and construct N1QL queries for their applications. This session covers how couchbase-cli, cbimport, Query Workbench, and cbq can be used to help build a cluster, analyze your data, and quickly build your application.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Taking advantage of the built-in developer tools – Couchbase Connect 2016

  1. 1. ©2016 Couchbase Inc. 1 The Couchbase Connect16 mobile app Take our in-app survey!
  2. 2. ©2016 Couchbase Inc. Taking Advantage of Built-In DeveloperTools Examining 2012 Presidential Campaign Finance Data with Couchbase 2
  3. 3. ©2016 Couchbase Inc.©2016 Couchbase Inc. Roadmap • Importing Data with cbimport • Exploring Data with QueryWorkbench • Power query features with Couchbase Query Shell (cbq) 3
  4. 4. ©2016 Couchbase Inc. 4 Importing Data Mike Wiederhold, Senior Software Engineer, Couchbase
  5. 5. ©2016 Couchbase Inc.©2016 Couchbase Inc. What is cbimport? • Allows importing of JSON and CSV/TSV data • Ability to read data locally or from an HTTP URL • Provides key generation capabilities based on the data being imported 5
  6. 6. ©2016 Couchbase Inc.©2016 Couchbase Inc. Importing JSON data • UnlikeCSV data JSON datasets can come in a variety of formats • Some datasets have a JSON object on each line • Some have a list of JSON objects • Some are Couchbase specific formats 6 Object Per Line: { “field1”: “value1”} { “field2”: “value2”} { “field3”: “value3”} { “field4”: “value4”} { “field5”: “value5”} { “field6”: “value6”} List of Objects: [ { “field1”: “value1”}, { “field2”: “value2”}, { “field3”: “value3”}, { “field4”: “value4”}, { “field5”: “value5”}, { “field6”: “value6”}, ] Folder containing objects: /root doc1.json doc2.json doc3.json doc4.json doc5.json doc6.json
  7. 7. ©2016 Couchbase Inc.©2016 Couchbase Inc. Importing CSV/TSV Data • Allows importing of JSON and CSV/TSV data • Separator is configurable so custom separated files also work 7 Comma Separated Files field1,field2,field3 value1,value2,value3 value1,value2,value3 value1,value2,value3 value1,value2,value3 value1,value2,value3 Tab Separated Files field1 field2 field3 value1 value2 value3 value1 value2 value3 value1 value2 value3 value1 value2 value3 value1 value2 value3
  8. 8. ©2016 Couchbase Inc.©2016 Couchbase Inc. Using Key Generators • Ability to generate keys based on the value of fields as well as custom generators 8 Given the document: { "name": "alice", "age": 40 } Given the key generator: key::%name%::#MONO_INCR# Result Key: key::alice::1
  9. 9. ©2016 Couchbase Inc. 9 Query Workbench Eben Haber, Senior Software Engineer & User Advocate, Couchase
  10. 10. ©2016 Couchbase Inc.©2016 Couchbase Inc. QueryWorkbench – new UI feature in 4.5 • cbq limitations: • Hard to edit long queries • Output hard to read • Little info on buckets, document structure • Query Workbench: • Multi-line query editor with syntax coloring, auto- completion • Query output shown as JSON, table, or tree • List of buckets shown, with bucket schema inferencing
  11. 11. ©2016 Couchbase Inc.©2016 Couchbase Inc. QueryWorkbench Enter N1QL queries here View previous queries/results See results as JSON, table, or tree View buckets, schemas here
  12. 12. ©2016 Couchbase Inc.©2016 Couchbase Inc. Schemas? • Couchbase does not enforce schemas, but documents usually share common structure(s) • The Query Workbench includes schema inferencing, which uses a random sample of documents to identify common fields/schemas • E.g., for beer-sample, it finds two distinct ‘flavors’ of schema • When you get fields in the Bucket Analysis panel, those fields become available for auto-complete • Hover the mouse over a field name to see sample values for that field as a tool tip
  13. 13. ©2016 Couchbase Inc.©2016 Couchbase Inc. Autocompletion • The query editor can autocomplete N1QL keywords, bucket names, and even field names when you click ‘tab’ in the middle of a word.
  14. 14. ©2016 Couchbase Inc.©2016 Couchbase Inc. Data Display • Reading/comparing multiple JSON docs can be hard, so data can also be shown as table, tree: • N.B. hovering over cell in table/tree view will show path to that value
  15. 15. ©2016 Couchbase Inc.©2016 Couchbase Inc. Other Features • File I/O: Can save queries or query results to a file • Query History: Past queries are stored in the browser’s local storage, so queries you have run remain visible across sessions (but not query results) • Can run schema inference as a query to get more details, e.g., • infer `beer-sample` with {“sample_size” : 2000} • This runs the inferencer on a sample of 2000 documents, and will show how many documents are in which flavor, how frequently each field appears, shows multiple sample values for each field, etc. • Schema inference can also highlight dirty data – fields with multiple types, etc.
  16. 16. ©2016 Couchbase Inc.©2016 Couchbase Inc. QueryWorkbench Limitations • Workbench times out for queries taking > 5 minutes • use cbq in for very long running queries) • Displaying large query results can be slow in the web browser • JSON view about 10x faster than table, tree views • If rendering expected to take > 5 seconds, “are you sure” link appears • Tip: even if results not shown, you can still save them to a file
  17. 17. ©2016 Couchbase Inc. DEMO
  18. 18. ©2016 Couchbase Inc. CBQ - Command line shell for N1QL Isha Kandaswamy, Software Developer, Couchbase
  19. 19. ©2016 Couchbase Inc.©2016 Couchbase Inc. CBQ- New CLI for N1QL in 4.5 • What is cbq ? • Command line shell that provides an interface to the Query engine allowing you to run queries interactively. Additionally one can also issue scripting commands. • Shell commands start with a ' ’ • When cbq starts up, it is given a set of command line options. If no options are present then it assumes default values for expected options. • It is important to note that all shell commands are case insensitive.Command line options on the other hand are case sensitive. • The set of command line options and shell commands can be classified into the following Connection Management Security Management Session Management Query Management Shell/ Server Management Scripting support
  20. 20. ©2016 Couchbase Inc. 20©2016 Couchbase Inc. CBQ commands and options • Connection Management • Manage connections between cbq and the server. • Connect to the cluster : Auto discover query services and round robin amongst them. • Connect to a specific query service directly. • Don’t connect to any endpoint • Switch between connections • Security Management • Provide mechanism for authentication. • Role based access : Provide username and password when bringing up the shell. • Protected bucket access : Pass single/ multi-bucket credentials on a per query basis. (using query rest api parameter creds) • Secure connections
  21. 21. ©2016 Couchbase Inc. 21©2016 Couchbase Inc. CBQ commands and options • Session Management • Set user preferences and manage different shell sessions. • Echo statements. • Scroll through history, reverse search, partial query auto lookup. • Set/ Unset session and server level parameter settings. • Alias • Query Management • Manage interaction of the shell with the query service • Cancel/ Abort running queries. • Ability to set positional and Named parameters. • Prepared and Named Prepared statements. • Shell comments (# and --)
  22. 22. ©2016 Couchbase Inc. 22©2016 Couchbase Inc. CBQ commands and options • Shell/ Server Management • Convey information about the shell/cluster endpoint • Display versions for the shell and server and display copyright information. • Shortcut-keys support • Edit commands saved in the history • Scripting support • Interpret and run single/multiple shell and N1QL commands in cbq – Multi-line input and Batch mode • Read commands from a file and execute them • Redirect output of queries to a file. • Piping commands into cbq • Exit on error • Display specific parts of the results of a query
  23. 23. ©2016 Couchbase Inc. 23©2016 Couchbase Inc. • In order to handle Query Rest API/Named parameters with the query service and allow for better scripting, cbq allows 3 kinds of parameters. • Every parameter is of the type <prefix><name>. The prefix is used to distinguish between each type. • The values for these parameters can be set using the SET command and unset using the UNSET command. SET –args [1, "Step 1”] UNSET –args • All settings via the Rest API are settable via the shell in the form of Query Parameters prefixed by - . Prefix Parameter type Example - Query parameters SET –creds beer-sample:pass; No prefix $ Session parameters Pre-defined parameters User-defined parameters SET histfile .new_cbqhistory; SET $temp 5; -$ Named parameters SET -$rate 9.5; CBQ – Parameters
  24. 24. ©2016 Couchbase Inc. 24©2016 Couchbase Inc. Shell Commands : Comman d Args Description Example SET Parameter Value Resets the topmost value of the stack for the input parameter with the input value. cbq> SET -args ["1", "Ok"] ; SET none Display the values of all the parameters for the current session. cbq>SET; Query Parameters : Parameter name : args Value : [["1","Ok"]] Named Parameters : User Defined Session Parameters : Predefined Session Parameters : Parameter name : histfile Value : [".cbq_history"] UNSET Parameter Deletes/resets the stack for the input parameter. Pops the while stack for input value and then deletes it. cbq> UNSET -args; CBQ – Parameter Configuration
  25. 25. ©2016 Couchbase Inc. 25©2016 Couchbase Inc. Named and Positional Parameters • In order to define these, we use the SET command. • For each named parameter, use the prefix -$. SET -$date "1-1-2014"; • For positional parameters we set the -args query parameter. SET -args [9.5, "1-1-2014"]; Prepared Statements – How to • Set the named and positional parameters that are present in the prepared statement. • PREPARE <N1QL Query>; (Prepared statement) OR • PREPARE <name> from <N1QL query>; (Named Prepare statement) • EXECUTE <name>; CBQ – Query Management – NAMED and POSITIONAL Query Parameters
  26. 26. ©2016 Couchbase Inc. 26©2016 Couchbase Inc. #Create a prepared statement to see which advertising companies made a lot of money prepare advAmount FROM SELECT cand_nm,sum(TONUMBER(disb_amt)) as total FROM `expenditures` WHERE cand_nm = $candidate AND recipient_nm = $1 AND disb_amt >= $2 GROUP BY cand_nm; SET -$candidate "Romney, Mitt"; # Set named parameter candidate for candidate name SET -args ["AMERICAN RAMBLER PRODUCTIONS LLC" ,500]; #Set positional parameters execute advAmount; # Execute prepared statement advAmount SET -$candidate "Obama, Barack" # Change candidate to Obama, Barack SET -args ["GMMB INC.", 500]; execute advAmount; # Run prepared statement airinfo with new candidate name CBQ – Query Management – NAMED and POSITIONAL Query Parameters
  27. 27. ©2016 Couchbase Inc. DEMO
  28. 28. ©2016 Couchbase Inc. 28 Isha Kandaswamy Software Engineer isha@couchbase.com
  29. 29. ©2016 Couchbase Inc. ThankYou! 29
  30. 30. ©2016 Couchbase Inc. 30 The Couchbase Connect16 mobile app Take our in-app survey!
  31. 31. ©2016 Couchbase Inc. 31 Share your opinion on Couchbase 1. Go here: http://gtnr.it/2eRxYWn 2. Create a profile 3. Provide feedback (~15 minutes)

×