SlideShare a Scribd company logo
Kyle Banerjee
https://bit.ly/alma_api
Getting Started with the
Alma API
● Perform mass modifications of user, item,
bib, holdings, or any other type of record
● Manage collections, portfolios, digital
assets, vendors, invoices, licenses
● Perform extractions and analyses that are
impossible in Analytics
● And more!
What can the Alma API do?
All an API does is send and receive
plain text* messages
APIs are MUCH simpler than they look
* You can also send and receive binary things like
images, but the process is the same as for text
...between using an API and a Web form
from 1995 is that something you wrote
rather than a browser interprets the text
the server sends
The only real difference...
● Focus on the main takeaways
● Ignore syntax. Pay attention to what is
being done, but not how -- you can
figure that out later
● Ask for help
If you’re new to APIs
● Syntax is hard because it’s super picky
and a little text does lot of work
● You can always look up syntax. Don’t try to
memorize everything
● If you don’t get it right the first (or tenth)
time, you can keep tweaking it until you get
it right
If you feel intimidated, remember...
● A way to interact with applications
● ReST (Representational State Transfer) API
is a fancy way of saying to interact with an
application the way a mid 1990’s Web
browser would
API (Application Programming Interface)
Anatomy of retrieving a Web page
Ordinary terminal connection
Request main page at root of site
Server sends back web page
The browser converts text to what you see
Real Alma API call
Set up encrypted connection
Request holdings record per API
specification
Holdings record returned
To use the Alma API, you typically need to
1. Read a file containing information allowing
you to build a command
2. Send that information to the API
3. Read and modify information you receive
from the API
Now what?
The command line
● Often by far the easiest way
● More flexible than a GUI
● Supported by all major platforms
● Process files of any size
● Combine the power of individual
programs -- they do all the heavy
lifting for you!
Handy tools for the Alma API
Linux or MacOS command line
curl -- Communicate with API service
xmlstarlet -- Extract data from XML or
edit XML
sed -- Clean data
cut -- Break data into fields
echo -- Print data
Learning the command line
● Learn one command at a time. Don’t worry
about what you don’t need.
● Don’t worry if you get things wrong -- one
advantage of the command line is you can try
things many times until you get it right
● Google solutions for specific problems --
there are many online examples
● Try, but give up fast. Ask linux geeks for help
Getting started with the command line
● MacOS (use Terminal)
○ Install Homebrew
xcode-select --install
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew install coreutils
● Windows 10
○ Enable linux subsystem and go to bash terminal
Scripting is the command line!
● Simple text files that allow you to combine
utilities and programs written in any
language
● No programming experience necessary
● Great for automating processes
● For unfamiliar problems, ask for help
Command Line Tools:
curl, xmlstarlet, cut, and sed
curl
● A tool to transfer data from or to a server
● Works with many protocols, can deal with
authentication
● Especially useful for the Alma API
curl examples
curl -s -H "Authorization: apikey [your API key]" 
-H "Accept: application/xml" 
GET https://api-na.hosted.exlibrisgroup.com/[api call]
curl -s -H "Authorization: apikey [your API key]" 
-d "$DATA"  # data stored in variable
-H "Content-Type: application/xml" 
PUT https://api-na.hosted.exlibrisgroup.com/[api call]
curl -s -H "Authorization: apikey [your API key]" 
-d "@datafile"  # data stored in file
-H "Content-Type: application/xml" 
PUT https://api-na.hosted.exlibrisgroup.com/[api call]
curl examples
Reading records
curl -s -H "Authorization: apikey [your API key]" 
-H "Accept: application/xml" 
GET https://api-na.hosted.exlibrisgroup.com/[api call]
Updating records
curl -s -H "Authorization: apikey [your API key]" 
-H "Content-Type: application/xml" -d "$DATA" 
PUT https://api-na.hosted.exlibrisgroup.com/[api call]
Creating records
curl -s -H "Authorization: apikey [your API key]" 
-H "Content-Type: application/xml" -d "@datafile" 
POST https://api-na.hosted.exlibrisgroup.com/[api call]
Xmlstarlet
● A tool to parse, selectively display, update,
and create XML data
● Syntax can be challenging - little
parameters have a lot of power!
● Examples are the best way to learn - we’ll
look at some next.
Linux (and Windows Linux subsystem)
sudo apt-get install xmlstarlet
Mac
brew install xmlstarlet
Installing Xmlstarlet
-t creates a template
-m matches an XPATH expression
-v selects a value
-o outputs literal text
-c + ‘count()’ gives a count
Protip: ALWAYS strip out namespaces using sed
-- xmlstarlet is picky about them, so they’re a
pain. You can skip this with the Alma API
because it doesn’t send a namespace
xmlstarlet is easier than it looks
Extract a value and store in a variable
holdingid=$(echo $record |xmlstarlet sel -t -m
"item/holding_data/holding_id" -v . )
Update a node within an XML document
holdings=$(echo $holdings | xmlstarlet ed -u
'/holding/record/datafield[@tag="852"]/subfield[@code="c"]
' -v 'oldstorbks')
Xmlstarlet examples
sed
● User regular expressions to select,
parse, and transform lines
● Great for “fixing” data so that it can be
used with other programs
● Extremely powerful and works great
with very large (terabytes) files
● Strip everything except numbers from
barcode variable
barcode=$(sed 's/[^0-9]//g' <<< "$barcode")
● Strip all nonnumeric data from file1 with
results sent to file2
cat file1 | sed 's/[^0-9]//g' > file2
sed examples
Quick Regular Expression Guide
^ Match the start of the line
$ Match the end of the line
. Match any single character
* Match zero or more of the previous character
[A-D,G-J,0-5]* [A-D,G-J,0-5]* = match zero or more of ABCDGHIJ012345
[^A-C] Match any one character that is NOT A,B, or C
(dog)
Match the word "dog", including case, and remember that text
to be used later in the match or replacement
1
Insert the first remembered text as if it were typed here (2 for
second, 3 for 3rd, etc.)

Use to match special characters.  matches a backslash, *
matches an asterisk, $ matches dollar sign, etc.
cut -- Extract fields from delimited data
Extract second field in comma delimited line
variable
secondfield="$(cut -d',' -f2 <<<$line)"
Extract first through third fields in comma
delimited file and send output to file2
cat file1 | cut -d',' -f1-3 > file2
Read and process file
# Read file named “barcodes” one line at a time
cat barcodes | while read line
do
# Clean barcode
barcode=$(sed 's/[^0-9]//g' <<< "$line")
# print barcode
echo barcode
done
Putting it All Together:
A Practical Example
1. Create list of barcodes
2. Get mmsids and holdingsids from Alma
3. Retrieve holdings records based on
mmsids and holdingsids
4. Update location in holdings record
5. Replace holdings record
Example: Update locations based on barcode
cat barcodes| while read barcode # Read input file named “barcodes” one line at a time
do
barcode=$(sed 's/[^0-9]//g' <<< "$barcode") # Clean barcode
# Get record and extract record ids
recinfo=$(curl -s -X GET -L -H "Authorization: apikey $(cat apikey.txt)" "https://api-
na.hosted.exlibrisgroup.com/almaws/v1/items?item_barcode=${barcode}")
mmsid=$(echo $recinfo |xmlstarlet sel -t -m "item/bib_data/mms_id" -v . )
holdingid=$(echo $recinfo |xmlstarlet sel -t -m "item/holding_data/holding_id" -v . )
# Retrieve holdings record
url="https://api-na.hosted.exlibrisgroup.com/almaws/v1/bibs/${mmsid}/holdings/${holdingid}"
holdings=$(curl -s -X GET -L -H "Authorization: apikey $(cat apikey.txt)" "${url}")
# Edit location in holdings record
holdings=$(echo $holdings | xmlstarlet ed -u
'/holding/record/datafield[@tag="852"]/subfield[@code="c"]' -v 'oldstorbks')
# Replace holdings record
newloc=$(curl -s -H "Authorization: apikey $(cat apikey.txt)" -H "Content-Type: application/xml" -
X PUT --data "${holdings}" "${url}")
done
A lot of work in just a few lines!
Retrieve the record via API
recinfo=$(curl -s -X GET -L -H "Authorization: apikey $(cat apikey.txt)" "https://api-
na.hosted.exlibrisgroup.com/almaws/v1/items?item_barcode=33231000051658"))
echo $recinfo |xmlstarlet fo
Extract fields we want with xmlstarlet
Then extract holdings and MMSID to construct another
query to retrieve the holdings record
Fix the XML
holdings=$(echo $holdings | xmlstarlet ed -u
'/holding/record/datafield[@tag="852"]/subfield[@code="c"]' -v 'oldstorbks')
● Use pipes “|” to make the output of one
command the input of another
● Use command substitution $(command) to
put use the output of one command within
another
● Use backslashes to break a single
command up across multiple lines to make
it more readable
Handy things to know with scripts
Running your scripts
● Script first must be made executable.
This must only be only once
chmod 700 myscript
● To run, just get in the same directory
and precede filename with a period and
forward slash
./myscript
Alma API Documentation
https://developers.exlibrisgroup.com/alma/apis/
This presentation
https://bit.ly/alma_api
A few starter scripts
https://github.com/banerjek/alma
Resources
Thank you!

More Related Content

What's hot

Informetrics final
Informetrics finalInformetrics final
Informetrics finalAamir Abbas
 
Evaluating your reference service
Evaluating your reference service Evaluating your reference service
Evaluating your reference service
AnnaTatherine
 
How ict used in libraries
How ict used in librariesHow ict used in libraries
How ict used in libraries
janjangammod
 
LIS Education in Pre-independent Era
LIS Education in Pre-independent EraLIS Education in Pre-independent Era
LIS Education in Pre-independent Era
Wahid Ullah
 
4 Clearing houses, Translation centres and Reprographic centres.pptx
4 Clearing houses, Translation centres and Reprographic centres.pptx4 Clearing houses, Translation centres and Reprographic centres.pptx
4 Clearing houses, Translation centres and Reprographic centres.pptx
JosephIThomas
 
Metadata: a library perspective
Metadata: a library perspectiveMetadata: a library perspective
Metadata: a library perspectivejody perkins
 
The IRRI Library 2010
The IRRI Library 2010The IRRI Library 2010
The IRRI Library 2010
mmramos
 
BIBFRAME
BIBFRAMEBIBFRAME
BIBFRAME
Thomas Meehan
 
The tools of our trade: AACR2/RDA and MARC
The tools of our trade: AACR2/RDA and MARCThe tools of our trade: AACR2/RDA and MARC
The tools of our trade: AACR2/RDA and MARC
Ann Chapman
 
Research in Librarianship
Research in LibrarianshipResearch in Librarianship
Research in Librarianship
Fe Angela Verzosa
 
Statistics for Librarians: How to Use and Evaluate Statistical Evidence
Statistics for Librarians: How to Use and Evaluate Statistical EvidenceStatistics for Librarians: How to Use and Evaluate Statistical Evidence
Statistics for Librarians: How to Use and Evaluate Statistical Evidence
John McDonald
 
Library Makeover: Retooling & Re-engineering of Library Services
Library Makeover:  Retooling & Re-engineering of Library ServicesLibrary Makeover:  Retooling & Re-engineering of Library Services
Library Makeover: Retooling & Re-engineering of Library Services
Fe Angela Verzosa
 
Digital Libraries: the process, initiatives and developmental issues in India...
Digital Libraries: the process, initiatives and developmental issues in India...Digital Libraries: the process, initiatives and developmental issues in India...
Digital Libraries: the process, initiatives and developmental issues in India...Sudesh Sood
 
Introduction to koha
Introduction to kohaIntroduction to koha
Introduction to koha
arslanone
 
ORGANIZATION OF INFORMATION RESOURCES
ORGANIZATION OF INFORMATION RESOURCES ORGANIZATION OF INFORMATION RESOURCES
ORGANIZATION OF INFORMATION RESOURCES
LILIBETHBAJOYO
 
Subjects Plus: Information Management Tool - A Case Study, with Special Refer...
Subjects Plus: Information Management Tool - A Case Study, with Special Refer...Subjects Plus: Information Management Tool - A Case Study, with Special Refer...
Subjects Plus: Information Management Tool - A Case Study, with Special Refer...
Indian Institute of Management Ahmedabad
 
Digital libraries: successfully designing developing and implementing your d...
Digital libraries:  successfully designing developing and implementing your d...Digital libraries:  successfully designing developing and implementing your d...
Digital libraries: successfully designing developing and implementing your d...
Beatrice Amollo
 
PAARL Standards for Academic Libraries 2010 (Final Draft Proposal)
PAARL Standards for Academic Libraries 2010 (Final Draft Proposal)PAARL Standards for Academic Libraries 2010 (Final Draft Proposal)
PAARL Standards for Academic Libraries 2010 (Final Draft Proposal)
Fe Angela Verzosa
 
An introduction to IFLA
An introduction to IFLAAn introduction to IFLA
An introduction to IFLA
Jesus Lau
 

What's hot (20)

Informetrics final
Informetrics finalInformetrics final
Informetrics final
 
Evaluating your reference service
Evaluating your reference service Evaluating your reference service
Evaluating your reference service
 
How ict used in libraries
How ict used in librariesHow ict used in libraries
How ict used in libraries
 
LIS Education in Pre-independent Era
LIS Education in Pre-independent EraLIS Education in Pre-independent Era
LIS Education in Pre-independent Era
 
4 Clearing houses, Translation centres and Reprographic centres.pptx
4 Clearing houses, Translation centres and Reprographic centres.pptx4 Clearing houses, Translation centres and Reprographic centres.pptx
4 Clearing houses, Translation centres and Reprographic centres.pptx
 
Metadata: a library perspective
Metadata: a library perspectiveMetadata: a library perspective
Metadata: a library perspective
 
The IRRI Library 2010
The IRRI Library 2010The IRRI Library 2010
The IRRI Library 2010
 
BIBFRAME
BIBFRAMEBIBFRAME
BIBFRAME
 
The tools of our trade: AACR2/RDA and MARC
The tools of our trade: AACR2/RDA and MARCThe tools of our trade: AACR2/RDA and MARC
The tools of our trade: AACR2/RDA and MARC
 
Research in Librarianship
Research in LibrarianshipResearch in Librarianship
Research in Librarianship
 
Statistics for Librarians: How to Use and Evaluate Statistical Evidence
Statistics for Librarians: How to Use and Evaluate Statistical EvidenceStatistics for Librarians: How to Use and Evaluate Statistical Evidence
Statistics for Librarians: How to Use and Evaluate Statistical Evidence
 
Library Makeover: Retooling & Re-engineering of Library Services
Library Makeover:  Retooling & Re-engineering of Library ServicesLibrary Makeover:  Retooling & Re-engineering of Library Services
Library Makeover: Retooling & Re-engineering of Library Services
 
Digital Libraries: the process, initiatives and developmental issues in India...
Digital Libraries: the process, initiatives and developmental issues in India...Digital Libraries: the process, initiatives and developmental issues in India...
Digital Libraries: the process, initiatives and developmental issues in India...
 
Introduction to koha
Introduction to kohaIntroduction to koha
Introduction to koha
 
ORGANIZATION OF INFORMATION RESOURCES
ORGANIZATION OF INFORMATION RESOURCES ORGANIZATION OF INFORMATION RESOURCES
ORGANIZATION OF INFORMATION RESOURCES
 
Subjects Plus: Information Management Tool - A Case Study, with Special Refer...
Subjects Plus: Information Management Tool - A Case Study, with Special Refer...Subjects Plus: Information Management Tool - A Case Study, with Special Refer...
Subjects Plus: Information Management Tool - A Case Study, with Special Refer...
 
Digital libraries: successfully designing developing and implementing your d...
Digital libraries:  successfully designing developing and implementing your d...Digital libraries:  successfully designing developing and implementing your d...
Digital libraries: successfully designing developing and implementing your d...
 
PAARL Standards for Academic Libraries 2010 (Final Draft Proposal)
PAARL Standards for Academic Libraries 2010 (Final Draft Proposal)PAARL Standards for Academic Libraries 2010 (Final Draft Proposal)
PAARL Standards for Academic Libraries 2010 (Final Draft Proposal)
 
Million Book Project
Million Book ProjectMillion Book Project
Million Book Project
 
An introduction to IFLA
An introduction to IFLAAn introduction to IFLA
An introduction to IFLA
 

Similar to Getting Started with the Alma API

Server Logs: After Excel Fails
Server Logs: After Excel FailsServer Logs: After Excel Fails
Server Logs: After Excel Fails
Oliver Mason
 
REST in pieces
REST in piecesREST in pieces
REST in pieces
sparkfabrik
 
[drupalday2017] - REST in pieces
[drupalday2017] - REST in pieces[drupalday2017] - REST in pieces
[drupalday2017] - REST in pieces
DrupalDay
 
Distributed, Incremental Dataflow Processing on AWS with GRAIL's Reflow (CMP3...
Distributed, Incremental Dataflow Processing on AWS with GRAIL's Reflow (CMP3...Distributed, Incremental Dataflow Processing on AWS with GRAIL's Reflow (CMP3...
Distributed, Incremental Dataflow Processing on AWS with GRAIL's Reflow (CMP3...
Amazon Web Services
 
AWS Hadoop and PIG and overview
AWS Hadoop and PIG and overviewAWS Hadoop and PIG and overview
AWS Hadoop and PIG and overview
Dan Morrill
 
Amazon EMR Masterclass
Amazon EMR MasterclassAmazon EMR Masterclass
Amazon EMR Masterclass
Ian Massingham
 
Amazon EMR Masterclass
Amazon EMR MasterclassAmazon EMR Masterclass
Amazon EMR Masterclass
Amazon Web Services
 
Creating a modern web application using Symfony API Platform Atlanta
Creating a modern web application using  Symfony API Platform AtlantaCreating a modern web application using  Symfony API Platform Atlanta
Creating a modern web application using Symfony API Platform Atlanta
Jesus Manuel Olivas
 
Catalyst - refactor large apps with it and have fun!
Catalyst - refactor large apps with it and have fun!Catalyst - refactor large apps with it and have fun!
Catalyst - refactor large apps with it and have fun!
mold
 
mastering libcurl part 1
mastering libcurl part 1mastering libcurl part 1
mastering libcurl part 1
Daniel Stenberg
 
Embulk, an open-source plugin-based parallel bulk data loader
Embulk, an open-source plugin-based parallel bulk data loaderEmbulk, an open-source plugin-based parallel bulk data loader
Embulk, an open-source plugin-based parallel bulk data loader
Sadayuki Furuhashi
 
AWS November Webinar Series - Advanced Analytics with Amazon Redshift and the...
AWS November Webinar Series - Advanced Analytics with Amazon Redshift and the...AWS November Webinar Series - Advanced Analytics with Amazon Redshift and the...
AWS November Webinar Series - Advanced Analytics with Amazon Redshift and the...
Amazon Web Services
 
Web data from R
Web data from RWeb data from R
Web data from Rschamber
 
NYPHP March 2009 Presentation
NYPHP March 2009 PresentationNYPHP March 2009 Presentation
NYPHP March 2009 Presentation
brian_dailey
 
Practical catalyst
Practical catalystPractical catalyst
Practical catalyst
dwm042
 
Killing the Angle Bracket
Killing the Angle BracketKilling the Angle Bracket
Killing the Angle Bracket
jnewmanux
 
Useful Python Libraries for Network Engineers - PyOhio 2018
Useful Python Libraries for Network Engineers - PyOhio 2018Useful Python Libraries for Network Engineers - PyOhio 2018
Useful Python Libraries for Network Engineers - PyOhio 2018
Hank Preston
 
OpenERP Technical Memento V0.7.3
OpenERP Technical Memento V0.7.3OpenERP Technical Memento V0.7.3
OpenERP Technical Memento V0.7.3Borni DHIFI
 
Big Data - Lab A1 (SC 11 Tutorial)
Big Data - Lab A1 (SC 11 Tutorial)Big Data - Lab A1 (SC 11 Tutorial)
Big Data - Lab A1 (SC 11 Tutorial)Robert Grossman
 

Similar to Getting Started with the Alma API (20)

Server Logs: After Excel Fails
Server Logs: After Excel FailsServer Logs: After Excel Fails
Server Logs: After Excel Fails
 
REST in pieces
REST in piecesREST in pieces
REST in pieces
 
[drupalday2017] - REST in pieces
[drupalday2017] - REST in pieces[drupalday2017] - REST in pieces
[drupalday2017] - REST in pieces
 
Distributed, Incremental Dataflow Processing on AWS with GRAIL's Reflow (CMP3...
Distributed, Incremental Dataflow Processing on AWS with GRAIL's Reflow (CMP3...Distributed, Incremental Dataflow Processing on AWS with GRAIL's Reflow (CMP3...
Distributed, Incremental Dataflow Processing on AWS with GRAIL's Reflow (CMP3...
 
AWS Hadoop and PIG and overview
AWS Hadoop and PIG and overviewAWS Hadoop and PIG and overview
AWS Hadoop and PIG and overview
 
Amazon EMR Masterclass
Amazon EMR MasterclassAmazon EMR Masterclass
Amazon EMR Masterclass
 
Amazon EMR Masterclass
Amazon EMR MasterclassAmazon EMR Masterclass
Amazon EMR Masterclass
 
Creating a modern web application using Symfony API Platform Atlanta
Creating a modern web application using  Symfony API Platform AtlantaCreating a modern web application using  Symfony API Platform Atlanta
Creating a modern web application using Symfony API Platform Atlanta
 
Catalyst - refactor large apps with it and have fun!
Catalyst - refactor large apps with it and have fun!Catalyst - refactor large apps with it and have fun!
Catalyst - refactor large apps with it and have fun!
 
mastering libcurl part 1
mastering libcurl part 1mastering libcurl part 1
mastering libcurl part 1
 
Embulk, an open-source plugin-based parallel bulk data loader
Embulk, an open-source plugin-based parallel bulk data loaderEmbulk, an open-source plugin-based parallel bulk data loader
Embulk, an open-source plugin-based parallel bulk data loader
 
AWS November Webinar Series - Advanced Analytics with Amazon Redshift and the...
AWS November Webinar Series - Advanced Analytics with Amazon Redshift and the...AWS November Webinar Series - Advanced Analytics with Amazon Redshift and the...
AWS November Webinar Series - Advanced Analytics with Amazon Redshift and the...
 
Wikilims Road4
Wikilims Road4Wikilims Road4
Wikilims Road4
 
Web data from R
Web data from RWeb data from R
Web data from R
 
NYPHP March 2009 Presentation
NYPHP March 2009 PresentationNYPHP March 2009 Presentation
NYPHP March 2009 Presentation
 
Practical catalyst
Practical catalystPractical catalyst
Practical catalyst
 
Killing the Angle Bracket
Killing the Angle BracketKilling the Angle Bracket
Killing the Angle Bracket
 
Useful Python Libraries for Network Engineers - PyOhio 2018
Useful Python Libraries for Network Engineers - PyOhio 2018Useful Python Libraries for Network Engineers - PyOhio 2018
Useful Python Libraries for Network Engineers - PyOhio 2018
 
OpenERP Technical Memento V0.7.3
OpenERP Technical Memento V0.7.3OpenERP Technical Memento V0.7.3
OpenERP Technical Memento V0.7.3
 
Big Data - Lab A1 (SC 11 Tutorial)
Big Data - Lab A1 (SC 11 Tutorial)Big Data - Lab A1 (SC 11 Tutorial)
Big Data - Lab A1 (SC 11 Tutorial)
 

More from Kyle Banerjee

Demystifying RDF
Demystifying RDFDemystifying RDF
Demystifying RDF
Kyle Banerjee
 
Keep it Safe, Stupid, or an Intro to Digital Preservation
Keep it Safe, Stupid, or an Intro to Digital PreservationKeep it Safe, Stupid, or an Intro to Digital Preservation
Keep it Safe, Stupid, or an Intro to Digital Preservation
Kyle Banerjee
 
Web Scraping Basics
Web Scraping BasicsWeb Scraping Basics
Web Scraping Basics
Kyle Banerjee
 
Future Directions in Metadata
Future Directions in MetadataFuture Directions in Metadata
Future Directions in Metadata
Kyle Banerjee
 
Переход от отдельных библиотечных систем к объединенной системе Альма
Переход от отдельных библиотечных систем к объединенной системе АльмаПереход от отдельных библиотечных систем к объединенной системе Альма
Переход от отдельных библиотечных систем к объединенной системе Альма
Kyle Banerjee
 
Normalizing Data for Migrations
Normalizing Data for MigrationsNormalizing Data for Migrations
Normalizing Data for Migrations
Kyle Banerjee
 
Dropping ACID: Wrapping Your Mind Around NoSQL Databases
Dropping ACID: Wrapping Your Mind Around NoSQL DatabasesDropping ACID: Wrapping Your Mind Around NoSQL Databases
Dropping ACID: Wrapping Your Mind Around NoSQL Databases
Kyle Banerjee
 
Batch metadata assignment to archival photograph collections using facial rec...
Batch metadata assignment to archival photograph collections using facial rec...Batch metadata assignment to archival photograph collections using facial rec...
Batch metadata assignment to archival photograph collections using facial rec...
Kyle Banerjee
 
Intro to XML in libraries
Intro to XML in librariesIntro to XML in libraries
Intro to XML in libraries
Kyle Banerjee
 

More from Kyle Banerjee (9)

Demystifying RDF
Demystifying RDFDemystifying RDF
Demystifying RDF
 
Keep it Safe, Stupid, or an Intro to Digital Preservation
Keep it Safe, Stupid, or an Intro to Digital PreservationKeep it Safe, Stupid, or an Intro to Digital Preservation
Keep it Safe, Stupid, or an Intro to Digital Preservation
 
Web Scraping Basics
Web Scraping BasicsWeb Scraping Basics
Web Scraping Basics
 
Future Directions in Metadata
Future Directions in MetadataFuture Directions in Metadata
Future Directions in Metadata
 
Переход от отдельных библиотечных систем к объединенной системе Альма
Переход от отдельных библиотечных систем к объединенной системе АльмаПереход от отдельных библиотечных систем к объединенной системе Альма
Переход от отдельных библиотечных систем к объединенной системе Альма
 
Normalizing Data for Migrations
Normalizing Data for MigrationsNormalizing Data for Migrations
Normalizing Data for Migrations
 
Dropping ACID: Wrapping Your Mind Around NoSQL Databases
Dropping ACID: Wrapping Your Mind Around NoSQL DatabasesDropping ACID: Wrapping Your Mind Around NoSQL Databases
Dropping ACID: Wrapping Your Mind Around NoSQL Databases
 
Batch metadata assignment to archival photograph collections using facial rec...
Batch metadata assignment to archival photograph collections using facial rec...Batch metadata assignment to archival photograph collections using facial rec...
Batch metadata assignment to archival photograph collections using facial rec...
 
Intro to XML in libraries
Intro to XML in librariesIntro to XML in libraries
Intro to XML in libraries
 

Recently uploaded

UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
Free Complete Python - A step towards Data Science
Free Complete Python - A step towards Data ScienceFree Complete Python - A step towards Data Science
Free Complete Python - A step towards Data Science
RinaMondal9
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Nexer Digital
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
KatiaHIMEUR1
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
DianaGray10
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
Ana-Maria Mihalceanu
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Paige Cruz
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Albert Hoitingh
 
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Neo4j
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Quotidiano Piemontese
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Adtran
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Neo4j
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
DianaGray10
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
danishmna97
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
KAMESHS29
 

Recently uploaded (20)

UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
Free Complete Python - A step towards Data Science
Free Complete Python - A step towards Data ScienceFree Complete Python - A step towards Data Science
Free Complete Python - A step towards Data Science
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
 
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
 

Getting Started with the Alma API

  • 2. ● Perform mass modifications of user, item, bib, holdings, or any other type of record ● Manage collections, portfolios, digital assets, vendors, invoices, licenses ● Perform extractions and analyses that are impossible in Analytics ● And more! What can the Alma API do?
  • 3. All an API does is send and receive plain text* messages APIs are MUCH simpler than they look * You can also send and receive binary things like images, but the process is the same as for text
  • 4. ...between using an API and a Web form from 1995 is that something you wrote rather than a browser interprets the text the server sends The only real difference...
  • 5. ● Focus on the main takeaways ● Ignore syntax. Pay attention to what is being done, but not how -- you can figure that out later ● Ask for help If you’re new to APIs
  • 6. ● Syntax is hard because it’s super picky and a little text does lot of work ● You can always look up syntax. Don’t try to memorize everything ● If you don’t get it right the first (or tenth) time, you can keep tweaking it until you get it right If you feel intimidated, remember...
  • 7. ● A way to interact with applications ● ReST (Representational State Transfer) API is a fancy way of saying to interact with an application the way a mid 1990’s Web browser would API (Application Programming Interface)
  • 8. Anatomy of retrieving a Web page Ordinary terminal connection Request main page at root of site Server sends back web page
  • 9. The browser converts text to what you see
  • 10. Real Alma API call Set up encrypted connection Request holdings record per API specification Holdings record returned
  • 11. To use the Alma API, you typically need to 1. Read a file containing information allowing you to build a command 2. Send that information to the API 3. Read and modify information you receive from the API Now what?
  • 12. The command line ● Often by far the easiest way ● More flexible than a GUI ● Supported by all major platforms ● Process files of any size ● Combine the power of individual programs -- they do all the heavy lifting for you!
  • 13. Handy tools for the Alma API Linux or MacOS command line curl -- Communicate with API service xmlstarlet -- Extract data from XML or edit XML sed -- Clean data cut -- Break data into fields echo -- Print data
  • 14. Learning the command line ● Learn one command at a time. Don’t worry about what you don’t need. ● Don’t worry if you get things wrong -- one advantage of the command line is you can try things many times until you get it right ● Google solutions for specific problems -- there are many online examples ● Try, but give up fast. Ask linux geeks for help
  • 15. Getting started with the command line ● MacOS (use Terminal) ○ Install Homebrew xcode-select --install ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" brew install coreutils ● Windows 10 ○ Enable linux subsystem and go to bash terminal
  • 16. Scripting is the command line! ● Simple text files that allow you to combine utilities and programs written in any language ● No programming experience necessary ● Great for automating processes ● For unfamiliar problems, ask for help
  • 17. Command Line Tools: curl, xmlstarlet, cut, and sed
  • 18. curl ● A tool to transfer data from or to a server ● Works with many protocols, can deal with authentication ● Especially useful for the Alma API
  • 19. curl examples curl -s -H "Authorization: apikey [your API key]" -H "Accept: application/xml" GET https://api-na.hosted.exlibrisgroup.com/[api call] curl -s -H "Authorization: apikey [your API key]" -d "$DATA" # data stored in variable -H "Content-Type: application/xml" PUT https://api-na.hosted.exlibrisgroup.com/[api call] curl -s -H "Authorization: apikey [your API key]" -d "@datafile" # data stored in file -H "Content-Type: application/xml" PUT https://api-na.hosted.exlibrisgroup.com/[api call]
  • 20. curl examples Reading records curl -s -H "Authorization: apikey [your API key]" -H "Accept: application/xml" GET https://api-na.hosted.exlibrisgroup.com/[api call] Updating records curl -s -H "Authorization: apikey [your API key]" -H "Content-Type: application/xml" -d "$DATA" PUT https://api-na.hosted.exlibrisgroup.com/[api call] Creating records curl -s -H "Authorization: apikey [your API key]" -H "Content-Type: application/xml" -d "@datafile" POST https://api-na.hosted.exlibrisgroup.com/[api call]
  • 21. Xmlstarlet ● A tool to parse, selectively display, update, and create XML data ● Syntax can be challenging - little parameters have a lot of power! ● Examples are the best way to learn - we’ll look at some next.
  • 22. Linux (and Windows Linux subsystem) sudo apt-get install xmlstarlet Mac brew install xmlstarlet Installing Xmlstarlet
  • 23. -t creates a template -m matches an XPATH expression -v selects a value -o outputs literal text -c + ‘count()’ gives a count Protip: ALWAYS strip out namespaces using sed -- xmlstarlet is picky about them, so they’re a pain. You can skip this with the Alma API because it doesn’t send a namespace xmlstarlet is easier than it looks
  • 24. Extract a value and store in a variable holdingid=$(echo $record |xmlstarlet sel -t -m "item/holding_data/holding_id" -v . ) Update a node within an XML document holdings=$(echo $holdings | xmlstarlet ed -u '/holding/record/datafield[@tag="852"]/subfield[@code="c"] ' -v 'oldstorbks') Xmlstarlet examples
  • 25. sed ● User regular expressions to select, parse, and transform lines ● Great for “fixing” data so that it can be used with other programs ● Extremely powerful and works great with very large (terabytes) files
  • 26. ● Strip everything except numbers from barcode variable barcode=$(sed 's/[^0-9]//g' <<< "$barcode") ● Strip all nonnumeric data from file1 with results sent to file2 cat file1 | sed 's/[^0-9]//g' > file2 sed examples
  • 27. Quick Regular Expression Guide ^ Match the start of the line $ Match the end of the line . Match any single character * Match zero or more of the previous character [A-D,G-J,0-5]* [A-D,G-J,0-5]* = match zero or more of ABCDGHIJ012345 [^A-C] Match any one character that is NOT A,B, or C (dog) Match the word "dog", including case, and remember that text to be used later in the match or replacement 1 Insert the first remembered text as if it were typed here (2 for second, 3 for 3rd, etc.) Use to match special characters. matches a backslash, * matches an asterisk, $ matches dollar sign, etc.
  • 28. cut -- Extract fields from delimited data Extract second field in comma delimited line variable secondfield="$(cut -d',' -f2 <<<$line)" Extract first through third fields in comma delimited file and send output to file2 cat file1 | cut -d',' -f1-3 > file2
  • 29. Read and process file # Read file named “barcodes” one line at a time cat barcodes | while read line do # Clean barcode barcode=$(sed 's/[^0-9]//g' <<< "$line") # print barcode echo barcode done
  • 30. Putting it All Together: A Practical Example
  • 31. 1. Create list of barcodes 2. Get mmsids and holdingsids from Alma 3. Retrieve holdings records based on mmsids and holdingsids 4. Update location in holdings record 5. Replace holdings record Example: Update locations based on barcode
  • 32. cat barcodes| while read barcode # Read input file named “barcodes” one line at a time do barcode=$(sed 's/[^0-9]//g' <<< "$barcode") # Clean barcode # Get record and extract record ids recinfo=$(curl -s -X GET -L -H "Authorization: apikey $(cat apikey.txt)" "https://api- na.hosted.exlibrisgroup.com/almaws/v1/items?item_barcode=${barcode}") mmsid=$(echo $recinfo |xmlstarlet sel -t -m "item/bib_data/mms_id" -v . ) holdingid=$(echo $recinfo |xmlstarlet sel -t -m "item/holding_data/holding_id" -v . ) # Retrieve holdings record url="https://api-na.hosted.exlibrisgroup.com/almaws/v1/bibs/${mmsid}/holdings/${holdingid}" holdings=$(curl -s -X GET -L -H "Authorization: apikey $(cat apikey.txt)" "${url}") # Edit location in holdings record holdings=$(echo $holdings | xmlstarlet ed -u '/holding/record/datafield[@tag="852"]/subfield[@code="c"]' -v 'oldstorbks') # Replace holdings record newloc=$(curl -s -H "Authorization: apikey $(cat apikey.txt)" -H "Content-Type: application/xml" - X PUT --data "${holdings}" "${url}") done A lot of work in just a few lines!
  • 33. Retrieve the record via API recinfo=$(curl -s -X GET -L -H "Authorization: apikey $(cat apikey.txt)" "https://api- na.hosted.exlibrisgroup.com/almaws/v1/items?item_barcode=33231000051658")) echo $recinfo |xmlstarlet fo
  • 34. Extract fields we want with xmlstarlet Then extract holdings and MMSID to construct another query to retrieve the holdings record
  • 35. Fix the XML holdings=$(echo $holdings | xmlstarlet ed -u '/holding/record/datafield[@tag="852"]/subfield[@code="c"]' -v 'oldstorbks')
  • 36. ● Use pipes “|” to make the output of one command the input of another ● Use command substitution $(command) to put use the output of one command within another ● Use backslashes to break a single command up across multiple lines to make it more readable Handy things to know with scripts
  • 37. Running your scripts ● Script first must be made executable. This must only be only once chmod 700 myscript ● To run, just get in the same directory and precede filename with a period and forward slash ./myscript
  • 38. Alma API Documentation https://developers.exlibrisgroup.com/alma/apis/ This presentation https://bit.ly/alma_api A few starter scripts https://github.com/banerjek/alma Resources