Building Complete State-of-the-art Natural Language Processing Projects with Free Software covers free AI-assisted text annotation tools & no-code model building, a Python library with one-line access to 10,000+ pre-trained models in 250+ languages, and an NLP Server to server NLP models & pipelines as REST API's.

1. Building Complete State-of-the-art
Natural Language Processing Projects
with Free Software
David Talby
CTO, John Snow Labs

2. 2
Agenda
Introduction
Python’s NLU Library
The Annotation Lab
The NLP Server
1.
2.
3.
4.

3. 60% of Technical Leaders stated that their NLP budget was at least 10% higher vs. 2020
33% of Technical Leaders stated that their NLP budget was at least 30% higher vs. 2020
NLP is Growing Into a Foundational Enterprise Technology
NLP Industry Survey 2021, Gradient Flow, July-August 2021
“Half of Technical Leaders are using technologies for
data integration, NLP, business intelligence, and data warehousing.”
Healthcare AI Survey 2021, Gradient Flow, January-March 2021

4. SO, HOW DO WE BEST HELP THE
COMMUNITY & INDUSTRY SUCCEED?

5. BY GIVING YOU MORE FREE SOFTWARE
(THAT IS PROVEN, ROBUST, SCALABLE, AND SECURE)

6. 6
BUILDING ON SPARK NLP
>10 million
O’Reilly Media
31% share
of NLP Practitioners
use Spark NLP
Gradient Flow
24x growth
In downloads of the
library during 2020
PyPI Download Stats
Downloads. Most popular
NLP tool in the enterprise

7. 7
Agenda
Introduction
Python’s NLU Library
The Annotation Lab
The NLP Server
1.
2.
3.
4.

8. INTRODUCING THE NLU LIBRARY
The Simplicity of Python, the Power of Spark NLP

9. EXAMPLE: TOKENIZATION & SPELL CHECKING

10. SENTIMENT ANALYSIS

11. LANGUAGE DETECTION

12. CALCULATING EMBEDDINGS

13. NAMED ENTITY RECOGNITION

14. UNSUPERVISED KEYWORD EXTRACTION

15. MULTI-CLASS TOXIC CONTENT CLASSIFIER

16. MULTI-CLASS TOXIC CONTENT CLASSIFIER

17. SUMMARIZATION

18. CLOSED-BOOK QUESTION ANSWERING

19. NLU ALSO HAS ONE-LINER VISUALIZATIONS

20. DEPENDENCY PARSER VISUALIZATION

21. NLU EVEN HAS ONE-LINER STREAMLIT APPS

22. NLU STREAMLIT APPS CHEAT SHEET

23. Getting Started with the NLU Library
ONE LINE INSTALLATION:
DOCUMENTATION:
https://nlu.johnsnowlabs.com/docs
COMMUNITY SUPPORT:
https://spark-nlp.slack.com @ #nlu

24. 24
Agenda
Introduction
Python’s NLU Library
The Annotation Lab
The NLP Server
1.
2.
3.
4.

25. JOHN SNOW LABS’ ANNOTATION LAB IS NOW FREE
Free
Unlimited users, documents,
projects, tasks, models, …
Share nothing:
Self hosted on your infrastructure
Robust
2 years in production
Now in its 23rd version
Actively developed & supported
Capable
Active learning, Transfer learning
Model training & deployment
Projects & Teams
Roles & Workflows
Versioning & Audit
Analytics: Productivity,
Agreement, Bias, Teamwork

26. High Productivity Annotation
Label entities, relations,
assertions, classes,
values, sequences, …
Keyboard Shortcuts
Versioned &
Auditable
Completions
Start from model
predictions
Shared Guidelines
One-click edits

27. Teamwork

28. Projects & Roles

29. Pre-Annotate: Don’t Start from Scratch
Filter Selected Tasks
Choose Model & Embedding
Filter Selected Model Labels
Asynchronous Model Server
(Runs on Kubernetes)

30. Automated Model Training & Active Learning
Async Training Container Realtime Training Log
Filter Completions
by State or Tag
Edit Hyper-Parameters
Choose
Embeddings
Train & Deploy
Or Fully Automate

31. Download, Upload, and Publish Models

32. Analytics
Project Tracking Productivity
Bias in Dataset or Guidelines

33. Inter-Annotator Agreement
Agreement between annotators,
or between human & model
Drill-down to specific cases

34. Getting Started with the Annotation Lab
ONE LINE INSTALLATION:
DOCUMENTATION:
https://nlp.johnsnowlabs.com/docs/en/alab/quickstart
COMMUNITY SUPPORT:
https://spark-nlp.slack.com @ #annotation-lab

35. 35
Agenda
Introduction
Python’s NLU Library
The Annotation Lab
The NLP Server
1.
2.
3.
4.

36. JOHN SNOW LABS’ NLP SERVER IS NOW FREE
Free
Unlimited models, calls, servers
Share nothing:
Self hosted on your infrastructure
Early Stage
Started in 2021
Now in its 3rd version
Actively developed & supported
Capable
No coding required
3,000+ models & growing
UI + REST API

37. The User Interface
1. Search spells by name or description
2. Type text to analyze or upload a CSV
3. Hit the ‘Preview’ button

38. Viewing Results
1. Models are downloaded to local storage
2. Models are loaded & cached in memory
3. Model inference is executed

39. Downloading Results

40. The REST API
Same functionality as the user interface, delivered as a single Docker container

41. NLP Server Roadmap
Model training & auto-tuning
Serve your custom trained NLP models
Run on air-gap enterprise environments
Model publishing from the Annotation Lab
Run Healthcare NLP & Spark OCR models

42. Getting Started with the NLP Server
ONE LINE INSTALLATION:
DOCUMENTATION:
https://nlp.johnsnowlabs.com/docs/en/nlp_server/nlp_server
COMMUNITY SUPPORT:
https://spark-nlp.slack.com @ #nlp-server

43. Please put it all to good use!
https://nlp.johnsnowlabs.com/docs

44. 44
Thank you!
© 2015-2021 John Snow Labs Inc. All rights reserved. The John Snow Labs logo is a trademarks of John Snow Labs Inc. The included information is for informational purposes only and represents the current
view of John Snow Labs as of the date of this presentation. Since John Snow Labs must respond to changing market conditions, it should not be interpreted to be a commitment on its part, and John Snow
Labs cannot guarantee the accuracy of any information provided after the date of this presentation. John Snow Labs makes no warranties, express
or statutory, as to the information in this presentation.
david@JohnSnowLabs.com
@davidtalby
in/davidtalby