Case Study from Analytics Project work for Leena.ai

1. Analytics solution for Leena.ai Products
Case Study 01
Rol
Defining vision and proces
Leading Product Desig
Planning and conducting UX Research
Tea
Product Designe
UI Designe
Product Manage
BE Enginee
FE Engineer
Timeline
2 months
Leena AI
Business Intelligence re-imagined

2. LeenaChatBot
Background
Hi How can I help you?
Leena AI
Online
Privilege leave policy
Privilege Leave policy
leave policy.pdf
下载付款收据
πολιτική άδειας μητρότητας
में
आपकी क
ैसेमदद कर सकता
हूं?
Demander un congé maternité
Do you want me to raise a
ticket?
Политика возмещения
Hi Bree, I can help you with the
following
Payslips
Raise a ticket
Holiday Calendar
Leave Policies
AI chatbotthatinteracts withEmployees to
solveany HR-related needs
The bot is used by the HR teams to automate
and streamline necessary services to thousands
of Employees.

3. Analytics
Problem Background
Understanding the user acquisition and
usage of the bot by employees is paramount
for improving the services that are provided
and adding new functionalities to the bot.
A third party tool, Grafana was being used to
host all analytics data.
It was falling short in many ways for the
growth of the company.

4. UserProblems
BusinessProblems
ProblemBackground
Havingtologintoathirdpartytoo
Antiquatedvisualdesig
Poornavigatio
Lackofanabilitytosearch
Baduserexperienc
Inabilitytoaddanyfunctionalitie
Hinderingcompanygrowth

5. Analytics
Usage Activity
200
150
100
50
0
Candidate added
Sessions per user
US
New hires
Indo
UK
50 40 30 20 10 0
New Users 413 12%
Total Users 116 4%
Retained 93 8%
App downloads 397 --
How might we
provide a great user experience
to users who want to understand
what’s happening on the Bot,
via the Platform?
Problem Statement

6. Process
DISCOVER DEFINE DESIGN DELIVER
Understand the metrics
User Interviews
Synthesize research
Prototyping Testing
Build & release
Final design and handoff
Card Sorting
Information Architecture
Wireframes
Make an easy&intuitivedashboard that sits within the Leena HR app suite
Goal
Make an easy&intuitivedashboard that sits within the Leena HR app suite
Goal
Process

7. Understand
the dashboard,
how the bot works,
and learn about
chatbot concepts and
terminologies
Discover

8. Discover
Client SPOCs
Responsible for making sure
the bot is made available to all
employees & promote its
usage in their organization.
Acquiring and retaining employee
Run campaigns based on data
gathered by the metrics on the
grafana dashboard to increase
usage on our bots.
Growth Marketing Managers
Make sure that the onboarding process is smooth for
the clients
Help clients set up the dashboard and give clients a
walkthrough of the product
Take inputs from the clients to make the custom
dashboards to see relevant data points.
CSMs
We conducted user interviews with
the following Research objectives:
Find what was effective in
Grafan
Find pain points with Grafana
Understand the expectations of
an ideal analytics dashboard.
Who are the users of
Leena Analytics?
Figjam link with all the Research - link

9. Define
Navigation and
search ability needed
to be fixed
Crisp and clear
explanation of
metrics is missing
Ability to add custom
dashboards, with
applied filters is
needed
Need to find
employee cohorts
based on specific
conditions
Information is
scattered throughout
the dashboard in a in
a confusing way
Phase 1
Bring all the as-is from
Grafana to Leena
Make a consumer grade UX
with Filtering, better nav,
intuitive and easy to use
Launch Beta version
with few clients
Custom Reporting
Cohort Analysis
Full rollout to all clients,
and enable use by other
Leena products
Presentation mode
Dashboard sharing and
access control
Keep innovating
Phase 2 Phase 3
Key Takeaways and Design Plan

10. Dashboard
Home User Metrics
User Acquisitio
User Demographics
Total Use
Total Sessions
User Queries
Engagemen
Bot Metrics
Session Metrics
Live Date
Queries
Use
User acquisitio
User distribution
Bot interactions (inc
Most viewe
Most viewe
Bot usage analytics
bot message count
bot usag
User List
Tickets
Ticket counts (total
Ticket distribution
Avg time fo
Avg time fo
Avg interactions
User sessions
Handled/no
Average
Interactions per
sessio
Average time
between session
Active user
Average time for
retentio
Login coun
New user coun
Most querie
Feature usage dist
Most viewed policy
Active users
Non acq use
Use
Login count
New use
Channel wis
Total interactions
Unused features
Total interactions
User says
User says
Active users
Login count
Bot performance
Thumbs up/down
Bot Response time
QRclicks
Handled/no
Session count (total
Overview
User metrics
Bot metrics
Session metrics
Queries & Interactions
Custom dashboard 1
Custom dashboard 2
Design
We conducted an open card
sort exercise with 20 internal
users (CSMs and GMMs) via
Optimal Sort, to arrive at the
best possible IA solution.
InformationArchitecture

11. Design
Dashboard Name Restricted Today Last week Last month Custom Edit
Bot Analytics
Overview
User Metrics
Bot metrics
Session Metrics
Queries & Interactions
Add new dashboard
Ticket Analytics
Workflow Analytics
Workflow Analytics
Analytics Search for something... Arlene McCoy
Export
configuration settings
Location 1
Location 2
Location 3
Location 4
Search for a metric...
Department 10 Regions 10 Regional Manager 10 Locations 10 Joining date 10 May, 2022 - 31 May, 2022 Clear all
I’m looking for
Cards Sections
Recent Searches
Bot Engagement
User Analytics
Acquisition
Menu&Controls-Layout

12. Design
Stacked Bar Charts
Amet minim mollit non deserunt ullamco est sit aliqua dolor do amet sint.
Count
Sessions Effective sessions Abandoned sessions
100
80
60
40
20
0
Mon Tue Wed Thu Fri Sat Sun
Stacked Bar Charts
Amet minim mollit non deserunt ullamco est sit aliqua dolor do amet sint.
Count
Sessions Effective sessions Abandoned sessions
100
80
60
40
20
0
Mon Tue Wed Thu Fri Sat Sun
Horizontal bar chart
Amet minim mollit non deserunt ullamco est sit aliqua dolor do amet sint.
Leena-WC 12 mins
Slack 19 mins
Whatsapp 5 mins
Workplace 8 mins
Horizontal bar chart
Amet minim mollit non deserunt ullamco est sit aliqua dolor do amet sint.
Leena-WC 12 mins
Slack 19 mins
Whatsapp 5 mins
Workplace 8 mins
Funnel Chart
Amet minim mollit non deserunt ullamco est sit aliqua dolor do amet sint.
Total employees
Signed up users
Acquired Users
Retained Users
Funnel Chart
Amet minim mollit non deserunt ullamco est sit aliqua dolor do amet sint.
Total employees
Signed up users
Acquired Users
Retained Users
Multiple line chart - time series
Amet minim mollit non deserunt ullamco est sit aliqua dolor do amet sint.
Active
users
1,000
800
600
400
200
0
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Month
March 28th 2022
Number 642
24% past month
Number 555
Multiple line chart - time series
Amet minim mollit non deserunt ullamco est sit aliqua dolor do amet sint.
Active
users
1,000
800
600
400
200
0
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Month
March 28th 2022
Number 642
24% past month
Number 555
Download as PNG
Download as CSV
Pie Chart
Amet minim mollit non deserunt ullamco est sit aliqua dolor do amet sint.
Location Department
Leena webclient 30%
Slack 20%
MS Teams 10%
Whatsapp 5%
Mail 5%
Workplace 30%
Slack 642 30%
24% past month
Pie Chart
Amet minim mollit non deserunt ullamco est sit aliqua dolor do amet sint.
Location Department
Leena webclient 30%
Slack 20%
MS Teams 10%
Whatsapp 5%
Mail 5%
Workplace 30%
Slack 642 30%
24% past month
Charts-DataViz

13. Design
Total users
9,076
Logged in users
2,076
Acquired users
1076
Retained users
976
Non-users
686
Total users
9,076
Logged in users
2,076
Acquired users
1076
Retained users
976
Non-users
686
Thumbs up
Number of user messages that were continued with a thumbs-up.
Search...
Username
Fattum Bozok
Riza Cakar
Ayse Lutfiye Atli
Kadin Bozkurt
Huseyin Akcagul
Yilmaz Orhan
Nurettin Ozturk
Channel
WC
WC
WC
Slack
Slack
Slack
Slack
Status
Active
terminated
Blue collar
Blue collar
Blue collar
Blue collar
Blue collar
Employment type
Blue collar
Blue collar
Blue collar
Blue collar
Blue collar
Blue collar
Blue collar
Department
Manufacturing
Manufacturing
Manufacturing
Manufacturing
Logistic
Logistic
Logistic
Location
Delhi
Delhi
Delhi
Delhi
Delhi
Delhi
Delhi
Total users
Total number of unique users that have access to the bot.
Username
Fattum Bozok
Riza Cakar
Ayse Lutfiye Atli
Kadin Bozkurt
Huseyin Akcagul
Yilmaz Orhan
Nurettin Ozturk
Channel
WC
WC
WC
Slack
Slack
Slack
Slack
Status
Not-terminated
terminated
terminated
terminated
Not-terminated
Not-terminated
Not-terminated
Employee type
Blue collar
Blue collar
Blue collar
Blue collar
Blue collar
Blue collar
Blue collar
Department
Manufacturing
Manufacturing
Manufacturing
Manufacturing
Logistic
Logistic
Logistic
Location
Delhi
Delhi
Delhi
Delhi
Delhi
Delhi
Delhi
Active users
5,499 (60%)
Active users
5,499 (60%)
Number card with icon
976
Number card with icon
976
Widget name - Text Card
The most used feature on the bot.
Feature
Feature name
Widget name - Text Card
The most used feature on the bot.
Feature
Feature name
Widget Name - number card
Card with percentage depiction - hover
Total users
6,758 /10,000 34%
Widget Name - number card
Card with percentage depiction - hover
Total users
6,758 /10,000 34%
Widget name - number card
Number card
Total session count
1,234
Widget name - number card
Number card
Total session count
1,234
Tool tip
Number of new & old users who have used
the bot in the selected time period.
Table &Cards-Data Viz

14. Design
Layout-withfullcontent
Dashboard Name Restricted Today Last week Last month Custom Edit
Tip: Amet minim mollit non deserunt ullamco est sit aliqua dolor do amet sint. Velit officia consequat duis enim velit mollit. Exercitation veniam
Bot Analytics
Overview
User Metrics
Bot metrics
Session Metrics
Queries & Interactions
Add new dashboard
Ticket Analytics
Workflow Analytics
Workflow Analytics
Analytics Search for something... Arlene McCoy
Widget name Till now
Lorem ipsum dolor sit amet
Total users
1000/10,000
Know more
Widget name
Lorem ipsum dolor sit amet
Acquired users
600 (60%)
Know more
Widget name
Lorem ipsum dolor sit amet
Retained users
200 (20%)
Know more
Section name
Lorem ipsum dolor sit amet
Note There was a huge spike in user activity last night View
Widget name - Funnel Chart
Amet minim mollit non deserunt ullamco est sit aliqua dolor do amet sint.
Total employees
Signed up users
View in full-screen
Download as PNG
Aquisition statistics
Users that have interacted with the bot till now. The chart shows distribution
based on channels and departments.
Channel wise Department wise
Leena webclient 30%

15. DeliveredPhase1
Deliver
We worked closely with the Dev team to fix UI
bugs and issues, come up with UI alternatives,
and have weekly check in calls.
Launched the Beta version to few clients in Jun
2022.
Clients and internal users preferred the new
dashboar
We co-created and prioritized the roadmap
further, with the
Conducted feedback sessions

16. CustomDashboards
Design Phase 2
New dashboard Today Last week Last month Custom New panel
Amet minim mollit non deserunt ullamco est sit aliqua dolor do amet sint. Velit officia consequat duis enim velit mollit. Exercitation veniam consequat sunt nostrud amet.
There are no panels here yet
Add a panel by clicking the blue button
Bot Analytics
Overview
User Metrics
Bot metrics
Session Metrics
Queries & Interactions
New dashboard
Add new dashboard
Analytics Search for something... Arlene McCoy

17. CustomDashboards
Design Phase 2
New dashboard Today Last week Last month Custom New panel
Amet minim mollit non deserunt ullamco est sit aliqua dolor do amet sint. Velit officia consequat duis enim velit mollit. Exercitation veniam consequat sunt nostrud amet.
There are no panels here yet
Add a panel by clicking the blue button
Bot Analytics
Overview
User Metrics
Bot metrics
Session Metrics
Queries & Interactions
New dashboard
Add new page
Analytics Search for something... Arlene McCoy
Add new panel
Panel type
Time series chart
Panel name
Active users over time (Desing)
Active users over time (Desing)
Description (Optional)
Write what it is about...
Panel controls
X Axis
Time
Y Axis
Active users
Filter
Add filter
New panel Add this panel
Active users over time
Active
users
1,000
800
600
400
200
0
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Month

18. CustomDashboards
Design Phase 2
New dashboard Today Last week Last month Custom New panel
Amet minim mollit non deserunt ullamco est sit aliqua dolor do amet sint. Velit officia consequat duis enim velit mollit. Exercitation veniam consequat sunt nostrud amet.
Bot Analytics
Overview
User Metrics
Bot metrics
Session Metrics
Queries & Interactions
New dashboard
Add new page
Analytics Search for something... Arlene McCoy
Active users over time
Total
2,758 (60%)
Active users over time
Active
users
1,000
800
600
400
200
0
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Month

19. Cohort Analysis
Design Phase 2
Employee name
EK
Emine Kiyag
emine@company.com
KT
Katheryn Murphy
karem@company.com
AA
Ayse Lutfiye Atli
ayse@company.com
IC
Ibrahim Cuce
Ibrahim@company.com
MT
Mehmet Rauf Tunc
mehmet@company.com
HG
Hanife Gunduz
hanife@company.com
OR
Olivia Rhye
olivia@company.com
AA
Aynur Alan
aynur@company.com
Employee ID
#3066
#3065
#3064
#3063
#3062
#3061
#3060
#3059
Designation
Head of HR
Software Engineer
QA Engineer
Programmer Analyst
Project Manager
Project Manager
Chief Marketing Officer
DevOps
Department
Human Resources
Engineering
Engineering
Engineering
Engineering
Engineering
Marketing
Engineering
Joining date
Jan 6, 2022
Jan 6, 2022
Jan 6, 2022
Jan 5, 2022
Jan 5, 2022
Jan 5, 2022
Jan 4, 2022
Jan 3, 2022
Show all users
Who did not Launch app in Last 3 months where Location is UAE and +3 more
Filter
Clear all Save
Swiggy Users 345 Cohort
Overview
User metrics
Bot metrics
Session metrics
Queries & Interactions
All users
Custom dashboard 1
Custom dashboard 2
Analytics Search for something... Arlene McCoy

20. Development&Rollout
Reflection
A full rollout is currently ongoing to customers
one by one
Users have already started benefiting from
all the added functionalit
Other Leena apps have started using our to
add product specific analytics
Satisfaction scores by internal users is great,
and there’s a rush amongst CSMs to push for
getting the new analytics added for their
clients before others.
Grafana Dashboard will not used or set up for
new customers and officially sunset by end of
Q2 2023.