For most organizations deploying Teams, only about 16% of employees turn out to be early adopters. That means, if you are or will be engaged in a Teams deployment, you’ve only got another 84% to get on board to ensure adoption budget isn’t wasted. In this hybrid technical & adoption session, you will learn from real-world customer examples how usage and engagement analysis has helped others like you overcome common adoption problems. We will go through all the available data sources such as the Teams Admin Center, Office365 Activity Reports, Microsoft Graph & PowerShell and how you can put them to work to better understand your organization's usage patterns. You will leave with a thorough understanding of the strengths and limitations of the data available to you and how to interpret and use this data to maximize user engagement in the shortest amount of time and ensure the best possible return on your investment.

1. Make Your Data Work For You
Analyzing Microsoft Teams engagement
& adoption: Why, What, How? Ben Menesi
Toronto, 2019

2. 2
Thank you to our sponsors!
PLATINUMSILVERGOLD

3. 3
Speaker
Ben Menesi
VP Products & Innovation at panagenda
Started out in the IBM world
SharePoint & Exchange Admin & Dev
Speaker at IT events around the globe
Owns a bar (recently)
@BenMenesi

4. 4
panagenda
Who we are
HQ in Vienna, Austria, offices in Germany, USA and Australia
10M+ user licenses across 80 countries

5. 5
Agenda
WHY
do we want Teams to succeed in our organization?
Should we measure adoption
Why is engagement important?
WHAT
questions do you need answers to?
does Teams rely on in the background?
data sources are available?
HOW
do you use those data sources to connect the dots?
do you set KPIs to measure?
do I get the missing analytics data?

6. Make Your Data Work For You
WHY

7. WHY do we want Teams to succeed in
our organization?

8. 8
Why do we want Teams to succeed?
No more working in Silos
„Oh, my team did that too!“
„Let me check if my predecessor sent an email about that!“
CapEX to OpEX
25% MS365 budget goes towards teams
5000 user organization: USD $1,8M over 3 years (Forrester TEI of Teams)
One stop shop collaboration
What are the benefits of working in one tool vs. Myriad of business
software?

9. 9
Teams becoming the heart of collaboration
Benefits of one tool to rule them all
Numbers Roman Numbers Alphabet
1 I A
2 II B
3 III C
4 IV D
5 V E
6 VI F
7 VII G
8 VIII H
9 IX I
10 XI J
40 sec
20 sec

10. WHAT does Teams rely on in the
background?

11. What does Teams rely on in the background?
13
Source: https://docs.microsoft.com/en-us/microsoftteams/teams-architecture-solutions-posters

12. So what happens when a user creates a Team?
14
Upon users creating a new Team, the following services are provisioned
Azure AD Group Directory Object (Office365 group)
Exchange Online Shared Mailbox and Calendar
SharePoint Team Site and document Library
OneDrive for Business
Shared OneNote Notebook
Planner
Microsoft Stream for meeting recordings
+ … SharePoint Online Site collections for each private channel

13. What are ways to create new Teams?
15
Source: https://docs.microsoft.com/en-us/microsoftteams/teams-architecture-solutions-posters

14. WHAT questions do we need answers
to?

15. Questions we need answers to
1.) Organization-wide Teams adoption trends
How does Teams adoption & usage look like in my organization?
2.) Department & Geographical adoption figures
Which are my departments leading the adoption effort?
What are the countries that are lagging behind the adoption effort?
3.) Who are my champions / early adopters?
Which users can I count on to spread the word?
Think SPS and MVP ecosystem

16. Questions we want answers to
4.) Working in silos – Teams vs. Email
Teams adoption should be inversely proportional to internal E-Mail volume
For which departments is this true, what departments need adoption love?
5.) Working in silos – Teams activity proportions
Public vs. Private Teams?
Chat messages: working in silos. Plus: Files & knowledge loss
6.) Working in silos – Teams vs. Teams vs. Teams Sprawl
What is my logical Teams architecture strategy? Do I have similar Teams?

17. Questions we need answers to
7.) Adoption – Is Teams facilitating cross-department work?
Communication within the same departments isn‘t enough.
6.) Am I overdoing it?
How much noise do I expose my employees to?
What is the ideal amount of Teams memberships, channel memberships to
optimize engagement?

18. WHAT data sources are available to me?

19. 21
Microsoft Teams Usage Data
Office365
Activity
Reports
Teams
Analytics
Reports
Teams
Admin Center
Microsoft
Graph
PowerShell

20. Microsoft Teams Admin Center

21. Microsoft Teams Admin Center
1.) The good:
Overall Teams activity analytics
Teams list – activity, guests, active channels
User Activity: channel messages, chats, calls
2.) The bad:
Limited range (7 / 28 days)
No users / teams correlation
No activity details, channel trends, organizational data

22. Office365 Usage Reports (Teams)

23. Office365 Usage Reports (Teams)
1.) The good:
2 reports: user activity and device usage
Number of activities by type and users by activity type
Compares usage across the entire Office365 stack
2.) The bad:
Still limited range (7 / 30 / 90 / 180 days)
No organizational context

24. Microsoft 365 Usage Analytics PowerBI Pack

25. Microsoft 365 Usage Analytics PowerBI Pack
1.) The good:
Global Active Users report across the entire office365 stack
Some BI on top of your data
2.) The bad:
Relies on Usage Reports (limited dataset)
Aggregated monthly data only – no way to drill down to more details
No Teams > Users > Activity correlation, just global trends

26. PowerShell & Microsoft Graph
1.) The good:
You get everything
As granular as it gets
2.) The bad:
As opposed to the above reports: not a turnkey solution for analytics
Requires regular queries, data storage & assembly, takes time
Expensive

27. Native Office365 reports & data
Reports are great and usable but:
Key: Data
Issue 1: Volume (only up to 180 days)
Issue 2: Granularity and lack of data-point connectivity

28. Make Your Data Work For You
HOW

29. HOW do I use these data sources to
connect the dots?

30. 32
Teams Admin Center
Teams > Manage Teams
What does it look like

31. 33
Teams Admin Center
Teams > Manage Teams
List of all teams
Available: # Channels, #Members, #Owners, #Guests, Privacy and Status
Helps you answer questions:
How many Teams do I have?
What are my Teams with the most members? What‘s the average #
channels?
What‘s the proportion of public vs. Private teams?

32. 34
Teams Admin Center
Teams > Manage Teams > {Team Name}
What does it look like

33. 35
Teams Admin Center
Teams > Manage Teams > {Team Name}
List of all members and channels
Members available: User names, titles, locations, roles
Channels available: Name, description, type, auto-pin (Watch out! No private
channels unless you‘re a member / owner!)
Helps you answer questions:
Members by location
# of members for each channel

34. 36
Teams Admin Center
Teams > Analytics & Reports > Teams user activity report
Activity Type for all users: 1:1 calls, channel messages, chat messages
Helps you answer questions:
General Teams usage volume, and by activity type
Working in silos: channel messages vs. Chat messages
Disadvantages
Only available up to 28 days
No filtering capabilities

35. 37
Teams Admin Center
Teams > Analytics & Reports > Teams device usage report
Not just devices: Windows / Mac and IOS and Android
Helps you answer questions:
Teams device usage trends: mobile vs. Computer
Disadvantages
Only available up to 28 days
No filtering capabilities

36. 38
Teams Admin Center
Teams > Analytics & Reports > Teams usage report
#active users, Teams & channels active users, active channels, #messages
Teams list: name, active users, guests, active channels
Helps you answer questions:
Globally active users and trends
Connects teams with their activity
Disadvantages
Volume, and no activity information (chat vs. Messages) per team

37. 39
Teams Admin Center Conclusion
Teams Admin Center Analytics
Answers a few questions and provides general usage overview
Personal favourites: Activity type breakdown
Disadvantages
Volume is a real issue, and the lack of deep-dive into departments and
organizational information is a huge loss.

38. 40
Office365 Usage Reports (Teams)
O365 Usage Reports
What it looks like

39. 41
Office365 Usage Reports (Teams)
O365 Usage Reports
Active users across all stacks – up to 180 days
Personal favourites: Activity type breakdown
Helps you answer questions:
Activity trends across the entire Office365 stack
Disadvantages
Volume is a real issue, and the lack of deep-dive into departments and
organizational information is a huge loss.

40. 42
Office365 Usage Reports PowerBI Pack
O365 Usage Reports
What it looks like

41. 43
Office365 Usage Reports PowerBI Pack
O365 Usage Reports PowerBI Pack
Active, returning, new, and enabled users for Teams and other products
Helps you answer questions:
Adoption progress across the entire Office365 stack
Disadvantages
Monthly aggregated data – no way to dig deeper
Only focuses on active users, not activities

42. HOW do we set KPIs to measure?

43. 45
The main question: Adoption curve
What are the personas in an organization based on their adoption behaviour?

44. HOW do I get the missing figures?

45. 47
Building your own app
Microsoft Graph is cool
Register your application
Select and consent to permissions
Build your data models and pick a data warehouse software
Harvest Teams data
Repeat

46. (Our) example implementation
Data Warehouse
Usage Data Azure Data
Data Warehouse
Data Processing
Data Representation
OE Interface Metabase DataMiner
Assembled
usage data
Web API

47. Assembled Usage Data
Teams List
Channels Info
Teams User
Usage Data
Office365
Usage Data
Teams Activity
Data Azure AD
Organization Data
Azure AD User Data
Office365 User
Usage Data
Teams App
Data
Teams
Member &
Owner Info

48. 50
Building your own app
1.) Registering the application
Portal.azure.com > Azure AD > App Registrations > New Application Reg.

49. 51
Building your own app
2.) Add APIs and permissions
Consider Reports
Work with users & gather organizational intel (department, country, etc)
Watch out! Teams API is protected: aka.ms/teamsgraph/requestaccess

50. 52
Building your own app
3.) Build data models
Consider joining user attribute tables with Teams activities

51. 53
Building your own app
4.) Select a DW software
You might want to deploy this into Azure

52. 54
Building your own app
5.) Harvest Teams data
And keep doing it

53. 55
Use the Reports Graph API
Use the Teams reports available via the Reports Graph API
https://docs.microsoft.com/en-us/graph/api/resources/microsoft-teams-user-
activity-reports?view=graph-rest-1.0
What you get:
Report Refresh Date
User Principal Name
Last Activity Date
Is Deleted
Deleted Date
Assigned Products
Team Chat Message Count
Private Chat Message Count
Call Count
Meeting Count
Has Other Action
Report Period

54. 56
panagenda
What we do: OfficeExpert | Teams Analytics
Virtual Appliance: Data stays secure
Cross-organizational intelligence
Usage & Engagement measurement
Integrated Teams Application
33+ PowerBI reports
We are looking for design partners!
https://panagenda.com/officeexpert-sandbox

55. Make Your Data Work For You
Sandbox:
https://www.panagenda.com/officeex
pert-sandbox/

56. Make Your Data Work For You
Q & A

57. Feedback
5959 Join the conversation using #OfficeExpert & @panagenda
http://bit.ly/spsto19eventevalhttp://bit.ly/spsto19sessioneval

58. THANK YOU!
6060 Join the conversation using #OfficeExpert & @panagenda
Linkedin.ca/in/benedekmenesi
@BenMenesi
Let‘s get in touch:
ben.menesi@panagenda.com

59. Headquarters, Austria:
panagenda GmbH (Ltd.)
Schreyvogelgasse 3/10
AT 1010 Vienna
Phone: +43 1 89 012 89
Fax: +43 1 89 012 89-15
E-Mail: info@panagenda.com
Headquarters, Germany:
panagenda GmbH (Ltd.)
Lahnstraße 17
DE 64646 Heppenheim
Phone: +49 6252 67 939-00
Fax: +49 6252 67 939-16
E-Mail: info@panagenda.com
USA:
panagenda Inc.
60 State Street, Suite 700
MA 02109 Boston
Phone: +1 617 855 5961
Fax: +1 617 488 2292
E-Mail: info@panagenda.com
Germany:
panagenda Consulting GmbH (Ltd.)
Donnersbergstrasse 1
DE 64646 Heppenheim
Phone: +49 6252 67 939-86
Fax: +49 6252 67 939-16
E-Mail: info@panagenda.com
The Netherlands:
Trust Factory B.V.
11th Floor,
Koningin Julianaplein 10
NL 2595 AA The Hague
Phone: +31 70 80 801 96
E-Mail: info@trust-factory.com
© 2007-2015 panagenda
Make Your Data Work for You