Back to blog

A look at Microsoft’s Newest Office 365 features: Usage and Adoption Reporting

Oct 13, 2016 by Emma Robinson

In the past, we’ve talked a lot about the capabilities of native reporting in Office 365, and we’ve monitored its progress as gradual improvements have been made. So when we attended Ignite last month, we were interested to attend the session on usage reporting, and hear about the newest updates, both in the Admin Center, and with the announcement of a brand new Office 365 Adoption Content Pack for PowerBI.
In the session, Anne Michels compared Office 365 to the Physics set that her father bought her when she was a child. Before she was shown how to use it, she had little interest in the set, but with her father’s guidance and instruction, it became one of her favourite toys. While the analogy was a little long-winded, the message was clear: If you don’t show your users how to get value out of Office 365, they will not use or value it, and your organisation won’t receive the benefits.
The problem with Office 365 is that, up until this point, admins have struggled to get comprehensive user-level usage information from their platform, so measuring service adoption (without the help of non-native solutions) has been challenging. On top of this, any useful reporting information that they have been able to glean from the native reports has been – as they termed it in the session – on ‘lock down’ – meaning that it was available for admin eyes only, with very limited sharing or export functionality.
With this in mind, the session announced two new features for usage reporting: an activity dashboard in the new Office 365 Admin Center, and an Office 365 Adoption Content Pack, which can be used with PowerBI. The activity dashboard (shown below) provides a cross-service overview of how your entire organisation is using Office 365, with basic metrics on areas like ‘Email activity’, ‘OneDrive files’ and ‘Skype for Business Activity’, and ability to drill down to user-level activity.
The graphs are responsive and customisable, with three (slightly rigid) date options, and the dashboard view makes it much easier to consume the information at a glance. While these reports are geared towards adoption, as ‘Office 365 admins often have to provide reports on a weekly or even daily basis’, it still isn’t possible to access these reports if you are not an admin, and the graphs cannot be shared (although some of the data can be exported to csv.). This means that the new activity dashboard doesn’t fully solve the problem it sets out to combat because when an admin needs to ‘report back on the investment to your management’ there’s still some mandatory Excel legwork to be done before the adoption reports can be presented in a useful form to stakeholders.
Now for the next feature – the Office 365 Adoption Content Pack
Anyone who has used PowerBI before will know that it is an extremely powerful tool. It uses your data to create rich, interactive dashboards, which can be shared across the entire organisation (as long as your users are licensed for the tool). With the announcement of the limited preview for a new Adoption Content Pack for Office 365, Shilpi Sinha talked about how users will finally be able to access intelligent and interactive reporting on usage, with the capability to drill down into specific services, departments and regions in order to truly understand adoption.
The session was largely demo-based, showing how the reports can be configured (you must be an admin to do this). For new PowerBI users, it may take a little while to get comfortable configuring the data to display what you need, but there is certainly more information available than there ever has been in the native reports. Another aspect of the feature is that there is (finally!) some visibility for non-admin users. As Shilpi explained, the core admins are the only ones who are able to set up the reports, but once they have published them onto the dashboard it is possible for non-admins to log into PowerBI and get view-only access to the reports dashboard. The dashboard is split into four areas: Adoption, Communication, Collaboration and Activation, and non-admins can view any published reports in these areas. Unfortunately, there is minimal ability to customise or manipulate the information available to these non-admin users once the reports are published, so if a non-admin needed something specific, they may have to go back to an admin in order to get their requirements configured into a report that they’re able to see. This system clearly continues to pose some issues in terms of accessibility, customisation and demands on admin-time, but there is no doubt that this is a significant step in the right direction.
At the moment, (as its title suggests) the content pack is focused on Adoption, so there is no reporting on security settings, or configurations such as in-place hold, or litigation hold. The reports only feature licensed users, with no details on groups, shared mailboxes or public folders, which seems odd as adoption and collaboration are trends that go hand in hand. Overall, as a starting point for better, and more accessible reporting and analytics, the new content pack is definitely an improvement on what has gone before it.
Be ‘judicious’… a shift in the Office 365 message?
One aspect of the session that we found interesting was the messaging around availability. The limited preview for the new features will be rolling out throughout the course of October (we applied for the preview at Ignite, so once we’ve got access we will be posting a deep dive into all areas of the Adoption content pack).
Once it is generally available, the content pack will not be enabled by default – so you will have to turn on (Note: the data will only begin to collect once it has been enabled). In her session Shilpi Sinha explained that you should be ‘judicious’ when deciding whether you would like this capability or not. This messaging is particularly curious in contrast to that of Anne Michels described above – and Microsoft in general. For years now we have become accustomed to the Microsoft heavily encouraging the use of every new feature that they release in Office 365, and this is a feature which is wholly concerned with the adoption of new features, maximising value and reaping the benefits. So why would we need to be ‘judicious’ in deciding whether we would like to enable it or not?
One possible reason is that PowerBI is as powerful as it is complex, and your organisation’s Office 365 usage data is likely to be on the large and unwieldy side too, meaning that it could require a lot of set up time (both for Microsoft to backfill the reports, and for you to set them up correctly). Shilpi and her team probably mean that this feature is here, and available, but (due to its complexity, capability, and resource demands) the advice is: use it, but only if you need it. As the feature is rolled out, Microsoft will be providing documentation to decode some of the complexities of configuring your Office 365 data within PowerBI, but they seemed to suggest that if you are able to get enough information using native reports, or PowerShell, or perhaps if you don’t need a drill down of all your pertinent adoption metrics, then it may not be worth investing your time and efforts in PowerBI.
Want to find out more? Our latest post on the Content Pack explores this new functionality in more detail.
Cogmotive is the leading global provider of enterprise level reporting and analytics applications for Office 365. Find out more now.