Application Telemetry in SentryOne

Please note that this blog has been updated to include a link to our 11.2 product release tour.

With version 11.2 of the SentryOne Platform we have introduced application telemetry. In the interests of being completely transparent I am going to provide information on what we capture, why, and how you can enable/disable this feature.

Why are we doing this?

Application telemetry is both contentious and essential in building great software solutions. Here at SentryOne we have traditionally worked closely with our user community and the SQL Server community at large, gaining insight from interviews, conversations, and our own experiences from working with SQL Server over the years. However, this can only scale so far. In our quest to build better software for the Data Platform community, we need to get more feedback. This is where application telemetry comes in.

We want to continue working with our users to understand how they are using our software, and where we can improve it. We want our users to be active in our engineering processes, from providing feedback through to testing Beta versions. Here at SentryOne we do not want to take an "if you build it, they will come" approach. For new feature requests, we will still rely heavily on personal interaction. But to understand how you use our software, we need it to tell us. Simply put, I cannot interview every single user of the SentryOne Platform.

What data we are NOT collecting?

Before we look at the data that we are collecting, I want to make it perfectly clear what we are not collecting.

We are not collecting any data related to activity on any of your Data Platform estate. I can personally assure you that if you enable application telemetry in the SentryOne client we will not get any of the following:

• T-SQL Queries
• Database schema information
• Data from your databases
• Usernames or passwords
• Server names, IP addresses etc.

Basically, anything that is not part of the SentryOne application.

Additionally, we designed our application telemetry system to anonymize information upon which SentryOne installation and client it is reporting. We have a consistent identifier that we refer to as LocationID and ClientID. This allows us to process the data (more on that later), but not determine from which customer it originated. We felt that putting this protection in place was important in an effort for our customers to be willing to enable the feature.

What data are we collecting?

The main reason for collecting data via application telemetry for SentryOne is to better understand which features our customers use the most. As well as how customers navigate through the client. The main reason for doing this is to make sure that customers can get to the features they need as quickly as possible. So what data are we collecting?

• Which features of SentryOne a client session is using.
• The navigation path through the SentryOne client.
• The EHO Score shown in the client.
• The number of Monitoring Services shown in the client navigator.

What does this data look like? An example of the data that we collect is as follows:

{

    "InstanceID": "5315207e-11d9-488f-bec5-b74018d44170",

    "SessionID": "bcbc6fb6-a271-46a5-86b6-7dcd0c8005b2",

    "TransactionID": null,

    "ClientID": "EE478EABE68B6957808C048BD97812B6",

    "LocationID": "5B57E565E2DC7C99FADCD8A6B042D344",

    "ProductID": "S1",

    "ProductVersion": "11.2.0.0",

    "TimestampUtc": "2017-04-27T14:13:10.4871637Z",

    "EventClassID": "e95861bb-cb49-4579-804c-6d0fafbed33c",

    "EventClassName": "Navigation",

    "EventPath": "WindowsPerformanceConsole\\TopSql\\ProcStats",

    "EventDuration": "N/A",

    "EventValue": "N/A",

    "EventPreviousValue": "N/A"

}

As you can see from here, we are looking at the SentryOne system.

How are we collecting the data?

Now that we have established what we are collecting, how is it sent, from and to where?

Collection and Sending

The telemetry engine resides in the SentryOne client, and each client installation will handle the data send process. This means that the SentryOne monitoring service plays no part in the application telemetry feature. The SentryOne client usage is where our interest lies, as that is the piece of the solution that our customers interact with.

The process for establishing if the client should send telemetry is quite simple, but best illustrated with the following diagram.

The process checks to see if an administrator has enabled Application Telemetry. The reason that we took this approach is to avoid different settings within a customer organization. Centralized control means if it is set to off, then subsequent client installations will honour this rather than prompt at install. The client will not attempt to send telemetry data to SentryOne when there is no internet connection.

Receiving and Storage

The client sends telemetry data from the SentryOne client to our systems via a secure messaging protocol to Azure Event Hubs. This ensures the data remains protected during transit with TLS. Once it reaches our back-end systems, we store the data in a combination of Azure Data Lake Store and CosmosDB. From here we will be using PowerBI to visualize the usage data that we are receiving. Once the data has arrived in the storage and processing layer, we are making use of the various Azure functionalities that exist to secure the data at rest.

This data is being stored in the Microsoft Azure regions in North America, specifically the US.

Insights from this data will influence the way we build our software. Monitoring feature usage over time will provide new navigation paths, and put different SentryOne SKUs in play.

Questions we ask

It is all well and good getting telemetry data, but this data is worth nothing if you are not asking it questions. So again, in the interests of transparency I will provide some insight into the processing that we are performing.

As I have mentioned previously, here at SentryOne we are looking to build better software. In light of this, it is important that we understand how our solutions are being used by our customers. I personally will be very interested in understanding three key elements of how our customers are using SentryOne:

1. Which features deliver the most value to our users?
2. What are the most common navigation paths through our software that our users take?
3. Which features are SentryOne users not leveraging?

With these three questions, it is possible for me to understand how I can make our software easier to use. Additionally, this allows me to focus my efforts on building surveys for customers that focused on the areas used most heavily, subsequently allowing me to focus engineering time on refining these to add value to all SentryOne customers. It allows me to identify and highlight existing features that need more awareness.

There will be more questions over time, some of which are likely to necessitate additional application telemetry to be added to the SentryOne solution. When this happens we will ensure that we are open with the community about what we are collecting and the processing that we do.

Enabling and Disabling Telemetry in SentryOne

Now that I have run through the details of the data it is time to explain how to enable/disable it.

New Installation or pre-11.2.x upgrade

The SentryOne installation process prompts users to enable or disable the application telemetry. This is the case for the first installation for the site. The reason for this is that the installer will check the SentryOne repository database to see if a configuration setting exists. In this scenario, there is no telemetry setting as it is not present in the version, or this is the first install. At this point the administrator can select whether to enable the feature or not.

Subsequent installations of the SentryOne client will not check with the user regarding enabling or disabling telemetry. Again, this is because the installer checks the database for the global configuration setting. What this means is that if it is supposed to be off, then no one will enable it by mistake on an install. Or, if it is supposed to be on, then they will pick up this setting in line with the rest of the team. The following section covers the details of enabling or disabling the feature.

Existing 11.2.x Installation

Once 11.2.x or above has been installed and configured, the option for the client to send telemetry has to be managed globally. The "Global Settings" node in the navigator controls this. Subsequently, each time a SentryOne client starts and checks the database to see if it can send telemetry it will pick up this globally configured setting and then proceed accordingly.

Call to Action

Hopefully you found this blog post useful in understanding the "what and why" for adding application telemetry to SentryOne. My objective was to reassure you that we are not collecting data about your servers, workloads, or any personal data. SentryOne transmits, stores, and processes the telemetry data securely and for good reason.

I would ask that if you can, that you do enable this feature. It is your chance to have a say in the way that we build our software. All that you, as a customer, have to do is check the box and use the software. You can then be sure in the knowledge that you are influencing the decisions that we make and helping to improve future versions of SentryOne for you and all our customers.