Using the Free Tier Data Consumption Tracker for larger environments[Update 11/29/2017: The query example in this blog post has been updated to the new query language for Log Analytics]
In March of this year, Wei released a great pre-built solution which provides visualization for how much data you are sending to a free-tier OMS workspace. If you haven’t already downloaded this solution or checked it out yet and you are using OMS you really should check this one out. In multiple meetings with customers the question was raised to see if it’s possible to use this solution for non-free tier workspaces. Specifically this was for environments where the client is budgeting the amount of data that they are sending but it is beyond 500 MB (potentially 1GB, 5GB, 10GB, etc). This blog post will explain how to utilize Wei’s solution for workspaces where you want to use larger amounts of data than is available in the free OMS workspace tier.
How does the solution visualize this data?
Wei’s solution includes a top level visualization so you can tell at a glance what is taking up the most majority of the data in this workspace (Security, LogManagement and then NetworkMonitoring in this example).
This top level dashboard expands into the views you see below which make it easy to track down your data usage. From the example below we can tell at a glance that security events from the security solution on a specific system are utilizing most of the data in this particular workspace.
Editing the solution
Because this solution was written in the view designer, we have the ability to edit these views both to see how they function and to change how they function. This is done by opening the solution and using the Edit option in the top left corner.
The math of the solution
An example query from the solution is listed below:
Usage | where QuantityUnit == “MBytes” and iff(isnotnull(toint(IsBillable)), IsBillable == true, IsBillable == “true”) == true and TimeGenerated > ago(24h) | extend UsedFreeTierPercent = (Quantity * 0.2) | summarize TotalUsedFreeTierPercent = sum(UsedFreeTierPercent) by Solution
The key piece in the query above is the .2 value in the “mul(Quantity,0.2)” section. Wei is using a very quick and logical way to move this into a percentage. He’s taking the MB value (500) and multiplying it by .2 to provide a value as a percent (500*.2=100) or (TotalMBData*Value=100). Our constant is 100 with the two variables being how much data (in MB) you want to have in the workspace and the value is what we will get from the equation.
This is the key to making changes to the solution. See the samples below for the appropriate values.
- 500 MB (1/2 GB), Value = .2 (500*.02=100)
- 1000 MB (1 GB), value = .1 (1000*.1=100)
- 10000 MB (10 GB), value = .1 (10000*.01=100)
- 50000 MB (50 GB), value = .05 (50000*.002=100)
Changing the solution
Once you have determined what the Value is for your data requirements, the change is pretty simple. You edit each of the queries contained in the solution and replace the value of 0.2 with the value which you calculated. Example for 10 GB:
Usage | where QuantityUnit == “MBytes” and iff(isnotnull(toint(IsBillable)), IsBillable == true, IsBillable == “true”) == true and TimeGenerated > ago(24h) | extend UsedFreeTierPercent = (Quantity * 0.01) | summarize TotalUsedFreeTierPercent = sum(UsedFreeTierPercent) by Solution
Note: These changes need to occur on both the donut query and the list query for each of the four dashboards which were created and for the title dashboard. You will also most likely want to change the summary dashboard to explain the new values which you are representing in the solution.
Summary: The free tier data consumption tracker solution can be easily changed to reflect the amount of data which you want to collect in your workspace if you are using more data than is available in the free tier. This approach makes it easy to see how much data you are ingesting compared to what you budgeted for the workspace.