Monitoring VMware Cloud on AWS vCenter alarms within vRealize Log Insight Cloud

vRealize Log Insight Cloud (vRLIC) gives us unified visibility across public and private clouds through robust log aggregation, analytics and faster root cause determination. The great news is that it is also included as part of your subscriptions to VMware Cloud on AWS (VMC) and as of VMworld Europe 2019 also now includes additional features and functionality:

Check out the official blog article here

With the core version, VMC customers now get access to real-time reporting which is what we are going to look at today. In a previous post, I talked using creating VMC vCenter alarms and setting notifications for specific events. The event we were particularly interested in was if the VSAN datastore reaches 70% utilisation because at 75% a new host will be added to ensure we stay within SLA. In the example, we used an alert that would trigger if the datastore was less than 100% utilised as this would ensure the alert would always trigger. We are now going to use vRLIC to query for the alert and then send us a notification once it has been triggered.

vRLIC is automatically configured to ingest logs for VMC and can be accessed via the Cloud Services Portal so there is nothing that you need to do to start using it, simply launch the application:

The initial landing page gives us a great overview of recent alerts and event observations over the last hour. It is definitely worth spending some time with vRLIC to see the level of information and default alerts that are available to VMC customers:

I've re-created the vCenter alert that triggers when VSAN Datastore Usage is below 100 percent just to ensure the log is sent instantly to vRLIC:

If we explore the logs and query for the alert name VSAN Datastore Usage is below 100 percent with a timeframe of the last ten minutes then we can see the triggered alert. We know this is the alert because we can see it change its state from Gray to Red:

Now that we have the query needed we can click on the save icon:

Give the query a suitable name and description and click Save:

Once we save the query we can click on the alert icon to create an alert based on the query:

Give the alert a suitable name and description and click Save:

The Alert Definition screen will appear which will allow you to customise the alert. Remember to add the Email address where you would like the alert to be sent, set the trigger to evaluate on every match and enable it before clicking on the save icon:

It's also worth sending a test alert to ensure you receive the notification:

Hopefully, if everything is set up correctly next time the alert is triggered you should receive a notification via email and also see it in the Recent Alerts:

With the example above we are triggering an email notification when we see any log that has been ingested and contains the text, VSAN Datastore Usage is below 100 percent. This is not ideal because it will also trigger the notification when any changes to the alarm are made i.e. reset to green or disabling and re-enabling. I tried testing this on alarm name as well as the text gray to red which is sent when the state of the alarm changes but during testing, I noticed that this was not always sent on certain alarm configuration changes which I have fed back to the BU and will be addressed in the future. I don't envisage these changes being made regularly in customer environments so it should not cause an influx of emails.

A point that I would like to highlight is that vRLIC currently runs out of one of the US AWS regions so if there are issues with logs residing outside of the UK/EU then please get in touch and I will continue to raise this internally.