Processor Metrics

There are three processor-related metric sets:

• Hyper-V Hypervisor Logical Processor
• Hyper-V Hypervisor Root Virtual Processor
• Hyper-V Hypervisor Virtual Processor

The most important counter set you want to monitor is the Hyper-V Hypervisor Logical Processor. This counter set allows you to determine how much of the physical processor is being used. The Virtual Processor counter set only shows part of the Hyper-V Hypervisor Logical Processor, so you want to make sure to use the Logical Processor counter set. The most useful counters in this counter set are the following;

• %Guest Run
• %Hypervisor Run Time
• %Idle Run Time
• %Total Run Time

Note that there is one logical processor in Hyper-V that that has a larger workload than the other logical processors and that is LP0. This Logical Processor is where all interrupts in the system are directed to. If this LP hits 100% utilization there is likely an I/O bottleneck.

The Hyper-V Hypervisor Root Virtual Processor and Hyper-V Hypervisor Virtual Processor are just slices of the Logical Processor counter and can help you understand how much total CPU the root and guests are using on the system.

If you want to monitor the root CPU you should use the Processor counter set. According to Microsoft this counter suffers from some skew, but is useful because it gives an idea of how busy the root is. This is important because the root is involved in all I/O, meaning if the root CPU’s are saturated your whole system is likely saturated. In general you want to see the root CPU lower than 10% utilization and over 50% might indicate an issue.

Memory Counters

The following are the counter sets you want to monitor;

• Hyper-V Hypervisor Partition
• Hyper-V Hypervisor Root Partition
• Hyper-V VM Vid Partition
• Memory

The Hyper-v [Root] Partition counters determine how much memory the Hypervisor is managing and using on behalf of a VM which includes the guest address space but not all the memory in the worker process and VID partition. The Hyper-V Hypervisor [ROOT] Partition counter set indicates in the 1G GPA Pages and 2M GPA Pages metric whether or not a VM is using large memory pages which improves overall VM performance.

The Hyper-V VM Vid Partition counters account for the guest address space and any additional memory the VID needs to manage the VM.

The Memory counters are not specific to any VM and can be used to measure memory used to service I/O on behalf of a virtual machine.

The Partition Counter set also indicates in the Deposited Pages metric how much memory the hypervisor is using for managing the VM. The last counter that is interesting in the partition counter set is the Virtual Processors counter which tells you how many Virtual Processors a VM is configured to use.

The Hyper-V VM Vid Partition counters have two counters that you want to monitor. The Physical Pages Allocated is the total number of guest pages and VID pages needed to manage the VM. The Remote Physical Pages let you know on NUMA based systems if a VM is spanning multiple nodes. Note that Microsoft does not recommend such a configuration.

The Memory counter set allows you to monitor how much memory is consumed in the root. The root is responsible for managing all memory in Hyper-V. When a VM starts you will see the Available Bytes go down by at least the amount of memory given to the guest plus around another 16 – 64MB for guests meta data structures.

The following counters are useful to monitor:

• Available Bytes – Returns how much memory is remaining for guests.
• Pages / Sec – Measures memory pressure. This metric can tell you if processes are competing with each other for physical RAM.

Network Counters

You can use the network counters to monitor overall networking performance on the system. The most important thing to monitor is the total throughput counters to make sure the NICs are not getting saturated. There are four relevant counter sets:

• Network Interface
• Hyper-V Virtual Switch
• Hyper-V Legacy Network Adapter
• Hyper-V Virtual Network Adapter

The Network Interface measures the overall performance of physical device whereas the other counter sets represent the activity of the virtual switches and network adapters in the virtual machines. For the Network Interface the following are the most important counters to monitor:

• Bytes Total / Sec
• Offloaded Connections
• Packets / Sec
• Packets Outbound Errors
• Packets Receive Errors

The Hyper-V Virtual Switch counters are good to monitor because depending on your configured network, traffic might only exist on the virtual switch. This is the case with network traffic between guest to guest. The most important counters are:

• Bytes/Sec
• Packets/Sec

The Hyper-V Virtual Network Adapter and the Hyper-V Legacy Network Adapter counter sets allow you to measure performance of a virtual machine. These counter sets are named with the friendly name of the VM plus the name of the network adapter followed by two GUIDs. The GUIDs are the internal id of the VM and adapter.

If you assign a Legacy Network Adapter to a VM then the counter set to use is the Hyper-V Legacy Network Adapter. In general you should not use this network adapter type because it is not enlightened (see our prior ‘Best Practices’ article). It creates a lot of CPU load in the root, and is slower than the Virtual Network Adapter. However, you need the Legacy Network Adapter to get a VM working before installing Integration Services. Once your VM is working with Integration Services you should use the Network Adapter and the “Hyper-V Virtual Network Adapter” counter set. Note that both Windows Server 2008 and Windows Server 2008 R2 have integration Services pre-installed.

The Hyper-V Legacy Network Adapter counters to monitor are:

• Bytes Dropped
• Bytes Sent / Sec
• Bytes Received / Sec

In the Hyper-V Virtual Network Adapter you should monitor;

• Bytes / Sec
• Packets / Sec

Storage Counters

The storage counters are used to monitor overall disk performance on the system and each VM. There are:

• Physical Disk
• Hyper-V Virtual Storage Device
• Hyper-V Virtual IDE Controller

The Physical Disk counter set will give overall storage performance on the system. The other two metrics are for VM’s only. Of the Physical Disk counter set there are three metrics that are useful to monitor. The Current Disk Queue Length measures how busy the drives are. This counter should be around two per drive, depending on your drive configuration. A queue length of 32 might indicate this disk is the bottleneck in your system.

Another useful counter is the Disk Bytes / Sec. You can typically expect to see about 10MB/sec per drive; a safe number for most drives. For random workloads, look at the counter Disk Transfers/Sec. Expect to see about 100 I/O’s per second (IOPs) for each drive. There are drives that can do much better, like 180 IOPs and some that perform worse.

Because of the way storage works in Hyper-V, there are two Hyper-V storage counter sets. Hyper-V provides two virtual storage buses for virtual machines. One IDE and one SCSI. The Virtual IDE counters show up in the Hyper-V Virtual IDE Controller counter set, unless the Integration Services are loaded. In that case you’ll see the activity for both virtual IDE and SCSI in the Hyper-V Virtual Storage Device counter set. If you don’t have integration services installed, only the Hyper-V Virtual IDE Controller will show the VM disk activity.

It wouldn’t hurt to monitor all the counters of both Hyper-V Virtual IDE Controller and Hyper-V Virtual Storage Device. Below are the basic metrics for these two counter sets:

For Hyper-V Virtual IDE Controller there are:

• Read Bytes / Sec
• Write Bytes / Sec
• Read Sectors / Sec
• Write Sectors / Sec

For Hyper-V Virtual Storage Device there are:

• Error Count
• Flush Count
• Read Bytes / Sec
• Write Bytes / Sec
• Read Count
• Write Count

That’s it for our discussion about the Hyper-V metrics that can be used to help you understand what the system is doing. In the table below, we’ll list all the counters:

• \Hyper-V Virtual Machine Health Summary \*
• \Hyper-V Hypervisor\*
• \Processor(_Total)\*
• \Hyper-V Hypervisor Logical Processor(*)\%Guest Run
• \Hyper-V Hypervisor Logical Processor(*)\%Hypervisor Run Time
• \Hyper-V Hypervisor Logical Processor(*)\%Idle Run Time
• \Hyper-V Hypervisor Logical Processor(*)\%Total Run Time
• \Hyper-V Hypervisor Root Virtual Processor (*)\%Guest Run
• \Hyper-V Hypervisor Root Virtual Processor (*)\%Hypervisor Run Time
• \Hyper-V Hypervisor Root Virtual Processor (*)\%Idle Run Time
• \Hyper-V Hypervisor Root Virtual Processor (*)\%Total Run Time
• \Hyper-V Hypervisor Virtual Processor (_Total)\*
• \Memory\Pages / Sec
• \Memory\Available Bytes
• \Hyper-V Hypervisor Partition(*)\2M GPA Pages
• \Hyper-V Hypervisor Partition(*)\Deposited Pages
• \Hyper-V Hypervisor Partition(*)\Virtual Processors
• \Hyper-V Hypervisor Root Partition(*)\*
• \Hyper-V VM Vid Partition(*)\Physical Pages Allocated
• \Hyper-V VM Vid Partition(*)\Remote Physical Pages
• \Network Interface(*)\*
• \Hyper-V Virtual Switch(*)\*
• \Hyper-V Legacy Network Adapter(*)\*
• \Hyper-V Virtual Network Adapter(*)\*
• \Physical Disk(*)\Current Disk Queue Length
• \Physical Disk(*)\Disk Bytes / sec
• \Physical Disk(*)\Disk Transfers/sec
• \Hyper-V Virtual Storage Device(*)\*
• \Hyper-V Virtual IDE Controller(*)\*

(_Total) means only the total should be collected.

(*) means collect all counters.

\* means collect all the counters in the counter set.

\<name> means only collect that counter.

Custom Monitor

We have provided a custom monitor in VBScript that uses remote WMI and the available performance counters. The most important counters that we discussed in this article are included in this monitor. You can download this script from GitHub.

Scheduling Custom Monitors with Monitis and PowerShell

In last article, we talked about creating custom monitor updater commands that would run a script and upload the values to Monitis.  These custom updaters make it a snap to convert any PowerShell script to a Monitis monitor.  Simply plug the script in and start using it’s more logical name: Update-YourMonitor.

Add-MonitisCustomMonitorUpdater -Name OutOfPaper -ScriptBlock {
    Get-Counter "\Print Queue(*)\Out of Paper Errors" |
        Select-Object -ExpandProperty CounterSamples |
        Where-Object { $_.InstanceName -ne '_total' -and $_.CookedValue -gt 0} |
        Select-Object @{
            Label = "Printer"
            Expression={$_.InstanceName}
        }, @{
            Label = "OutOfPaperErrors"
            Expression={$_.CookedValue}
        }
} -Property Printer, OutOfPaperErrors

Once you’ve run this code, you can simply do:

Import-Module Monitis
Update-MonitisOutOfPaper

To update the values.

You can simply schedule the update by scheduling the two lines.  To click thru it, run TaskSched.msc, then create a new task.  Run PowerShell.exe as the task, and use the arguments –command Import-Module Monitis;Update-MonitisOutOfPaper.

You can also schedule the task using a script, but not one built into the box.

All throughout this article series, you’ve been focused on using the Monitis module and in-the-box capabilities of Powershell, but there are hundreds of community modules to choose from.  You can easily leverage these community modules in your own custom monitors.

In one very large collection of these modules, called the PowerShellPack (http://code.msdn.microsoft.com/PowerShellPack), there is a task scheduler module you can use to schedule the monitor.

Like every command in PowerShell and every command in the monitis module, you can find out about the commands in this module with Get-Command and Get-Help.

If you want to schedule a task that runs your custom monitor with this module, it will look something like this:

New-Task |
    Add-TaskTrigger -OnBoot -Repeat "0:0:10" |
    Add-TaskAction -Script {
        Import-Module Monitis
        Update-OutOfPaper
    } |
    Register-ScheduledTask -name "CheckForOutOfPaper" -Credential (Get-Credential)

Register-ScheduledTask -name “CheckForOutOfPaper” -Credential (Get-Credential)

New task creates a new task

Add-TaskTrigger gives it a new trigger, on boot and repeating every 10 minutes thereafter.

Add-TaskAction runs the monitor, importing the module and running Update-OutOfPapeer.

Register-ScheduledTask registers the task, and –Credential registers the credential the task will be run under.

That’s it.  Now you can create a monitor to hold any piece of information, collect that information with PowerShell, and update it on a regular basis.  Thanks to your notification rule, your printer lackey will automatically head off to keep your printers filled, and you can monitor your friends Facebook updates while Monitis and PowerShell monitor your data center.

Monitis is an incredible monitoring platform with the flexibility to monitor whatever you want.  PowerShell is an incredible task management and automation platform that can do anything you need it to.  Together, you truly can Monitis anything with Monitis.

See also:

Monitor Everything with Monitis – And do it easily with PowerShell – Part 1

Part 2: Managing External Monitors with Monitis and PowerShell

Part 3: Mining External Monitor Results with Monitis and PowerShell

Part 4: Monitoring Web Applications with Monitis

Part 5: Testing Web Content with Monitis, Excel, and PowerShell

Part 6: Monitoring Anything with a Custom Monitor

Part 7:  Hardware Inventory with Monitis Custom Monitors

Part 8: Monitoring Logons with Monitis

Part 9: Monitoring Connections to Shared Folders with Monitis and Custom Monitors

Part 10: Inventory Windows Installations with Monitis and PowerShell

Part 11: Monitoring Removable Disks on Many Computers with Monitis and PowerShell

Part 12: Monitoring Event Logs and Using Monitis Notifications

Part 13: Monitoring out of Paper with Monitis (Printer Monitoring)

As we have discussed on the Monitis blog virtualization is one of the hottest IT subjects today.

One leader in this field is Citrix — with the XenServer product family. XenServer is the most known commercial implementation of of the open source Xen hypervisor. By the way, you should pronounced Xen like “zen.” It comes from a Greek word meaning “guest.”

In the IT world, Xen originated as a research project, led by Ian Pratt at England’s Cambridge University. It was first developed by XenSource Inc., and after acquisition, the work was continued by Citrix. Currently, you can choose between 4 editions of XenServer:

• XenServer [Free Edition]
• XenServer Advanced Edition
• XenServer Enterprise Edition
• XenServer Platinum Edition

Other Citrix “Xen” technology, like XenApp or XenDesktop, are not related to the original Xen project.

In 2009, Citrix decided to open the source of XenServer, and the company created XCP (Xen Cloud Platform). In the first phase, XCP was a whole Linux distribution. Recently, the initiative has been extended to provide XAPI toolstack on standard Linux distribution, in the first place for Debian and Ubuntu. What this means is that you can try XAPI on your home machine.

Monitoring

In the article below, we’ll learn how to apply daemon exporting XenServer metrics to Monitis. This is only proof of concept, but it can be easily modified to do exactly what you want. (This is the power you get using Monitis Open API).

XenServer/XCP

There is a dedicated XenServer monitoring tool included in all paid editions. You can also find some third-party tools. Obviously, we are interested in how we can monitor XenServer and XCP with Monitis. One possible way is to enable SNMP on XenServer and treat it as a standard Linux in Monitis. After all, XenServer and XCP dom0 are based on CentOS. Another method that we are going to give an overview of today is XenServer/XCP-specific. XenServer enables you to download, host and store all VM statistic in a  RRD (Round Robin Database) using HTTP handler. You can find some information on how to use XenServer RRD data in two documents from the Citrix Community page (Click here and here). In the second document, you’ll see that there are two examples of Python scripts that we are going to use for the base to our daemon updating XenServer monitors in Monitis. Both scripts use the python “rrd_update” module. You can also find on this on the Citrix page or grab it from github.

Monitis

Monitis has dedicated libraries in many programing languages. You need to use python to export XenServer RRD; therefore to update results on Monitis, we are going to use the python library, too. The code can be obtained from our github repository. Please remember that you need the elementtree module, which is not a part of standard python distribution. Plus, you have to get it from the official website or obtain it from your distribution repository. There is a note about it in the code, but not in the original article.

xen2monitis

Having these two weapons, we are now ready to attack. First, there is the example code in our git repository that you can install on XenServer or Linux distribution with XCP. Alternately, you can choose another server with a connection to both Monitis and the XenServer that you want to monitor. However, installing our daemon on XenServer/XCP seems to be the best choice.

Here are some easy directions:

1.) Create the Xen2Monitis default directory:

mkdir /var/local/lib/xen2monitis

2.) Download the xen2monitis code which is part of Monitis-Linux-Scripts:

git clone https://github.com/monitisexchange/Monitis-Linux-Scripts.git

3.) Copy Xen2Monits file to default location:

cp  Monitis-Linux-Scripts.git/Xen2Monits/* /var/local/lib/xen2monitis

4.) Go to created directory:

cd /var/local/lib/xen2monitis

5.) Obtain the elementtree module:
- You can get the source code:

wget http://effbot.org/media/downloads/elementtree-1.2.6-20050316.tar.gz

- Or you can get a dedicated package for your distribution. If you plan to use XenServer, the elementtree is already installed there.

6.) Install the elementtree (if not already done by your distribution)
- for source code (the last command as root):

tar -xzf elementtree-1.2.6-20050316.tar.gz
cd elementtree-1.2.6-20050316
python setup.py install

7.) Next, add your API Key and Secret to a file. Please separate them with “,”.The default ‘secret file’ is  /usr/local/xen2monitis/secret

8.) Add the xenserver root password to your secret file, so it should be similar to:

5DQQ0460S18ERNRNRUI7M905N1, 7MA9XJNDL3KAMBAKFR9981N04J, secret_password

9.) Now you can start the daemon:

/usr/local/lib/xen2monitis/xen2monitis.py start

10.) You need to wait few minutes. You can check if your script works without any problems in /var/log/xs2monitis.log.

11.) After a few minutes, you should have a bunch of new graphs that you can add to the Dashboard by Monitors->Custom Monitor menu.

Monitor Name Conventions

When you look at newly created monitor names, you might be a bit confused. The names tend to be very long and have a strange set of strings separated by ‘-’, for example:

lisbon – cpu1 (HOST: 0c1bb847-d867-75ff-27e0-7b468f1385f2)

Such name conventions were chosen to ensure the uniqueness of each monitor. Imagine that you have two VMs called “Win7″. Daemon would not be able to distinguish between them and would save all data into one monitor! To address it, XenServer UUID (universally unique identifier), what looks like set of strings separated by “-”, is added to each title.

Daemon Modification

Using xen2monitis.py script, please remember that there are some values you can define at the beginning of the file, for example:

• where is the secret file (default /usr/local/lib/xen2monitis/secret)
• where is the lock file (default /var/run/xen2monitis.pid)
• where is the log file (default /var/log/xen2monitis.log)
• the url to pool master (default http://localhost)

You might prefer not to use daemon, but, instead, call up script from cron job. It’s not a big problem. You need to copy the main loop, start after line 115 to separate the file and convert it.

We hope this article showing you how to monitor your XenServer with Monitis makes your IT life a lot easier. Please share some tips of your own!

Need for Speed?

With its first public release in 2004, NGINX now is the second most used web server in the world. This is quite impressive considering NGINX was born among several big players including Apache, Microsoft IIS, and other popular web server software. By sharing best practices and a few tips today, we would like to bring NGINX’s new blood up to speed. Even if you come from the Apache world, you will learn how to build an Apache server that married NGINX, an extremely fast and high-performance web server. Let’s go:

1. Make sure your OS is well supported by NGINX:
   In particular, double-check NGINX supports the event polling system call under your operating system. This is important because the beauty of NGINX is its asynchronous architecture for event handling. (This design is used to address the C10k problem, literally, to handle 10,000 clients simultaneously.) Under Windows, however, NGINX uses select() for event polling. While select() is non-blocking, select() can limit a process to have only 1024 open file descriptors. Therefore if you expect your NGINX server to be high-performance, Windows might not be your best environment to work with.
2. Improve disk I/O if possible:
   • You can turn off NGINX access logs and use the client site scripts such as Google Analytics to collect statistics. This way NGINX will not record every single request to your log file, reducing disk I/O significantly. It is not recommended you also turn off the error logs. After all, errors are important information, and they don’t occur that often anyway.
   • Configure “open file cache” if your system supports it. Note that the actual file is not cached, but the pointer to the file instead. That is, an inode in your filesystem. Why caching inodes can speed up disk I/O? Think of inodes as a back-of-the-book index, which can help you look up information quickly.
3. Improve network I/O if possible:
   Turn on the “Gzip” module so you can compress data before sending them over the network. Nowadays the CPUs are so fast that you can even tell NGINX to use level 6 compression just to save a bit bandwidth. If you are worried about CPU hogs, use level 1 compression since it has the best compression to time ratio.
4. Tweak the HTTP keep alive setting:
   NGINX is so memory efficient that it is possible to use only ~2.5 M to maintain 10k HTTP keep-alive connections. (too scary to be true, isn’t it?) This gives systemadmin plenty of room to lift the “keep alive” limit. The idea is to have as many keep alive connections as possible, and set the keep alive timeout with a larger number as well. Why not setting them both to infinite? It is because you want to think about malicious users. Attackers can abuse your generous setting and possibly exhaust your server memory.
5. Configure NGINX as a load balancer:
   This is easy to configure. The example below shows how NGINX uses  round-robin and client IP to achive load-balancing across backend servers.
   
6. Configure NGINX as a reverse proxy:
   Combining Apache with Nginx is a very popular setup. The idea is to have NGINX serve static files, and let Apache handle dynamic content.
   
7. Only Enable the NGINX modules you need:
   In other words, turn off the modules you are not using. Keep your NGINX simple and small. Not only you can save memory, this can also improves overall security. For example, if you are not using NGINX as a load balancer, disable the “Upstream” module.
8. Avoid NGINX configuration pitfalls:
   First of all, NGINX is not Apache. If you are new to NGINX’s configuration, you might want to check out the wiki first to have a glimpse of NGINX’s rich configuration options. The following are some quick tips for configuring NGINX:
   • Factor out “root” outside of a location block.
   • Factor out “index” to the http block.
   • Avoid using “if” unless you really have to.
   • Use “try_files” if you want to check if a file exists.
   • Use “try_files” first to serve static content on dynamic pages.
   • Use “script_filename” to avoid hardcoding absolute paths.
   • Use ” $request_uri” to avoid using regular expression
   • Use “rewrite” and “http://” to force absolute path.
   • Use “map” to customize your key-value pairs.
   • Use “stub_status” to monitor your server.
   • You can combine http and https block in NGINX
   • Set “proxy_connect_timeout” higher under heavy load.
   • Clear browser cache before testing the new configuration.

Conclusion:

NGINX is the king of serving static contents. Believe it or not, you don’t need to tweak much to optimize your NGINX server. The default configuration works quite well. It gives you a sense of “you don’t pay for what you don’t use.” If you do need to customize your NGINX, please follow NGINX best practices and, remember NGINX’s underlying principle: lean and fast.

We hope you enjoy the article. As a bonus, Monitis® offers enterprise-level solutions to monitor your NGINX servers. You can get your free signup here.

—

Useful Links:
Integrate NGINX Monitoring into Monitis.com:
http://blog.monitis.com/index.php/2012/01/23/integrate-nginx-monitoring-into-monitis-com/
Monitis Free Registration:
https://www.monitis.com/free_signup.jsp

Windows Hyper-V has three main components, each of which can be monitored; the virtstack, devices, and the hypervisor. When Windows 2008 boots the system it launches the virtstack and hypervisor. The virtstack handles the emulated devices, manages virtual machines, services I/O, etc. The Hypervisor schedules the virtual processors, manages interrupts, services timers, and controls other chip-level functions.

The root partition in Hyper-V provides the monitoring information via WMI and performance counters. Although you can monitor a virtual machine from within the running guest operating system, you only have access to the metric provided by that specific OS. One thing to keep in mind is that when querying the metric %Processor, it reports on the virtual processor(s) used in the guest, not the physical processor of the host system.

So to monitor Hyper-V and all virtual machines, you should monitor the root. Now let’s take a look at what you should monitor. We’ll cover the major resources and what to typically monitor to provide a good starting point. The top level metrics to monitor are the following:

Overall health:

• Hyper-V Virtual Machine Health Summary
• Hyper-V Hypervisor

Processor:

• Processor
• Hyper-V Hypervisor Logical Processor
• Hyper-V Hypervisor Root Virtual Processor
• Hyper-V Hypervisor Virtual Processor

Memory:

• Memory
• Hyper-V Hypervisor Partition
• Hyper-V Root Partition
• Hyper-V VM Vid Partition

Networking:

• Network Interface
• Hyper-V Virtual Switch
• Hyper-V Legacy Network Adapter
• Hyper-V Virtual Network Adapter

Storage:

• Physical Disk
• Hyper-V Virtual Storage Device
• Hyper-V Virtual IDE Controller

In this first part of the ‘monitoring explained’ articles, we’ll look at the Overall Health Metrics set of performance counters and how to use them. The other sets are discussed in Part 2.

Overall Health Metrics

There are two counter sets you can use to get an overall picture of the system:

1. Hyper-V Virtual Machine Health Summary which has two counters; “Health Ok” and “Health Critical”. Critical means some resource has been exhausted or another unrecoverable error has occurred. If you see “Health Critical” you should take action to figure out what has happened.

2. Hyper-V Hypervisor. This set has the Logical Processor counter, the number of virtual machines running is measured by the Partitions counter, and the total number of Virtual Processors is provided by the Virtual Processors counter. The logical processors are important because this is where all the work is done and are a representation of the physical processor. Microsoft recommends that you do not exceed an 8:1 ratio for the the Virtual Processor to Logical Processor ratio.

The Total Pages counter indicates how much memory the Hypervisor is using to manage the virtual machine. This counter does not capture all the overhead because another component called the Virtual Interface Driver (VID) also has overhead to manage partitions. Note that in Windows 2008 the counter Hyper-V VID Partition does not work, but it does in Windows Server 2008 R2.

In our next article in this series we’ll discuss the Processor, Memory, and Storage counter sets in detail. We will also provide a custom monitor for Hyper-V that utilizes most of the metrics that are discussed in this series.

See also Monitoring Hyper-V With PowerShell on Monitis

2. Using Tomcat With a Database – Performance Optimization Tips
Do you use Apache Tomcat with a database? Here are 6 tips for optimizing performance for that combination. For example, it’s important to determine the cause of database connection timeouts. One of the common causes is garbage collection. Tip 2 describes how this happens and what can be done to improve performance.

3. Cassandra and Monitis
Cassandra is one of the more popular highly scalable NoSQL databases. If you work with Cassandra, this article will show you how to use Monitis to keep track of the performance of your Cassandra cluster. Code is provided on the Monitis Github page that shows you how many Cassandra nodes you have up and running as well as read and write latencies. You can set alerts on these metrics.

4. M3 code refactor & DBI support
M3, the Monitis Monitor Manager, was first described in this article. Monitis has recently decided to make M3 pluggable to make it easy to add execution and parsing plugins. This posts introduces some of the things you can do with this pluggability. It describes Database integration with Monitis using M3. There are many benefits monitoring your database health in Monitis. This post shows how to make a query and monitor its results. Monitis is anxious to hear your input on how best to use M3 to meet your monitoring needs.

Pluggable M3 (Monitis Monitor Manager) Framework

Who needs an introduction about M3? – Perhaps no one!
After gaining some reputation with M3, providing extra-easy integration of any monitor into Monitis it was time to take it to the next level.

Generally speaking, the work flow of M3 was described in detail in this article.

After some thought and design, we’ve decided it’d be best if M3 was pluggable. Pluggable in terms of being able to easily add execution and parsing plugins.
The interface and behavior of M3 stayed exactly the same, however now it is much easier to obtain data from any source and parse the data the way you want it.
Saying that, it was time to put the new design for a test. We tried to integrate the DBI support into M3.
Guess what – it was much easier than expected!

The general idea of DB integration with Monitis

DB (as in Database) integration with Monitis has shown to be invaluable. Being able to extract application related data from a DB and load it to Monitis means you can graph anything going on in your database related application. Monitoring your DB is an integral part of server monitoring in today’s world.

Be it maximum number of users at peak hours, gathering statistics about record usage or just checking if your DB is serving requests quickly enough – graphing performance and uptime. Your imagination is the limit – considering your application is DB related of course.

How it’s done – the technicalities beyond

The sharp-eyed among you would notice that M3 got beefed up with a directory named “Execution” and a directory named “Parsing”.
These directories hold the plugins M3 uses for each.

All that I had to do to add DBI support is to create a new execution plugin called “DBI.pm” and fill it up with code from the Monitis DBI module which already existed.

And that my friends, that, was easy. Mission accomplished – Pluggable M3 eases the job for us.

Another way to integrate MySQL

MySQL could have been integrated previously into M3 by executing mysql and feeding it with a query, having something similar to:

 <exectemplate>echo "select count(*) from mysql.user;" | mysql</exectemplate> 

This form of execution will invoke the ‘mysql’ executable, feeding it the above query. Execution of this will result in the response of the query being returned with some other useless output.

Please avoid this form of execution, as now DBI support is fully integrated into M3. In the next paragraph I’ll show how it should be done.

Lets fetch

While conveying ideas about M3, I decided to prefer to provide very simple examples of operation, however, I encourage you, the end user to let your imagination go wild, adding interesting counters which show how your application really perform and behave.

In this example we’ll just count the number of users in an installed MySQL database. An example anyone should be able to imitate and then scale to his own need.

We’ll examine the following example. I suggest opening the mentioned link in a new window while continuing the read.
Looking at this example it is quite straight-forward to understand. The query to execute is over there, together with some other parameters which are mandatory to connect to a database, such as the database name, user name, password, etc.

An explanation should be provided however for the odd db_statistics parameter. Those of you who have read the HTTP extraction article about M3 would have probably remembered the http_statistics parameter. Then, the db_statistics one does more or less the same, providing some meta counters for the query:

1. How long did the query take to process (in seconds)
2. The success/failure of the query (1 for success, 0 for failure)

I encourage to do the parsing of query results, or actually the lack-of-parsing of query results in a way that takes the whole output returns from the query, hence the regular expression of (.*).

If your query returns more than what you need, please tune your query better.
Test your execution with:

 # ./DryRun.pl dbi_sample.xml DB: 'root@DBI:mysql:mysql:localhost', Query: 'select count(*) from mysql.user;' at MonitisMonitorManager.pm line 223 Matched '(.*)'=>'6' in '6' at MonitisMonitorManager.pm line 237 OK This is a dry run, data for monitor 'Sample MySQL Monitor' was not really updated. at ./DryRun.pl line 18 

It should work. Then you can work your query to fit your own needs.

Next in line

We’re looking for some more new ideas to add as plugins for M3. If you have any idea about any plugin (execution or parsing plugin) that you would like to see in M3 – we want to hear you!

Integrating software into the powerful Monitis has never been easier, now that you have M3!

Monitis can monitor anything! Follow us also on Twitter and GitHub.

Cassandra is a robust and highly scalable NoSQL datastore that usually consists of multiple nodes spread out across multiple datacenters.  If you are the system administrator for a large Cassandra deployment then you might be curious as to how your cluster is doing.  In fact your job probably depends on it!  So how can you combine a great service like Monitis with Cassandra to make sure you cluster is buzzing along smoothly?

git clone git://github.com/monitisexchange/Monitis-Linux-Scripts.git

cd Monitis-Linux-Scripts/cassandra

./setup_monitors

./send_data

The Internet is a great resource, but it is sometimes too much to wade through hundreds of web pages looking for tips about using Tomcat with a database. So we at Monitis have done it for you! Below are some of the more useful tips found in various places on the Internet.

Tip #1 – Don’t Ignore Database Performance

Webapps typically use databases. So, you have to tune the webapp along with the database if you want optimal performance. Take our advice. Performance test the database and define an appropriate database connection pool (maxActive, maxIdle, and maxWait in the Resource element).

Tip #2 – Determine the Cause of Database Connection Timeouts

Simply increasing maxWait in the resource element may not be the best way to deal with database connection timeouts. The timeouts may be the result of garbage collection.

Garbage collection requires that all running processes in the Java virtual machine be temporarily suspended. If garbage collection takes too long, the database connections will time out. The timeout here is not because of some problem in the application. It is simply because garbage collection is taking too long.

The more frequently garbage is collected, the less garbage there will be to collect each time. Configure garbage collection so it does not take more than one second.

Tip #3 – Share Database Connections

If every application opens and closes its own database connections, all applications run slower. Instead, if applications share connections by returning them to a common pool rather than closing them, all applications run faster.

Tomcat can be configured for this type of connection pooling. Simply set maxActive and maxIdle to appropriate values in the resource element.

Tip #4 – Return Database Connections to the Pool Properly

Do not close database connections twice. When using connection pooling, the first time you close you return the connection to the pool, and the second time you close, you could inadvertently close the connection after another application has started using it. If you are experiencing “Connection Closed” exceptions, this may be the cause. So, don’t do that!

Tip #5 – Configure Your Database as a Global Naming Resource or For a Single Context

Identify the Java Database Connectivity (JDBC) database in a resource element in the context.xml file if you want the resource to be available to one application only. Identify the JDBC resource in a resource element in the server.xml file if you want the resource to be available to all applications.

If multiple applications need access to a database, the database resource can be created globally in the server.xml file or each application can create its own resource in its own context.xml file. You need to decide which is best.

Making databases available to one application only (in context.xml) creates the resource when the application starts. Making databases available globally (in server.xml) creates the resource when the server starts, which means restarting the server every time you make changes to the resource configuration.

Configuring the database resource in the context.xml file once for each application is recommended because it makes the application more portable.

If different applications have different access levels and those access levels are determined by the database user id, then the database resource must be configured in each application’s context.xml file.

Tip #6 – Do Not Use the JDBC-ODBC Bridge

The JDBC-ODBC (Open Database Connectivity) Bridge bundled with Tomcat’s Java Development Kit (JDK) was never intended for a production environment. Use a third-party driver instead.

Using Tomcat with a database is easy. All you have to know are some good tips to keep your database performing trouble-free. Monitis is happy to provide you with this knowledge and advice. Read more about Tomcat here!

More about Apache Tomcat performance optimization tips here.