Ruby’s various versions and implementations have different performance profiles also and this must be taken into consideration before a decision can be made definitively about Ruby’s overall performance.  Benchmarking is a flawed means of determining the real performance of a tool because it relies too heavily on the system configuration on which the tests are ran and what activity is occurring on the system during test runs.

Benchmarks and Ruby-haters not-withstanding you can get good performance from your Ruby applications by following a few simple guidelines.  Some of the guidelines are just good programming techniques in general and others are specific to addressing issues in the Ruby language and it’s associated libraries.  Performance gains achievable through the implementation of these tips and tricks can range from minimal to breathtaking.

1. Don’t make your code do unnecessary work

For experienced developers this may seem a no-brainer but even old hat coders have been known to slap a class together that might not exactly be the most efficient.  Write just enough code to do the job and do it correctly.  Don’t over complicate your algorithms and be sure not to do unnecessary things such as looping more than required.

2. Avoid nesting loops more than three levels deep

Nesting not only slows you code down but also can make maintenance of the codebase difficult if it goes too many levels deep.  Limiting nesting of loops and functions to three levels or less is a good rule of thumb to keep your code performant.

3. Avoid unnecessary variable assignments

Many people, especially new programmers, tend to assign variables more than necessary.  A great example is when someone defines a variable to store a return value and then returns that variable; just return the value directly.

4. Reduce usage of disk I/O

Disk I/O is one of the biggest bottlenecks remaining in computing.  Read/write operations to disk are extremely slow and it’s best to avoid using the disk whenever possible.  Many people are now using software such as memcached which allows data to be stored in memory and only periodically written to disk.  The speed increases when using a memory caching system are tremendous.

5. Use Ruby Enterprise Edition

Ruby Enterprise edition provides up to 33% lower memory usage.  In order to take advantage of these performance gains though you must be sure to program according to their guidelines.

6. Avoid method calls as much as possible

Method calls are very expensive operations and should be avoided when necessary.

7. Use efficient Ruby idioms

Program into the language rather than in the language.  Performance will suffer if you try to write your Ruby code like you would PHP, Perl, or any other language.  Learn the Ruby way of doing things.

8. Use interpolated strings instead of concatenated strings

Interpolated strings are faster than concatenated strings in almost all interpreted languages; Ruby is no exception.  Using the << operator makes a method call which and method calls should be avoided when possible.

put “Hello there, #{name}!”

vs.

puts “Hello there, ” << name = “!”

9. Destructive operations are faster

Ruby’s in-place methods that modify the actual value instead of working on a copy of it are much faster, but be careful as they sometimes behave strangely (i.e., for!)

10. Avoid unnecessary calls to uniq on arrays

In many cases methods are already calling uniq on an array and there’s no need for you to call it yet again.

11. For loops are faster than .each

When you use .each you encounter per-request execution; for loops avoid this expensive operation.

12. Use x.blank? over x.nil? || x.empty?

When using ActionPack there’s no need for x.nil? or x.empty?; x.blank? checks for both of these.

13. Avoid calls to parse_date and strftime

Both of these are very expensive operations.  Use regular expressions when parsing out date/time components.

14. Don’t use unnecessary block parameters

If you won’t be using the parameter in the block don’t specify it in the parameter list.  Go through your code and ensure that any parameters declared are used or removed.

15. Know your gems

Not all libraries are created with performance in mind.  Many gems are slapped together to solve a particular problem that author was having.  Before you introduce a new gem into your performance-oriented production codebase be sure to perform thorough benchmarking and testing against other gems that perform the same tasks.

16. Profile your code regularly

If you profile you code regularly you’ll be able to tell if the latest change to the code will have an adverse effect on performance.  Integrate profiling into your testing process and make it automated to ensure that it’s not forgotten.  Like unit testing and BDD profiling goes a long way.

17. Improve your algorithms before you try to improve your code

Algorithmic improvements are almost always going to have more of an impact on the performance of your code than tweaks to the way your code is written will.  Make sure your algorithm is designed to be efficient and has no extraneous methods or calls.  Also, test for most frequent cases first and exit loops as soon as possible.

18. Test the most frequently occurring case first

When using if statements or a case statement always test the cases in the order that they occur most frequently.  This allows less code to run before a decision is made.  It may not seem like much but over several hundred or thousand runs through the decision logic you’ll notice a definite performance gain.

19. Optimize the way you access global constants

Be sure to precede a global constant with it’s namespace and the double colon operator (Namespace::constant_name) to reduce the time needed to query the library.

20. Use explicit returns

Although Ruby will automatically return the result of the last completed operation if no return value is provided you should use explicit return values.  Explicit returns are faster, especially in older Ruby versions such as 1.8.x.

First, let’s take a look at the performance counters we used for this monitor and the information each metric provides.

Metrics

Source Code

The complete source code can be downloaded from Git. We’ll go over the parts of the source code that create the monitor and add the data.

Total Throughput Monitors

In the snippet below the monitor name is provided and the parameters are defined. The TotalBytesThroughput represents the sum of all network adapters’ throughput. The Total Packets/Sec monitor is the sum of all network adapters’ packet counters. The function “AddCustMon” adds the monitor to the dashboard using the standard Monitis API calls.

'Total Bytes Troughput
Row = 1 : Column = 1
MonitorName = "Total+Bytes+Throughput"
monitorParams = "Total+Bytes+Throughput:Bandwidth:Bandwidth:3:false;"
resultParams = "TotalBytesReceivedPerSec:Received/Sec:N%2FA:2;TotalBytesSentPerSec:Sent/Sec:N%2FA:2;TotalBytesTotalPerSec:Total/Sec:N%2FA:2;"
AddCustMon

'Total Packets/Sec
Row = 1 : Column = 2
MonitorName = "Total+Packets"
monitorParams = "Total+PacketsPerSec:Packets:Packets/Sec:3:false;"
resultParams = "TotalPacketsReceived:Received/Sec:N%2FA:2;TotalPacketsSent:Sent/Sec:N%2FA:2;TotalPacketsPerSec:Packets/Sec:N%2FA:2;TotalPacketsReceivedDiscarded:Discarded:N%2FA:2;TotalPacketsReceivedError:Error:N%2FA:2;"
AddCustMon

Individual Network Adapter Monitors

Next we need to add the two monitors for each network adapter in our system. We query the Win32_NetworkAdapterConfiguraiton object to only retrieve those adapters with a valid TCP/IP configuration. This will prevent inactive network connections from being added to the dashboard. We also need to name the monitors using the unique name of the network adapter so we can address the correct adapter when adding data.

Set colConfs = objWMI.ExecQuery("SELECT * FROM Win32_NetworkAdapterConfiguration WHERE IPEnabled = True")
For Each objConf in colConfs 
	strAdapterName = GetAdapterName(objConf.Caption)
	strIPAddress = GetMultiString_FromArray(objConf.IPAddress, ", ")

	Set colNetAdapters = objWMI.ExecQuery("Select * From Win32_NetworkAdapter WHERE Caption = '" & objConf.Caption & "'")
	For Each objAdapter In colNetAdapters

		WScript.Echo "Adding monitor for network connection: " & objAdapter.NetConnectionID

		'Bandwidth and Bytes
		Column = 1
		MonitorName = "Bytes+Throughput:+" & Replace(objAdapter.NetConnectionID, " ", "+")
		monitorParams = "Bandwidth:Bytes+Throughput:Bytes+Throughput:3:false;"
		resultParams = "BytesReceivedPerSec:Received/Sec:N%2FA:2;BytesSentPerSec:Sent/Sec:N%2FA:2;BytesTotalPerSec:Total/Sec:N%2FA:2;"
		AddCustMon

		'Packets/Sec
		Column = 2
		MonitorName = "Packets:+" & Replace(objAdapter.NetConnectionID, " ", "+")
		monitorParams = "PacketsPerSec:Packets:Packets/Sec:3:false;"
		resultParams = "PacketsReceivedPerSec:Received/Sec:N%2FA:2;PacketsSentPerSec:Sent/Sec:N%2FA:2;TotalPacketsPerSec:Packets/Sec:N%2FA:2;PacketsReceivedDiscarded:Discarded:N%2FA:2;PacketsReceivedError:Error:N%2FA:2;"
		AddCustMon

		'Make sure we add the monitor on the correct row and column
		Row = Row + 1
	Next
Next

 Pushing Data to the Monitors

Now that we have the monitor definitions added to the dashboard, it’s time to add the actual performance counter data to them.

First we initialize our variables to calculate the total throughput and total packets for all adapters. Then we loop through each network adapter in similar fashion as we did before and retrieve the metrics from the WMI class Win32_NetworkAdapter.

The metrics for each adapter are pushed to the custom monitor on the dashboard and we make sure to add the individual adapter metrics to the total counters. Once we’ve looped through all adapters, we push the calculated totals to the summary counters.

Function GetNetworkData
	'Clear total counters
	TotalBytesReceivedPerSec = 0
	TotalBytesSentPerSec = 0
	TotalBytesTotalPerSec = 0
	TotalPacketsReceivedPerSec = 0
	TotalPacketsSentPerSec = 0
	TotalPacketsReceivedDiscarded = 0
	TotalPacketsReceivedError = 0
	TotalPacketsSentDiscarded = 0

	Set colNetAdapters = objWMIService.ExecQuery("Select * From Win32_NetworkAdapter") ' WHERE Caption = '" & objConf.Caption & "'")
	For Each objAdapter In colNetAdapters

		strAdapterName = GetAdapterName(objAdapter.Name)

		'Handle the bandwidth and bytes counters
		MonitorName = "Bytes Throughput: " + objAdapter.NetConnectionID
		MonitorID = FindMonitorID(MonitorName)

		If Trim(MonitorID) <> "" Then
			For Each objItem in objNetworkData
				strPerfDataName = GetAdapterName(objItem.Name)

				WScript.Echo strPerfDataName & " - " & GetAdapterName(objItem.Name)

				If strPerfDataName = strAdapterName Then
					Wscript.echo "Name: " & strPerfDataName
					Wscript.Echo "Bytes Received/Sec: " & objItem.BytesReceivedPersec
					Wscript.Echo "Bytes Sent/Sec: " & objItem.BytesSentPersec
					Wscript.Echo "Bytes Total/Sec: " & objItem.BytesTotalPersec

					Results = "BytesTotalPerSec:" & CStr(objItem.BytesTotalPerSec) & _
							  ";BytesReceivedPerSec:" & CStr(objItem.BytesReceivedPerSec) & _		
							  ";BytesSentPerSec:" & CStr(objItem.BytesSentPerSec)		
					AddResult

					'Accumulate Totals
					TotalBytesReceivedPerSec = TotalBytesReceivedPerSec + objItem.BytesReceivedPerSec
					TotalBytesSentPerSec = TotalBytesSentPerSec + objItem.BytesSentPerSec
					TotalBytesTotalPerSec = TotalBytesTotalPerSec + objItem.BytesTotalPerSec
				End If
			Next
		End If

		'Handle the packet counters
		MonitorName = "Packets: " & objAdapter.NetConnectionID
		MonitorID = FindMonitorID(MonitorName)
		If Trim(MonitorID) <> "" Then
			For Each objItem in objNetworkData
				strPerfDataName = GetAdapterName(objItem.Name)

				WScript.Echo strPerfDataName & " - " & GetAdapterName(objItem.Name)

				If strPerfDataName = strAdapterName Then

					Wscript.echo "Name: " & strPerfDataName
					Wscript.Echo "Packets Received/Sec: " & objItem.PacketsReceivedPersec
					Wscript.Echo "Packets Sent/Sec : " & objItem.PacketsSentPersec

					Results = "PacketsReceivedPerSec:" & CStr(objItem.PacketsReceivedPersec) & _
							  ";PacketsSentPerSec:" & CStr(objItem.PacketsSentPersec) & _
							  ";PacketsReceivedDiscarded:" & CStr(objItem.PacketsReceivedDiscarded) & _
							  ";PacketsReceivedError:" & CStr(objItem.PacketsReceivedErrors) & _
							  ";TotalPacketsPerSec:" & CStr(objItem.PacketsPerSec)
					AddResult

					'Accumulate Totals
					TotalPacketsReceivedPerSec = TotalPacketsReceivedPerSec + objItem.PacketsReceivedPersec
					TotalPacketsSentPerSec = TotalPacketsSentPerSec + objItem.PacketsSentPersec
					TotalPacketsReceivedDiscarded = TotalPacketsReceivedDiscarded + objItem.PacketsReceivedDiscarded
					TotalPacketsReceivedError = TotalPacketsReceivedError + objItem.PacketsReceivedErrors
					TotalPacketsPerSec = TotalPacketsPerSec + objItem.PacketsPerSec

				End If
			Next
		End If

	Next


	'Add Total results to the monitor
	MonitorName = "Total Bytes Throughput"
	MonitorID = FindMonitorID(MonitorName)
	If Trim(MonitorID) <> "" Then
		Results = "TotalBytesTotalPerSec:" & CStr(TotalBytesTotalPerSec) & _
				  ";TotalBytesReceivedPerSec:" & CStr(TotalBytesReceivedPerSec) & _		
				  ";TotalBytesSentPerSec:" & CStr(TotalBytesSentPerSec)	
		AddResult
	End If

	MonitorName = "Total Packets"
	MonitorID = FindMonitorID(MonitorName)
	If Trim(MonitorID) <> "" Then
		Results = "TotalPacketsReceived:" & CStr(TotalPacketsReceivedPerSec) & _
				  ";TotalPacketsSent:" & CStr(TotalPacketsSentPerSec) & _
				  ";TotalPacketsReceivedDiscarded:" & CStr(TotalPacketsReceivedDiscarded) & _
				  ";TotalPacketsReceivedErrors:" & CStr(TotalPacketsReceivedError) & _
				  ";TotalPacketsPerSec:" & CStr(TotalPacketsPerSec)
		AddResult
	End If		

End Function

 Installing the monitor

After you downloaded the code from Git, you can install the custom monitor by executing the following command from a command window:

Cscript AddCustomBandwidthMonitor.vbs

This command creates a new monitor page on the Monitis dashboard called “Bandwidth” and adds the specific monitors. Of course no data is displayed until we actually run the monitor. To run the monitor and start collecting performance data execute the following command:

Cscript PushDataBandwidthMonitor .vbs

Once performance data is being collected you’ll be able to bring up your dashboard, select the Bandwidth tab and monitor the performance of your network adapter(s).

As said, the monitor will support multiple network adapters. We’ll create a monitor window for total bytes throughput for all adapters and another one for total packet traffic. We’ll also create the same monitors for each individual network adapter. The screenshot below shows two sets of monitors; one for the total traffic and one set for the network adapter in our system.

After you download the code from Git, you can of course edit the script and add other counters if you want.

How many of us take for granted Microsoft’s family of tools that contribute to the security of your organization? The most commonly used and appreciated tools are:

• Forefront Family
• Microsoft Security Essentials
• Windows Intune / Windows Update / Microsoft System Center Family
• Windows Firewall (with Advanced Security)

Let’s take a look at all of these tools and their features. Then, we can look at other less popular tools offered by Microsoft — such as Microsoft Security Compliance Manager and Microsoft Security Assessment Tool.

1. Forefront Family

Microsoft has spent a great amount of time trying to supply a manageable easy-to-use security solution for its products. The result of these efforts is the Forefront Family. It consists of several tools you can use to secure your Microsoft infrastructure. Also, it is designed to interact with other Microsoft tools, such as Active Directory, Group Policy and Windows Update.

Forefront Endpoint Protection is the enterprise-oriented product that delivers real-time, anti-virus, malware and spyware protection. Its integration with the System Center Family of managing products makes it extremely popular and easy for adoption.

Another product in the Forefront Family is Forefront Threat Management Gateway. It is the successor of the Internet Security and Acceleration (ISA) Server and provides advanced firewall functionalities, including URL filtering, intrusion prevention, HTTP/HTTPS inspection, and much more.

The two popular solutions for applications are Forefront protection for Exchange Server and Forefront protection for SharePoint. With the increasing popularity of cloud solutions, Forefront Online Protection for Exchange is used to secure the cloud-based version of Exchange – Exchange Online. All of these products can easily be managed through the Forefront Protection Server Management Console. It supports multiple servers and has great reporting capabilities.

Microsoft Forefront Identity Manager is a great tool that can help you manage access between heterogeneous systems, including Active Directory, Novell, Sun, IBM, Lotus Notes, Exchange, Oracle and SQL Server databases, SAP, and even flat file systems. It supports both password- and certificate-based access.
If you need to provide access to users outside your organization, such as partners or home-based employees, Forefront Unified Access Gateway is there to help you. It supports both VPN and Direct Access to your network and introduces policies and configurations over these connections.

2. Microsoft Security Essentials

For end clients, who can’t afford to pay for the manageable Forefront client, Microsoft has delivered the free tool Microsoft Security Essentials, which uses the same definitions as Forefront, but lacks the manageability part.

Microsoft Security Essentials does provide antivirus, antispyware, and rootkit protection; it also supports Windows 7 and Windows Server 2008 R2; and it’s still using Dynamic Signature Service, which contributes to the daily definitions updates by detecting newly identified malware.

However, Microsoft Security Essentials doesn’t provide some of the handiest Forefront functionalities, such as Group Policy and External Device control; Network Access Protection integration (the Windows Server 2008 capability to granularly control network access based on who the client is and the groups to which the client belongs); and integrated host firewall management.

It also has the extremely useful capability of limiting processor usage during scans. Do youu ever encounter the problem of processor usage reaching 100% and the computer freezing with some other products? Microsoft promises that doesn’t happen with Microsoft Security Essentials.

3. Updates

As keeping your system updated is a main part of its hardening, Microsoft has many tools that help you get important updates on time.

Your first option is to set your Windows Update feature to automatically download the updates from the Microsoft site. However, this way you don’t have much control over which updates get installed. You can set the feature to let you make a decision as to whether to install a particular update or not, but this then requires extra administrative efforts to achieve what is normally a simple task.

Another option is to use the Microsoft System Center Configuration Manager (SCCM) or Windows Intune to centrally manage the updates on all your clients’ computers. Both of these products offer a considerable amount of functionalities, including keeping computers up to date. While SCCM is a hosted application and needs to be installed on a local machine to work, Windows Intune is an entirely cloud-based Microsoft solution that can help you manage your network. All you need is a web browser. And with that, you can make sure your computers all over the world are properly updated.

4. Windows Firewall (with Advanced Security)

Windows Firewall (In Windows Server 2008, it is called Windows Firewall with Advanced Security) can contribute to your current security configuration, providing a defense-in-depth mechanism for end users. If you haven’t purchased Forefront protection, then you can use the built-in Windows Firewall to specify rules regarding your inbound and outbound traffic.

5. Microsoft Security Compliance Manager

Microsoft Security Compliance Manager includes various baseline security policies — both for client and server Windows systems and applications. The policies are based on industry practices and let you reduce the security threats your systems are exposed to. You can easily compare your existing policies with these baseline security policies for reference, or deploy the baseline policies to be sure your infrastructure is secured.

6. Microsoft Security Assessment Tool

Microsoft Security Assessment Tool is a product that can help you secure your entire IT infrastructure by asking you various questions with a Yes/No answer. Questions are based on the ISO 17799 and NIST-800.x standards. Your answers are compared to the best practices that Microsoft has developed. Then a summary with lots of recommendations and relevant online topics is delivered to you. It can be very useful after your initial setup is completed.

If your infrastructure is now secured, take the time to set monitoring on your main servers. You can do that very easily with Monitis.

2. NGINX Best Practices
NGINX is now the world’s second most popular server, which is quite impressive considering the strong competition from Apache and Microsoft IIS. These NGINX best practices will help you run the popular web server with optimal performance. Eight steps cover everything from improving Disk I/O to configuring NGINX as a load balancer to avoiding common configuration pitfalls.

3. You Can Monitor Your XenServer With Monitis!
Citrix’s XenServer is a leader in the virtualization platform space. This post shows how you can use the Monitis API to monitor XenServer. There is sample code in the Monitis Github repository for you to download. Then just follow 11 steps to run the daemon and create the custom Monitors to start monitoring XenServer.

4. Monitor Everything with Monitis – And do it easily with PowerShell – Part 14 (Final)
This is a continuation of the last article, which showed you how to create custom monitor updater commands which run a script to upload values to Monitis. This makes it very easy to convert any PowerShell script into a Monitis monitor. This post recaps some of the important points covered in this series of articles. This series focused on using Montiis with PowerShell’s in-the-box capabilities, but there are many other community modules out there you can use with Monitis. This post shows how to apply the approach covered in this series to those modules.

5. VDI on Windows Server 2008 R2 Hyper-V: Performance Monitoring Explained–Part 2
The last post in this series focused on Overall Health metrics of VDI. This post focuses more on processor, memory, and storage monitoring. For each of these, the most important counters are listed. For example, for memory monitoring, the metrics are Hyper-V Hypervisor Partition, Hyper-V Hypervisor Root Partition, Hyper-V VM Vid Partition, Memory. For storage, they are Physical Disk, Hyper-V Virtual Storage Device, and Hyper-V Virtual IDE Controller. All the necessary scripts are on Monitis’ Gifthub page.

6. Mongo and Monitis: A NoSQL Dream Come True
MongoDB is a great NoSQL database that’s grown very popular among developers. It’s known for being highly scalable and easy to work with. These best practices highlight some of the common issues developers face with Mongo and how to work around them, like how to deal with too many open connections. Load testing is recommended to test how many connections, or queries, your database can handle. And remember, using custom monitors, all these metrics can be tracked in Monitis.

Processor Metrics

There are three processor-related metric sets:

• Hyper-V Hypervisor Logical Processor
• Hyper-V Hypervisor Root Virtual Processor
• Hyper-V Hypervisor Virtual Processor

The most important counter set you want to monitor is the Hyper-V Hypervisor Logical Processor. This counter set allows you to determine how much of the physical processor is being used. The Virtual Processor counter set only shows part of the Hyper-V Hypervisor Logical Processor, so you want to make sure to use the Logical Processor counter set. The most useful counters in this counter set are the following;

• %Guest Run
• %Hypervisor Run Time
• %Idle Run Time
• %Total Run Time

Note that there is one logical processor in Hyper-V that that has a larger workload than the other logical processors and that is LP0. This Logical Processor is where all interrupts in the system are directed to. If this LP hits 100% utilization there is likely an I/O bottleneck.

The Hyper-V Hypervisor Root Virtual Processor and Hyper-V Hypervisor Virtual Processor are just slices of the Logical Processor counter and can help you understand how much total CPU the root and guests are using on the system.

If you want to monitor the root CPU you should use the Processor counter set. According to Microsoft this counter suffers from some skew, but is useful because it gives an idea of how busy the root is. This is important because the root is involved in all I/O, meaning if the root CPU’s are saturated your whole system is likely saturated. In general you want to see the root CPU lower than 10% utilization and over 50% might indicate an issue.

Memory Counters

The following are the counter sets you want to monitor;

• Hyper-V Hypervisor Partition
• Hyper-V Hypervisor Root Partition
• Hyper-V VM Vid Partition
• Memory

The Hyper-v [Root] Partition counters determine how much memory the Hypervisor is managing and using on behalf of a VM which includes the guest address space but not all the memory in the worker process and VID partition. The Hyper-V Hypervisor [ROOT] Partition counter set indicates in the 1G GPA Pages and 2M GPA Pages metric whether or not a VM is using large memory pages which improves overall VM performance.

The Hyper-V VM Vid Partition counters account for the guest address space and any additional memory the VID needs to manage the VM.

The Memory counters are not specific to any VM and can be used to measure memory used to service I/O on behalf of a virtual machine.

The Partition Counter set also indicates in the Deposited Pages metric how much memory the hypervisor is using for managing the VM. The last counter that is interesting in the partition counter set is the Virtual Processors counter which tells you how many Virtual Processors a VM is configured to use.

The Hyper-V VM Vid Partition counters have two counters that you want to monitor. The Physical Pages Allocated is the total number of guest pages and VID pages needed to manage the VM. The Remote Physical Pages let you know on NUMA based systems if a VM is spanning multiple nodes. Note that Microsoft does not recommend such a configuration.

The Memory counter set allows you to monitor how much memory is consumed in the root. The root is responsible for managing all memory in Hyper-V. When a VM starts you will see the Available Bytes go down by at least the amount of memory given to the guest plus around another 16 – 64MB for guests meta data structures.

The following counters are useful to monitor:

• Available Bytes – Returns how much memory is remaining for guests.
• Pages / Sec – Measures memory pressure. This metric can tell you if processes are competing with each other for physical RAM.

Network Counters

You can use the network counters to monitor overall networking performance on the system. The most important thing to monitor is the total throughput counters to make sure the NICs are not getting saturated. There are four relevant counter sets:

• Network Interface
• Hyper-V Virtual Switch
• Hyper-V Legacy Network Adapter
• Hyper-V Virtual Network Adapter

The Network Interface measures the overall performance of physical device whereas the other counter sets represent the activity of the virtual switches and network adapters in the virtual machines. For the Network Interface the following are the most important counters to monitor:

• Bytes Total / Sec
• Offloaded Connections
• Packets / Sec
• Packets Outbound Errors
• Packets Receive Errors

The Hyper-V Virtual Switch counters are good to monitor because depending on your configured network, traffic might only exist on the virtual switch. This is the case with network traffic between guest to guest. The most important counters are:

• Bytes/Sec
• Packets/Sec

The Hyper-V Virtual Network Adapter and the Hyper-V Legacy Network Adapter counter sets allow you to measure performance of a virtual machine. These counter sets are named with the friendly name of the VM plus the name of the network adapter followed by two GUIDs. The GUIDs are the internal id of the VM and adapter.

If you assign a Legacy Network Adapter to a VM then the counter set to use is the Hyper-V Legacy Network Adapter. In general you should not use this network adapter type because it is not enlightened (see our prior ‘Best Practices’ article). It creates a lot of CPU load in the root, and is slower than the Virtual Network Adapter. However, you need the Legacy Network Adapter to get a VM working before installing Integration Services. Once your VM is working with Integration Services you should use the Network Adapter and the “Hyper-V Virtual Network Adapter” counter set. Note that both Windows Server 2008 and Windows Server 2008 R2 have integration Services pre-installed.

The Hyper-V Legacy Network Adapter counters to monitor are:

• Bytes Dropped
• Bytes Sent / Sec
• Bytes Received / Sec

In the Hyper-V Virtual Network Adapter you should monitor;

• Bytes / Sec
• Packets / Sec

Storage Counters

The storage counters are used to monitor overall disk performance on the system and each VM. There are:

• Physical Disk
• Hyper-V Virtual Storage Device
• Hyper-V Virtual IDE Controller

The Physical Disk counter set will give overall storage performance on the system. The other two metrics are for VM’s only. Of the Physical Disk counter set there are three metrics that are useful to monitor. The Current Disk Queue Length measures how busy the drives are. This counter should be around two per drive, depending on your drive configuration. A queue length of 32 might indicate this disk is the bottleneck in your system.

Another useful counter is the Disk Bytes / Sec. You can typically expect to see about 10MB/sec per drive; a safe number for most drives. For random workloads, look at the counter Disk Transfers/Sec. Expect to see about 100 I/O’s per second (IOPs) for each drive. There are drives that can do much better, like 180 IOPs and some that perform worse.

Because of the way storage works in Hyper-V, there are two Hyper-V storage counter sets. Hyper-V provides two virtual storage buses for virtual machines. One IDE and one SCSI. The Virtual IDE counters show up in the Hyper-V Virtual IDE Controller counter set, unless the Integration Services are loaded. In that case you’ll see the activity for both virtual IDE and SCSI in the Hyper-V Virtual Storage Device counter set. If you don’t have integration services installed, only the Hyper-V Virtual IDE Controller will show the VM disk activity.

It wouldn’t hurt to monitor all the counters of both Hyper-V Virtual IDE Controller and Hyper-V Virtual Storage Device. Below are the basic metrics for these two counter sets:

For Hyper-V Virtual IDE Controller there are:

• Read Bytes / Sec
• Write Bytes / Sec
• Read Sectors / Sec
• Write Sectors / Sec

For Hyper-V Virtual Storage Device there are:

• Error Count
• Flush Count
• Read Bytes / Sec
• Write Bytes / Sec
• Read Count
• Write Count

That’s it for our discussion about the Hyper-V metrics that can be used to help you understand what the system is doing. In the table below, we’ll list all the counters:

• \Hyper-V Virtual Machine Health Summary \*
• \Hyper-V Hypervisor\*
• \Processor(_Total)\*
• \Hyper-V Hypervisor Logical Processor(*)\%Guest Run
• \Hyper-V Hypervisor Logical Processor(*)\%Hypervisor Run Time
• \Hyper-V Hypervisor Logical Processor(*)\%Idle Run Time
• \Hyper-V Hypervisor Logical Processor(*)\%Total Run Time
• \Hyper-V Hypervisor Root Virtual Processor (*)\%Guest Run
• \Hyper-V Hypervisor Root Virtual Processor (*)\%Hypervisor Run Time
• \Hyper-V Hypervisor Root Virtual Processor (*)\%Idle Run Time
• \Hyper-V Hypervisor Root Virtual Processor (*)\%Total Run Time
• \Hyper-V Hypervisor Virtual Processor (_Total)\*
• \Memory\Pages / Sec
• \Memory\Available Bytes
• \Hyper-V Hypervisor Partition(*)\2M GPA Pages
• \Hyper-V Hypervisor Partition(*)\Deposited Pages
• \Hyper-V Hypervisor Partition(*)\Virtual Processors
• \Hyper-V Hypervisor Root Partition(*)\*
• \Hyper-V VM Vid Partition(*)\Physical Pages Allocated
• \Hyper-V VM Vid Partition(*)\Remote Physical Pages
• \Network Interface(*)\*
• \Hyper-V Virtual Switch(*)\*
• \Hyper-V Legacy Network Adapter(*)\*
• \Hyper-V Virtual Network Adapter(*)\*
• \Physical Disk(*)\Current Disk Queue Length
• \Physical Disk(*)\Disk Bytes / sec
• \Physical Disk(*)\Disk Transfers/sec
• \Hyper-V Virtual Storage Device(*)\*
• \Hyper-V Virtual IDE Controller(*)\*

(_Total) means only the total should be collected.

(*) means collect all counters.

\* means collect all the counters in the counter set.

\<name> means only collect that counter.

Custom Monitor

We have provided a custom monitor in VBScript that uses remote WMI and the available performance counters. The most important counters that we discussed in this article are included in this monitor. You can download this script from GitHub.

Scheduling Custom Monitors with Monitis and PowerShell

In last article, we talked about creating custom monitor updater commands that would run a script and upload the values to Monitis.  These custom updaters make it a snap to convert any PowerShell script to a Monitis monitor.  Simply plug the script in and start using it’s more logical name: Update-YourMonitor.

Add-MonitisCustomMonitorUpdater -Name OutOfPaper -ScriptBlock {
    Get-Counter "\Print Queue(*)\Out of Paper Errors" |
        Select-Object -ExpandProperty CounterSamples |
        Where-Object { $_.InstanceName -ne '_total' -and $_.CookedValue -gt 0} |
        Select-Object @{
            Label = "Printer"
            Expression={$_.InstanceName}
        }, @{
            Label = "OutOfPaperErrors"
            Expression={$_.CookedValue}
        }
} -Property Printer, OutOfPaperErrors

Once you’ve run this code, you can simply do:

Import-Module Monitis
Update-MonitisOutOfPaper

To update the values.

You can simply schedule the update by scheduling the two lines.  To click thru it, run TaskSched.msc, then create a new task.  Run PowerShell.exe as the task, and use the arguments –command Import-Module Monitis;Update-MonitisOutOfPaper.

You can also schedule the task using a script, but not one built into the box.

All throughout this article series, you’ve been focused on using the Monitis module and in-the-box capabilities of Powershell, but there are hundreds of community modules to choose from.  You can easily leverage these community modules in your own custom monitors.

In one very large collection of these modules, called the PowerShellPack (http://code.msdn.microsoft.com/PowerShellPack), there is a task scheduler module you can use to schedule the monitor.

Like every command in PowerShell and every command in the monitis module, you can find out about the commands in this module with Get-Command and Get-Help.

If you want to schedule a task that runs your custom monitor with this module, it will look something like this:

New-Task |
    Add-TaskTrigger -OnBoot -Repeat "0:0:10" |
    Add-TaskAction -Script {
        Import-Module Monitis
        Update-OutOfPaper
    } |
    Register-ScheduledTask -name "CheckForOutOfPaper" -Credential (Get-Credential)

Register-ScheduledTask -name “CheckForOutOfPaper” -Credential (Get-Credential)

New task creates a new task

Add-TaskTrigger gives it a new trigger, on boot and repeating every 10 minutes thereafter.

Add-TaskAction runs the monitor, importing the module and running Update-OutOfPapeer.

Register-ScheduledTask registers the task, and –Credential registers the credential the task will be run under.

That’s it.  Now you can create a monitor to hold any piece of information, collect that information with PowerShell, and update it on a regular basis.  Thanks to your notification rule, your printer lackey will automatically head off to keep your printers filled, and you can monitor your friends Facebook updates while Monitis and PowerShell monitor your data center.

Monitis is an incredible monitoring platform with the flexibility to monitor whatever you want.  PowerShell is an incredible task management and automation platform that can do anything you need it to.  Together, you truly can Monitis anything with Monitis.

See also:

Monitor Everything with Monitis – And do it easily with PowerShell – Part 1

Part 2: Managing External Monitors with Monitis and PowerShell

Part 3: Mining External Monitor Results with Monitis and PowerShell

Part 4: Monitoring Web Applications with Monitis

Part 5: Testing Web Content with Monitis, Excel, and PowerShell

Part 6: Monitoring Anything with a Custom Monitor

Part 7:  Hardware Inventory with Monitis Custom Monitors

Part 8: Monitoring Logons with Monitis

Part 9: Monitoring Connections to Shared Folders with Monitis and Custom Monitors

Part 10: Inventory Windows Installations with Monitis and PowerShell

Part 11: Monitoring Removable Disks on Many Computers with Monitis and PowerShell

Part 12: Monitoring Event Logs and Using Monitis Notifications

Part 13: Monitoring out of Paper with Monitis (Printer Monitoring)

As we have discussed on the Monitis blog virtualization is one of the hottest IT subjects today.

One leader in this field is Citrix — with the XenServer product family. XenServer is the most known commercial implementation of of the open source Xen hypervisor. By the way, you should pronounced Xen like “zen.” It comes from a Greek word meaning “guest.”

In the IT world, Xen originated as a research project, led by Ian Pratt at England’s Cambridge University. It was first developed by XenSource Inc., and after acquisition, the work was continued by Citrix. Currently, you can choose between 4 editions of XenServer:

• XenServer [Free Edition]
• XenServer Advanced Edition
• XenServer Enterprise Edition
• XenServer Platinum Edition

Other Citrix “Xen” technology, like XenApp or XenDesktop, are not related to the original Xen project.

In 2009, Citrix decided to open the source of XenServer, and the company created XCP (Xen Cloud Platform). In the first phase, XCP was a whole Linux distribution. Recently, the initiative has been extended to provide XAPI toolstack on standard Linux distribution, in the first place for Debian and Ubuntu. What this means is that you can try XAPI on your home machine.

Monitoring

In the article below, we’ll learn how to apply daemon exporting XenServer metrics to Monitis. This is only proof of concept, but it can be easily modified to do exactly what you want. (This is the power you get using Monitis Open API).

XenServer/XCP

There is a dedicated XenServer monitoring tool included in all paid editions. You can also find some third-party tools. Obviously, we are interested in how we can monitor XenServer and XCP with Monitis. One possible way is to enable SNMP on XenServer and treat it as a standard Linux in Monitis. After all, XenServer and XCP dom0 are based on CentOS. Another method that we are going to give an overview of today is XenServer/XCP-specific. XenServer enables you to download, host and store all VM statistic in a  RRD (Round Robin Database) using HTTP handler. You can find some information on how to use XenServer RRD data in two documents from the Citrix Community page (Click here and here). In the second document, you’ll see that there are two examples of Python scripts that we are going to use for the base to our daemon updating XenServer monitors in Monitis. Both scripts use the python “rrd_update” module. You can also find on this on the Citrix page or grab it from github.

Monitis

Monitis has dedicated libraries in many programing languages. You need to use python to export XenServer RRD; therefore to update results on Monitis, we are going to use the python library, too. The code can be obtained from our github repository. Please remember that you need the elementtree module, which is not a part of standard python distribution. Plus, you have to get it from the official website or obtain it from your distribution repository. There is a note about it in the code, but not in the original article.

xen2monitis

Having these two weapons, we are now ready to attack. First, there is the example code in our git repository that you can install on XenServer or Linux distribution with XCP. Alternately, you can choose another server with a connection to both Monitis and the XenServer that you want to monitor. However, installing our daemon on XenServer/XCP seems to be the best choice.

Here are some easy directions:

1.) Create the Xen2Monitis default directory:

mkdir /var/local/lib/xen2monitis

2.) Download the xen2monitis code which is part of Monitis-Linux-Scripts:

git clone https://github.com/monitisexchange/Monitis-Linux-Scripts.git

3.) Copy Xen2Monits file to default location:

cp  Monitis-Linux-Scripts.git/Xen2Monits/* /var/local/lib/xen2monitis

4.) Go to created directory:

cd /var/local/lib/xen2monitis

5.) Obtain the elementtree module:
- You can get the source code:

wget http://effbot.org/media/downloads/elementtree-1.2.6-20050316.tar.gz

- Or you can get a dedicated package for your distribution. If you plan to use XenServer, the elementtree is already installed there.

6.) Install the elementtree (if not already done by your distribution)
- for source code (the last command as root):

tar -xzf elementtree-1.2.6-20050316.tar.gz
cd elementtree-1.2.6-20050316
python setup.py install

7.) Next, add your API Key and Secret to a file. Please separate them with “,”.The default ‘secret file’ is  /usr/local/xen2monitis/secret

8.) Add the xenserver root password to your secret file, so it should be similar to:

5DQQ0460S18ERNRNRUI7M905N1, 7MA9XJNDL3KAMBAKFR9981N04J, secret_password

9.) Now you can start the daemon:

/usr/local/lib/xen2monitis/xen2monitis.py start

10.) You need to wait few minutes. You can check if your script works without any problems in /var/log/xs2monitis.log.

11.) After a few minutes, you should have a bunch of new graphs that you can add to the Dashboard by Monitors->Custom Monitor menu.

Monitor Name Conventions

When you look at newly created monitor names, you might be a bit confused. The names tend to be very long and have a strange set of strings separated by ‘-’, for example:

lisbon – cpu1 (HOST: 0c1bb847-d867-75ff-27e0-7b468f1385f2)

Such name conventions were chosen to ensure the uniqueness of each monitor. Imagine that you have two VMs called “Win7″. Daemon would not be able to distinguish between them and would save all data into one monitor! To address it, XenServer UUID (universally unique identifier), what looks like set of strings separated by “-”, is added to each title.

Daemon Modification

Using xen2monitis.py script, please remember that there are some values you can define at the beginning of the file, for example:

• where is the secret file (default /usr/local/lib/xen2monitis/secret)
• where is the lock file (default /var/run/xen2monitis.pid)
• where is the log file (default /var/log/xen2monitis.log)
• the url to pool master (default http://localhost)

You might prefer not to use daemon, but, instead, call up script from cron job. It’s not a big problem. You need to copy the main loop, start after line 115 to separate the file and convert it.

We hope this article showing you how to monitor your XenServer with Monitis makes your IT life a lot easier. Please share some tips of your own!

Need for Speed?

With its first public release in 2004, NGINX now is the second most used web server in the world. This is quite impressive considering NGINX was born among several big players including Apache, Microsoft IIS, and other popular web server software. By sharing best practices and a few tips today, we would like to bring NGINX’s new blood up to speed. Even if you come from the Apache world, you will learn how to build an Apache server that married NGINX, an extremely fast and high-performance web server. Let’s go:

1. Make sure your OS is well supported by NGINX:
   In particular, double-check NGINX supports the event polling system call under your operating system. This is important because the beauty of NGINX is its asynchronous architecture for event handling. (This design is used to address the C10k problem, literally, to handle 10,000 clients simultaneously.) Under Windows, however, NGINX uses select() for event polling. While select() is non-blocking, select() can limit a process to have only 1024 open file descriptors. Therefore if you expect your NGINX server to be high-performance, Windows might not be your best environment to work with.
2. Improve disk I/O if possible:
   • You can turn off NGINX access logs and use the client site scripts such as Google Analytics to collect statistics. This way NGINX will not record every single request to your log file, reducing disk I/O significantly. It is not recommended you also turn off the error logs. After all, errors are important information, and they don’t occur that often anyway.
   • Configure “open file cache” if your system supports it. Note that the actual file is not cached, but the pointer to the file instead. That is, an inode in your filesystem. Why caching inodes can speed up disk I/O? Think of inodes as a back-of-the-book index, which can help you look up information quickly.
3. Improve network I/O if possible:
   Turn on the “Gzip” module so you can compress data before sending them over the network. Nowadays the CPUs are so fast that you can even tell NGINX to use level 6 compression just to save a bit bandwidth. If you are worried about CPU hogs, use level 1 compression since it has the best compression to time ratio.
4. Tweak the HTTP keep alive setting:
   NGINX is so memory efficient that it is possible to use only ~2.5 M to maintain 10k HTTP keep-alive connections. (too scary to be true, isn’t it?) This gives systemadmin plenty of room to lift the “keep alive” limit. The idea is to have as many keep alive connections as possible, and set the keep alive timeout with a larger number as well. Why not setting them both to infinite? It is because you want to think about malicious users. Attackers can abuse your generous setting and possibly exhaust your server memory.
5. Configure NGINX as a load balancer:
   This is easy to configure. The example below shows how NGINX uses  round-robin and client IP to achive load-balancing across backend servers.
   
6. Configure NGINX as a reverse proxy:
   Combining Apache with Nginx is a very popular setup. The idea is to have NGINX serve static files, and let Apache handle dynamic content.
   
7. Only Enable the NGINX modules you need:
   In other words, turn off the modules you are not using. Keep your NGINX simple and small. Not only you can save memory, this can also improves overall security. For example, if you are not using NGINX as a load balancer, disable the “Upstream” module.
8. Avoid NGINX configuration pitfalls:
   First of all, NGINX is not Apache. If you are new to NGINX’s configuration, you might want to check out the wiki first to have a glimpse of NGINX’s rich configuration options. The following are some quick tips for configuring NGINX:
   • Factor out “root” outside of a location block.
   • Factor out “index” to the http block.
   • Avoid using “if” unless you really have to.
   • Use “try_files” if you want to check if a file exists.
   • Use “try_files” first to serve static content on dynamic pages.
   • Use “script_filename” to avoid hardcoding absolute paths.
   • Use ” $request_uri” to avoid using regular expression
   • Use “rewrite” and “http://” to force absolute path.
   • Use “map” to customize your key-value pairs.
   • Use “stub_status” to monitor your server.
   • You can combine http and https block in NGINX
   • Set “proxy_connect_timeout” higher under heavy load.
   • Clear browser cache before testing the new configuration.

Conclusion:

NGINX is the king of serving static contents. Believe it or not, you don’t need to tweak much to optimize your NGINX server. The default configuration works quite well. It gives you a sense of “you don’t pay for what you don’t use.” If you do need to customize your NGINX, please follow NGINX best practices and, remember NGINX’s underlying principle: lean and fast.

We hope you enjoy the article. As a bonus, Monitis® offers enterprise-level solutions to monitor your NGINX servers. You can get your free signup here.

—

Useful Links:
Integrate NGINX Monitoring into Monitis.com:
http://blog.monitis.com/index.php/2012/01/23/integrate-nginx-monitoring-into-monitis-com/
Monitis Free Registration:
https://www.monitis.com/free_signup.jsp

Windows Hyper-V has three main components, each of which can be monitored; the virtstack, devices, and the hypervisor. When Windows 2008 boots the system it launches the virtstack and hypervisor. The virtstack handles the emulated devices, manages virtual machines, services I/O, etc. The Hypervisor schedules the virtual processors, manages interrupts, services timers, and controls other chip-level functions.

The root partition in Hyper-V provides the monitoring information via WMI and performance counters. Although you can monitor a virtual machine from within the running guest operating system, you only have access to the metric provided by that specific OS. One thing to keep in mind is that when querying the metric %Processor, it reports on the virtual processor(s) used in the guest, not the physical processor of the host system.

So to monitor Hyper-V and all virtual machines, you should monitor the root. Now let’s take a look at what you should monitor. We’ll cover the major resources and what to typically monitor to provide a good starting point. The top level metrics to monitor are the following:

Overall health:

• Hyper-V Virtual Machine Health Summary
• Hyper-V Hypervisor

Processor:

• Processor
• Hyper-V Hypervisor Logical Processor
• Hyper-V Hypervisor Root Virtual Processor
• Hyper-V Hypervisor Virtual Processor

Memory:

• Memory
• Hyper-V Hypervisor Partition
• Hyper-V Root Partition
• Hyper-V VM Vid Partition

Networking:

• Network Interface
• Hyper-V Virtual Switch
• Hyper-V Legacy Network Adapter
• Hyper-V Virtual Network Adapter

Storage:

• Physical Disk
• Hyper-V Virtual Storage Device
• Hyper-V Virtual IDE Controller

In this first part of the ‘monitoring explained’ articles, we’ll look at the Overall Health Metrics set of performance counters and how to use them. The other sets are discussed in Part 2.

Overall Health Metrics

There are two counter sets you can use to get an overall picture of the system:

1. Hyper-V Virtual Machine Health Summary which has two counters; “Health Ok” and “Health Critical”. Critical means some resource has been exhausted or another unrecoverable error has occurred. If you see “Health Critical” you should take action to figure out what has happened.

2. Hyper-V Hypervisor. This set has the Logical Processor counter, the number of virtual machines running is measured by the Partitions counter, and the total number of Virtual Processors is provided by the Virtual Processors counter. The logical processors are important because this is where all the work is done and are a representation of the physical processor. Microsoft recommends that you do not exceed an 8:1 ratio for the the Virtual Processor to Logical Processor ratio.

The Total Pages counter indicates how much memory the Hypervisor is using to manage the virtual machine. This counter does not capture all the overhead because another component called the Virtual Interface Driver (VID) also has overhead to manage partitions. Note that in Windows 2008 the counter Hyper-V VID Partition does not work, but it does in Windows Server 2008 R2.

In our next article in this series we’ll discuss the Processor, Memory, and Storage counter sets in detail. We will also provide a custom monitor for Hyper-V that utilizes most of the metrics that are discussed in this series.

See also Monitoring Hyper-V With PowerShell on Monitis