Mike Barratt

What is Windows Server CSV Cache and why use it with Hyper-V platforms?

Blog Post created by Mike Barratt on Oct 13, 2015

Cluster Shared Volumes (CSV) Cache is a feature first introduced in Windows Server 2012 (and is also applicable to the later Windows Server 2012 R2 version) that allows a portion of the system memory to be used as a write-through cache.  This is similar to some other vendor implementations of using specific PCI Flash or SSD drives to perform host server read-only caching, however in this instance a portion of the system memory (RAM) is being used instead.

 

This can improve read performance for applications such as Hyper-V that use unbuffered I/O operations (and remember that unbuffered I/O operations will not be cached by the default Windows Cache Manager).

 

CSV Cache therefore has a positive impact in environments with a high number of read operations, such as Virtual Desktop Infrastructure, where the use of CSV Cache can significantly reduce the impact of virtual machine boot storms and allow a higher VM density on the underlying physical host.  However, due to the positive effects noted in the field, Microsoft now recommend enabling this for all Hyper-V clusters.

 

Unfortunately, there are a few minor differences in enabling CSV Cache for Windows Server 2012 and Windows Server 2012 R2 and there are different usage guidelines depending whether block storage or Microsoft Scale Out File Servers (SoFS) are being used to present the CSV volumes but lets see if we can clarify this a little bit.


Enabling on Windows Server 2012 Clusters

 

1. Open an elevated Windows PowerShell prompt on one of the cluster nodes

 

2. Define the size of the size of the cache to be reserved (example of setting to 512MB)

(Get-Cluster). SharedVolumeBlockCacheSizeInMB = 512

 

3. Enable CSV Cache on an individual disk (this must be executed for every disk you wish to enable caching)

Get-ClusterSharedVolume “Cluster Disk X” | Set-ClusterParameter CsvEnableBlockCache 1

 

Further Considerations for Windows 2012:

1. CSV cache is integrated into the Windows Failover Clustering feature, so it can only be used on clusters using cluster shared volumes.

2. The CSV cache size itself can be modified with no downtime, however for the Hyper-V memory on the host server itself to be modified to accommodate the memory allocated to the CSV cache, it does require a server reboot with Windows Server 2012.

3. Enabling CSV Cache on an individual disk also requires that the Physical Disk resource be recycled (taken Offline / Online) for it to take effect.

4. To ensure resource contention is avoided, it is recommended to reboot each node in the cluster after modifying the memory allocated to the CSV cache.

For these reasons, it is advisable to configure CSV cache on any new Windows Server 2012 deployments as a setup task to reduce the impact and additional steps required on a production platform that is being used.

 

Enabling on Windows Server 2012 R2 Clusters

 

In Windows Server 2012 R2, CSV caching is set for all CSV clustered disks by default, however no size is set so it is effectively disabled. To enable CSV caching, adjusting the size of the cache is all that is required.

 

1. Open an elevated Windows PowerShell prompt on one of the cluster nodes

 

2. Define the size of the size of the cache to be reserved (example of setting to 512MB)

(Get-Cluster). BlockCacheSize = 512

    

Further Considerations for Windows 2012 R2:

1. CSV cache is integrated into the Windows Failover Clustering feature, so it can only be used on clusters using cluster shared volumes.

2. On Windows Server 2012 R2 the Hyper-V cache memory reserve will be dynamically updated for each node and a reboot is not necessary.

However, it is advisable to configure CSV Cache on any new deployments as an initial setup task.

 

Additional Information and Limitations

 

You can allocate up to 20% with Windows Server 2012 and 80% with Windows Server 2012 R2 of the total physical RAM for CSV write-through cache, which will be consumed from non-paged pool memory.

 

Hyper-V (host, using block storage) – There is a wealth of information out there where folk have tested CSV Cache with different sizes.  Typically, it has been found that a setting of 512MB will deliver excellent gain at minimal cost, and is the recommend starting point / minimal value if enabled. Then based on the specific deployment and the I/O characteristics of the workloads in the VMs you may wish to increase the amount of memory allocated. Since system memory is a contended resource on a Hyper-V clustered host server, it is recommended to keep the CSV Cache size moderate.

 

Scale-out File Server – If you are using SoFS for the Hyper-V storage, it is recommended to allocate a significantly larger CSV Cache on the SoFS servers as physical memory is typically not a contended resource. This should be determined on a case by case basis but 8/12/16GB is feasible but it should not exceed 64GB.

 

CSV Cache will also be disabled on platforms that use;

 

- Tiered Storage Space with Heat Map tracking enabled

 

- De-duplicated files using the Microsoft Windows Server Data Deduplication feature

 

- ReFS Volumes with integrity stream enabled (Note: You should really be using NTFS on all production Windows 2012/2012R2 CSV Volumes)

 

Monitoring the effect of CSV Cache

The CSV Cache also provides a set of counters you can use to monitor the performance of the cache. Just open Performance Monitor and find the counters under "Cluster CSV Volume Cache"

Just a word of caution that I've seen in the field.  Although CSV Cache can be enabled on Windows Server 2012R2 by just setting the size of the cache, sometimes the performance counters will not show any data until the host is rebooted. However, be assured that the CSV caching WILL be working during this time.  At the time of writing, this bug may be have been fixed by a Microsoft Hotfix as it doesn't seem to affect fully patched Windows 2012/2012R2 systems but details are a bit sketchy.  Again, I'd strongly recommend setting the CSV cache when deploying new Hyper-V clusters and rebooting the systems before production workloads are placed on them.

 

Here's a quick table to summarise the supported options when considering CSV Cache.  Please seriously consider enabling it, I've never experienced an issue with it in the field, it's very easy to configure and if it's sized correctly doesn't require a lot of memory in order to get good results (all my deployments have a 512MB size configured initially).

TypeWindows Server 2012 R2Windows Server 2012Windows Server 2008 R2Windows Server 2008
CSV Cache SupportYesYesNoNo
Maximum CSV Cache Size80% System Memory20% System Memory--
Enable Per DiskYes (enabled for all disks by default)Yes--

Outcomes