The case of failed VSS backups on Windows Server 2012 with KB2996928 & KB3000853
By: Netanel Ben-Shushan & Yaniv Totshvili
Special thanks to Mr. Kobi Ghan for allowing us to share his issue and take screenshots.
Background and Symptoms
On a customer environment which running Windows Server 2012 and Exchange Server 2013 at Database Availability Group (DAG) for short we found that Volume Shadow Copy Service (VSS) not truncating Exchange transaction logs.
The customer backing up his DAG environment using NetApp SnapManager for Exchange (SME).
We found no VSS-related errors on operating system, so we start investigating…
First, we tried to uninstall NetApp SME (since there are no VSS errors at the Event Viewer) and try taking backup using other utility – Symantec Backup Exec. At this point, we get no VSS-errors but we’ve find that some backups are successfully truncating the Exchange transaction logs and some backups are not. So, this is still no good, and there’s no evidence to this issue at the Event Viewer. Finally, we choose to take backups using Windows Server Backup Feature and still no logs are truncated, but… we notice that the Last full backup field at the Exchange databases-level was cleared (no information about the last full backup even if we took one right now!).
After round table decision we choose to return to the Symantec Backup Exec, after taking the first backup on a DAG-member with Windows Server Backup Feature installed we start getting VSS error with Event ID 8229 at the Event Viewer!
Refreshing the Event Viewer again, and finding two additional Microsoft Exchange Replication (MSExchangeRepl) Event ID 2034 & 2140.
After some investigations on the internet we found that two updates that are installed on the DAG members –KB3000853 & KB2996928 (Update KB3000853 contains KB2996928 hotfix).
Resolution
On the KB3000853 page we found that the VSS failed backups are known issue and we download manually the correct version of hotfix KB2996928 and install it on all Exchange DAG members.
Restarting the DAG members after the hotfix installation, and taking a new backup with Symantec Backup Exec again and the Exchange transaction logs are truncated, no VSS and MSExchangeRepl errors at the Event Viewer and the last full backup field was filled with the correct information!