Needles in a Haystack: DisableLoopbackCheck – Failed Installation of OpsMgr 2012 SP1 Databases and Reporting with Server 2012 and SQL 2012 SP1 over Hyper-V and ESX 5.1 (E1000E vs. VMXNET3 Networking)
Recap of situation: Failed Installation of OpsMgr 2012 SP1 on Server 2012 and SQL 2012 SP1 over Hyper-V and ESX 5.1 (E1000E vs. VMXNET3 Networking)
This started becoming a real problem when I found the System Center Operations Manager 2012 SP1 Installation could not complete successfully in an all-in-one or remote DB\Reporting (SSRS)configuration. The installation of SSRS natively shows a good health of the virtual directories for Reports and ReportServer. I was able to create a successful lab installation using Hyper-V, but could not perform a successful installation using ESX.
***Workarounds posted at the bottom of this page***
One specific error I ran into:
Setup is unable to create database on SQL Server instance <Alias or CNAME>. Please make sure that the current user has permissions to create database on the SQL Server instance specified.
What I verified: Using SQL Server Management Studio, I connected to the Instance using the CNAME and created a new database successfully.
Another issue I noted when installing Operations Manager Reporting:
Here’s some related links with similar issues although the MOM Team blog by JC Hornbeck has a slightly different error that is unrelated.
Stages during the OpsMgr Reporting Installation:
- Processing initial configuration
- Processing reporting server
- Generating script operations for action
- Copying new file
- Writing system registry values
- Registering COM+ Applications and Components
- Registering product
- Copying new files
- Writing system registry values
- Registering product
- Publishing product information
- Removing backup files
- Removing backup files
- Retrieving SRS properties
- Configuring the SRS account
- Enabling My Reports
- Creating the SSRS data source
- If failed here > primary Key constraint problem (opsmgrsetupwizard.log)
- Add SRS Service Account to Local Admins
Error: Publishing App Diagnostic Reports failed
Error Code: 0x80131501
- Check DB Permissions (Local Admin on Remote SQL Instance and potentially add as SysAdmin for Install if BUILTIN\Administrators or SCOM OpsMgr Admins was not added for SQL Security)
- If failed here >
Access is denied. (Exception from HREULT: 0x80070005 (E_ACCESSDENIED)
- [23:02:18]: Warn: :Message:SRSPolicySetter SoapException Exception: System.Web.Services.Protocols.SoapException: An error occurred when invoking the authorization extension. —> Microsoft.ReportingServices.Diagnostics.Utilities.AuthorizationExtensionException: An error occurred when invoking the authorization extension. —> System.ServiceModel.FaultException`1[[System.ServiceModel.ExceptionDetail, System.ServiceModel, Version=184.108.40.206, Culture=neutral, PublicKeyToken=b77a5c561934e089]]: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED)) at Microsoft.ReportingServices.Library.ReportingService2005Impl.GetRoleProperties(String Name, Task& Tasks, String& Description) at Microsoft.ReportingServices.WebServer.ReportingService2005.GetRoleProperties(String Name, Task& Tasks, String& Description). Will retry..
- Ensure local Administrator for current user performing the installation and the SSRS service account
- Run the following from Powershell:
Get-userrole | format-List Name, ID > c:\ListRoles.txt
- Check to change the value from 1 to 0 in field IsSystem on table dbo.UserRole on the Operation Manager Report Operator
- Check this article and add the SDK account to the Windows Authorization Group http://support.microsoft.com/kb/938627/en-us
- Check this article for the same error code: http://blogs.technet.com/b/momteam/archive/2012/10/24/kb-installation-or-upgrade-of-reporting-for-system-center-2012-operations-manager-fails-with-error-0xffffffff.aspx
- Notice that both Dundas dll’s DundasRSGauge.dll and DundasWebChart.dll and the Microsoft.EnterpriseManagement.Reporting.Code.dll and Microsoft.EnterpriseManagement.Reporting.Security.dll are missing from the installation
- An error occurred when invoking the authorization extension. (rsAuthorizationExtensionError)
- Could not load file or assembly ‘Microsoft.EnterpriseManagement.OperationsManager.Common, Version=7.0.5000.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35’ or one of its dependencies. The system cannot find the file specified.
20. Processing final configuration
NOTE: After this, you can perform a Reset of SSRS to its default configuration. Utilize ResetSRS.exe from within the SupportTools folder of the System Center 2012 Operations Manager Media by running this within an Admistrator Command prompt:
Command = \SupportTools\ResetSRS.exe MSSQLSERVER
What is changed?
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\Reporting]
<assemblyIdentity name="Microsoft.ReportingServices.ProcessingCore" publicKeyToken="89845dcd8080cc91" culture="neutral" />
<bindingRedirect oldVersion="220.127.116.11" newVersion="10.0.0.0" />
<add key="ManagementGroupId" value="23b322bf-6965-70c9-3503-a0b44868dd47" />
<Extension Name="Windows" Type="Microsoft.EnterpriseManagement.Reporting.Security.GroupAuthorization, Microsoft.EnterpriseManagement.Reporting.Security">
<Extension Name="Windows" Type="Microsoft.EnterpriseManagement.Reporting.Security.GroupAuthentication, Microsoft.EnterpriseManagement.Reporting.Security">
<ReportItem Name="EnterpriseManagementChartControl" Type="Dundas.ReportingServices.DundasChart,MicrosoftRSChart" />
<ReportItem Name="DundasGaugeControl" Type="Dundas.ReportingServices.DundasGauge,DundasRSGauge" />
I noticed that my Hyper-V lab machines all had a loopback adapter by default. I had to perform the registry fix on my Hyper-V guests to DisableLoopbackCheck in order to complete the installation (details in ‘This results in secton’). With the loopback adapter fix in mind, I had a hunch and decided to search some VMWare forums that got some ideas flowing on that front. This seemed to be two separate issues.
What I found:
In ESX, you must use the E1000E NIC for OpsMgr 2012 SP1 using SQL 2012 SP1 on top of Windows Server 2012. You cannot use the high performance NIC called VMXNET3, even though both have support for Server 2012.
Note: You MUST also disable the loopback check. You will also see failures of SQL, SSRS and OpsMgr components when using CNAMEs without disabling the loopback adapter. I also saw failures when not using CNAMEs and installing OpsMgr Reporting until we performed the workaround.
The root cause is related to:
- Loopback Adapter causing OS installation problems with Virtualized guests including Hyper-V and ESX
- Related to the following article although it is older (I could not find one for 2012): http://support.microsoft.com/?kbid=926642
- It appears that in ESX 5.1 (for Windows Server 2012 support) the loopback adapter doesn’t appear to exist in VMXNET3 resulting in localhost failing to perform local name resolution.
- This workaround from the KB on VMWare’s site does not work in this case: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004779
This results in:
- Name resolution issues
- Unable to utilize CNAMEs
- Unable to create SQL Directories
- Unable to copy specific files during the installation process
- Unable to UNC using DB CNAME
- Able to UNC using Server name
- Unable to install SQL RDBMS or SSRS
- Unable to Create OpsMgr databases during Install
- Incomplete Installation of OpsMgr RS with Dundas and Enterprise Management dll’s not being copied during the step ‘ Set OM Extensions’ and ‘Set Security Extensions”
Workaround performed on all virtual ESX guests:
- If you are using ESX 5.1 for Server 2012 Support, change the ESX guest adapter to use the E1000E NIC
- This workaround from the KB on VMWare’s site does not work in this case, you must use E1000E: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004779
Workaround performed on all machines:
- Open regedit.exe
- Go to HKEY_LOCAL_MACHINE / SYSTEM / CurrentControlSet / Control / Lsa
- Create REG_DWORD with name DisableLoopbackCheck and value 1
- Disable the NIC
- Re-enable the NIC
- Try to UNC to the following paths and rerun the installation
- Note: This article was not the resolution for loopback in VMWare, instead it was changing the NIC to E1000E and performing the DisableLoopbackCheck registry edit:
Method 2: Disable the authentication loopback check
Re-enable the behavior that exists in Windows Server 2003 by setting the DisableLoopbackCheck registry entry in the
subkey to 1. To set the DisableLoopbackCheck registry entry to 1, follow these steps on the client computer:
1. Click Start, click Run, type regedit, and then click OK.
2. Locate and then click the following registry subkey: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa
3. Right-click Lsa, point to New, and then click DWORD Value.
4. Type DisableLoopbackCheck, and then press ENTER.
5. Right-click DisableLoopbackCheck, and then click Modify.
6. In the Value data box, type 1, and then click OK.
7. Exit Registry Editor.
8. Restart the computer.
Note You must restart the server for this change to take effect. By default, loopback check functionality is turned on in Windows Server 2003 SP1, and the DisableLoopbackCheck registry entry is set to 0 (zero). The security is reduced when you disable the authentication loopback check, and you open the Windows Server 2003 server for man-in-the-middle (MITM) attacks on NTLM.