Agent discovery and push troubleshooting in OpsMgr 2007

OpsMgr 2007 Agent troubleshooting:
There is a GREAT graphical display of the Agent discovery and push process, taken from:
http://blogs.technet.com/momteam/archive/2007/12/10/how-does-computer-discovery-work-in-opsmgr-2007.aspx

Agent Prerequisites:

1. Supported Operating System Version (see below)
2. Windows Installer 3.1
3. MSXML 6 Parser

Agent push requirements (including firewall ports):

• The account being used to push the agent must have local admin rights on the targeted agent machine.
• The following ports must be open:
• RPC endpoint mapper Port number: 135 Protocol: TCP/UDP
• *RPC/DCOM High ports (2000/2003 OS) Ports 1024-5000 Protocol: TCP/UDP
• *RPC/DCOM High ports (2008 OS) Ports 49152-65535 Protocol: TCP/UDP
• NetBIOS name service Port number: 137 Protocol: TCP/UDP
• NetBIOS session service Port number: 139 Protocol: TCP/UDP
• SMB over IP Port number: 445 Protocol: TCP
• MOM Channel Port number: 5723 Protocol: TCP/UDP
• The following services must be set:
• Display Name: Netlogon Started Auto Running
• **Display Name: Remote Registry Started Auto Running
• Display Name: Windows Installer Started Manual Running
• Display Name: Automatic Updates Started Auto Running

*The RPC/DCOM High ports are required for RPC communications. This is generally why we don't recommend/support agent push in a heavily firewalled environment, because opening these port ranges creates a potential security issue that negates the firewall boundary. For more information:
http://support.microsoft.com/kb/154596/
http://support.microsoft.com/default.aspx?scid=kb;EN-US;929851
Important: Don’t change the RPC high ports without have an deep understanding of your environment and the potential impact !!!

**Not required for agent push, but required for some management packs.

• The remote management server must be able to connect to the remote agent machine via WMI and execute WMI Query "Select * from Win32_OperatingSystem". WMI must be running, and healthy, and allowing remote connections.
• The management server must be able to connect to the targeted agent machine via \\servername\c$

Logging:

• When pushing an agent from a management server, a log will be written in the event of a failure to: \Program Files\System Center OpsMgr\AgentManagement\AgentLogs\ on the Management Server.
• The log on an agent is not enabled by default (like MOM 2005) when using agent push. If you manually install an agent using the MSI – it will place a verbose logfile at C:\documents and settings\%user%\local settings\temp\momagent.log

To troubleshoot agent push with a verbose log – you need to enable verbose MSI logging: http://support.microsoft.com/kb/314852/en-us

Common Agent Push errors:
Below are some common push failures. Also see my troubleshooting table for more details: Console based Agent Deployment Troubleshooting table

The MOM Server detected that the following services on computer "(null);NetLogon" are not running. These services are required for push agent installation. To complete this operation, either start the required services on the computer or install the MOM agent manually by using MOMAgent.msi located on the product CD. Operation: Agent Install
Remote Computer Name: dc1.opsmgr.net
Install account: OPSMGR\localadmin
Error Code: C000296E
Error Description: Unknown error 0xC000296E

Solution: Netlogon service is not running. It must be set to auto/started

The MOM Server detected that the Windows Installer service (MSIServer) is disabled on computer "dc1.opsmgr.net". This service is required for push agent installation. To complete this operation on the computer, either set the MSIServer startup type to "Manual" or "Automatic", or install the MOM agent manually by using MOMAgent.msi located on the product CD.

Operation: Agent Install
Install account: OPSMGR\localadmin
Error Code: C0002976
Error Description: Unknown error 0xC0002976

Solution: Windows Installer service is not running or set to disabled – set this to manual or auto and start it.

The Agent Management Operation Agent Install failed for remote computer dc1.opsmgr.net.
Install account: OPSMGR\localadmin
Error Code: 80070643
Error Description: Fatal error during installation.
Microsoft Installer Error Description:
For more information, see Windows Installer log file "C:\Program Files\System Center Operations Manager 2007\AgentManagement\AgentLogs\DC1AgentInstall.LOG
C:\Program Files\System Center Operations Manager 2007\AgentManagement\AgentLogs\DC1MOMAgentMgmt.log" on the Management Server.

Solution: Enable the automatic Updates service…. Install the agent – then disable the auto-updates service if desired.


Additional Info:
There are sub-components to the OpsMgr Agent installer service
1. The service is a standard NT Service. The service also handles registration/un-registration of DCOM object that has logic for handling MSI/MSP.
2. The DCOM object takes directive from the module on OpsMgr Server, this object provides asynchronously installing/uninstalling/updating OpsMgr. It also returns list of currently installed QFEs, verifies pre-requisites like channel connectivity before completing agent install. It handles multi-homing of agent, and reads agent parameters such as version, install dir, etc.
3. RPC is used to establish a connection to the target machine, SMB is used to copy the source files over.
4. WMI is used to check prerequisites.

Agents Inside a Trust Boundary
Discovery:
Discovery requires that the TCP 135 (RPC), RPC range, and TCP 445 (SMB) ports remain open and that the SMB service is enabled.
Installation:
After a target device has been discovered, an agent can be deployed to it. Agent installation requires:

• Opening Remote procedure call (RPC) ports beginning with endpoint mapper TCP 135 and the Server Message Block (SMB) port TCP/UDP 445.
• Enabling the File and Printer Sharing for Microsoft Networks and the Client for Microsoft Networks services (this ensures that the SMB port is active).
• If enabled, Windows Firewall Group Policy settings for Allow remote administration exception and Allow file and printer sharing exception must be set to Allow unsolicited incoming messages from: to the IP address and subnets for the primary and secondary Management Servers for the agent. For more information, see How to Configure the
• Windows Firewall to Enable Management of Windows-Based Computers from the Operations Manager 2007 Operations Console.
• An account that has local administrator rights on the target computer.
• Windows Installer 3.1. To install, see article 893803 in the Microsoft Knowledge Base (http://go.microsoft.com/fwlink/?LinkId=86322).
• Microsoft Core XML services (MSXML) 6 on the Operations Manager product installation media in the \msxml sub directory.

Ongoing Management:
Ongoing management of an agent requires that the TCP 135 (RPC), RPC range, and TCP 445 (SMB) ports remain open and that the SMB service remains enabled.

Kevin Holman