Microsoft Project Server, SharePoint, Security and other cool things…

Rolly Perreaux

   LinkedIn - MVP-Press Networking Group   @RollyPerreaux   RSS Feed    

 

How to Fix Project Server Events Service and Queue Service Event ID: 7000 Error (Graphical Step-by-Step)

Posted on September 06, 2009 by - 4,200 views
facebooktwittergoogle_plusredditpinterestlinkedin

In my last posting, I showed you how to Slipstream SP2 and Cumulative Updates into the Project Server 2007 RTM bits.

Well an interesting thing happened after I rebooted the Project Server for the first time.
I received the following Service Control Manager error:

Microsoft CRL

So now I’m thinking, Uh Oh
That’s not good

So as a good Administrator, I log in and check the Event Viewer for my clues as to what happened. I immediately notice in the System logs the following errors:

Microsoft CRL2

Here they are in more detail starting from the bottom up:

Event Type:    Error
Event Source:    Service Control Manager
Event Category:    None
Event ID:    7009
Date:        9/7/2009
Time:        9:24:42 PM
User:        N/A
Computer:    PS07
Description:
Timeout (30000 milliseconds) waiting for the Microsoft Office Project Server Events Service service to connect.



Event Type:    Error
Event Source:    Service Control Manager
Event Category:    None
Event ID:    7000
Date:        9/7/2009
Time:        9:24:42 PM
User:        N/A
Computer:    PS07
Description:
The Microsoft Office Project Server Events Service service failed to start due to the following error:
The service did not respond to the start or control request in a timely fashion.



Event Type:    Error
Event Source:    Service Control Manager
Event Category:    None
Event ID:    7009

Date:        9/7/2009
Time:        9:24:42 PM

User:        N/A
Computer:    PS07
Description:
Timeout (30000 milliseconds) waiting for the Microsoft Office Project Server Queue Service service to connect.



Event Type:    Error
Event Source:    Service Control Manager
Event Category:    None
Event ID:    7000

Date:        9/7/2009
Time:        9:24:42 PM

User:        N/A
Computer:    PS07
Description:
The Microsoft Office Project Server Queue Service service failed to start due to the following error:
The service did not respond to the start or control request in a timely fashion.

 

So my next logical step was to check out the Services console and verify if in fact the Events Service and Queue Service did not start. Sure enough the services did not start:

Microsoft CRL9 

So now off I go to search the Internet for a solution.

My first stop, the Office Project Server 2007 Solution Center web site.
Very nice site. Even has a Browse by Topic area

Microsoft CRL10

But alas, nothing listed for my problem.

Then I remembered that there was a problem with Project Server 2007 after applying Service Pack 1.
So I searched using the following query:

SP1 Project Server Services Do Not Start

And there it was, a posting from the Microsoft Communities newsgroup:
http://www.microsoft.com/communities/newsgroups/en-us/default.aspx?dg=microsoft.public.project.server&tid=0ed80ffd-693b-427b-ba57-5ab819554055&p=1

About half way through the posts there was a reply from David CB

At the beginning your answer seems to me really weird. But using a sniffer I
checked that the Queue service was trying to contact crl.microsoft.com, that
is, the certificates revocation list from Microsoft. What the service really
wants is the file
http://crl.microsoft.com/pki/crl/products/CodeSignPCA.crl
So, if you cannot give the security account access to the Internet (as it is
my case), you can publish this file in any accessible web site of the intranet
with the same name and forward the request of the service to this site.
After the queue service starts for the first time, it no longer connect to
the crl site, so it can be deleted.

BINGO!!  The Certificates Revocation List (CRL)
Now it makes sense

When Service Pack 1 was deployed it installs new .NET packages for varying Project Server components. The Microsoft Office Project Server Queue Service service and the Microsoft Office Project Server Events Service service are trying to load their .NET packages. These packages are cryptographically signed with a Microsoft Code Signing PCA certificate which are already installed on the Windows Server (under Trusted Root Certification Authorities / Certificates as shown below:

Microsoft CRL11

So the first time the .NET packages are loaded, the application (in this case the Event Service) needs to validates the certificate of the package. The certificate validation process does the following:

  1. Checks to ensure that the Certificate chains to a Trusted Root
  2. Checks to ensure that the Certificate is time valid
  3. Checks the CRL

Well we know from the screenshot of the Certificates console that the certificate is chained to a trusted root and the time is valid. So that leaves us with checking the CRL and that’s where the problem lies.

Now most servers in a production environment will have Internet access based on a Firewall policy.
However, if you work in the Government, Military or Police, these environments are typically locked down.
Nothing coming in or out to the Internet.

Well guess what else?
Virtual Machines with Project Server with static IP address set to Local Only.
Which is my case

So what do you do?

Well for me, I decided to test for myself what would happen if the Project Server would have Internet access.

But first I needed to know the IP address of crl.microsoft.com so that I know what to look for in the Firewall logs

Microsoft CRL4

 

Next I add another virtual network card to the virtual machine.
This card is connected to my notebook’s Intel Gigabit NIC.

Then I went to my Firewall and started to logged all activity.

And finally I turned on the virtual machine.

Here’s what I saw on the log:

Microsoft CRL5 

 

Client Agent Destination Host Name Client IP Destination IP HTTP
Method
URL
Microsoft-CryptoAPI/5.131.3790.3959 crl.microsoft.com 192.168.1.16 24.244.3.43 GET http://crl.microsoft.com/pki/crl/products/CSPCA.crl
Microsoft-CryptoAPI/5.131.3790.3959 crl.microsoft.com 192.168.1.16 24.244.3.43 GET http://crl.microsoft.com/pki/crl/products/CSPCA.crl
Microsoft-CryptoAPI/5.131.3790.3959 crl.microsoft.com 192.168.1.16 24.244.3.57 GET http://crl.microsoft.com/pki/crl/products/CSPCA.crl
Microsoft-CryptoAPI/5.131.3790.3959 crl.microsoft.com 192.168.1.16 24.244.3.43 GET http://crl.microsoft.com/pki/crl/products/CodeSignPCA2.crl

 

So it appears that the CryptoAPI client agent actually downloaded two CRL files

  • CSPCA.crl
  • CodeSignPCA2.crl

I then rebooted my virtual machine and checked the Event viewer and now both services started normally

So we now know that allowing the Project Server computer to have Internet access completes the certificate validation process and also completes the Project Server SP1 configuration.

But if you can’t give Project Server Internet access?

 

Step 1 – Manually download the CRL files on a computer that has Internet access.

 

Step 2 – Copy the CRL files to the Non-Internet Project Server to a temporary location like C:\Temp.

Microsoft CRL12

 

Step 3 – Open a Command Prompt as Administrator and run the following commands:

cd\Temp

certutil -addstore CA CSPCA.crl

certutil -addstore CA CodeSignPCA2.crl

Please Note

Copying the CRLs to C:\Program Files\Microsoft Office Servers\12.0\Bin will make no difference as the CertUtil.exe utility will run anywhere

You should see the following messages:

Microsoft CRL13

 

Step 4 – Create a Certificates Console using MMC and verify CRLs were added

  1. Click Start, Run and type MMC and click OK.
  2. Maximize all console windows and click File, Add/Remove Snap-in.
  3. In the Add/Remove Snap-in dialog box, click Add.
  4. In the Add Standalone Snap-in dialog box, select Certificates and click Add.
  5. In the Certificates snap-in dialog box, select Computer Account and click Finish.
  6. In the Select Computer dialog box, select Local computer and click Finish.
  7. In the Add Standalone Snap-in dialog box, click Close.
  8. In the Add/Remove Snap-in dialog box, click OK.
  9. From the Console Root folder expand Certificates (Local Computer) –> Intermediate Certification List –> Certificate Revocation List and you should see the two Microsoft CRLs listed:

Microsoft CRL14

 

Step 5 – Close all Windows and Reboot Project Server

After rebooting and logging in,  open the Event Viewer and review the Application logs.

You should see two Information entries. One for pjevtsvc (Event Service) and one for ProjectQueueService stating “Service started successfully

Microsoft CRL7

Now you’re done!

PS

Many thanks goes out to Paul Adare, MVP – Identity Lifecycle Manager, for clarifying the certificate validation process to me. I greatly appreciate your help Paul.

Comments are closed.




↑ Top