Microsoft Exchange Rpc Client Access Not Starting Sbs 2011 Remote

If your machine is also a global catalog server, (Note: As mine is an SBS server, and the only DC I don’t really have a choice.)You may find that the ‘Microsoft Exchange RPC Client Access’ service is not running, start it manually then attempt the import again.

  • Microsoft Exchange RPC Client Access Service Does Not Start. Home » Exchange » Microsoft Exchange RPC Client Access Service Does Not Start. KB ID 0000802 Problem. I build a new SBS 2011 environment last week, and I had just put on the new digital certificate, and as no one was using it, I simply rebooted the SBS Server and went for a.
  • The Module DLL 'C: Program Files Microsoft Exchange Server V14 Bin kerbauth.dll' could not be loaded because of a configuration problem. The current configuration only supports loading images built for a x86 processor architecture.

So you got a call from help-desk stating they have a lot of users complaining they cannot access their mailbox or performance is terribly slow? Ok, What next?I blogged onthe background process involved where exchange processes transactions and commit to the database.

Let’s take a look at troubleshooting this issue.

Once you ensure all mailbox stores are mounted, the next step would be to do this.

Start — Run —Perfmon

You see in the figure:

RPC requests –> Clients are requesting for data

RPC Operations/sec –>Server is acknowledging the client request.

Let’s look at how exchange prepares itself to handle client requests.

  1. Microsoft Exchange Information Store service (store.exe) starts on a server
  2. 500 RPC threads is allocated to the service upon registering with the RPC service
  3. Clients connect and disconnect to individual RPC threads as they perform operations from their outlook like reading and sending e-mail, creating appointments and tasks, and viewing calendars.
  4. The MSExchangeISRPC Requests performance counter indicates how many threads are currently in use (“owned” by clients).
  5. The MSExchangeISRPC Operations/sec performance counter reflects the number of operations the server received in the past second.
  6. If the number of RPC requests increases steadily over time, it is a clear indication that the server cannot process client operations quickly enough.
  7. When the MSExchangeISRPC Requests performance counter reaches 500, all RPC threads have been exhausted, and clients will be unable to submit new requests to the server until all operations on an existing thread have been completed and that thread is released.

Right now, you are facing an issue mentioned in step7, maybe that’s why you’re reading this.

How to troubleshoot?

Microsoft Exchange Rpc Client Access Not Starting Sbs 2011 Remotely

I am listing down the various conditions the 2 counters will be and what they mean.

MSExchangeISRPC Requests and MSExchangeISRPC Operations/sec are either low or at zero

  • Bottleneck might not be on the server.
  • Something external to the server is preventing information from reaching the server. Review Active Directory performance, network performance, client configuration etc.
  • Get information about the GC the server is talking to using the NLTest command: nltest /SC_QUERY:<Type Domain name here>
  • Reset the GC to another one using: nltest /sc_reset:<Type Domain name here>
  • To get a list of DC’s for the domain, use: nltest /DCLIST:<Type Domain name here>
Remote

MSExchangeISRPC Requests is increasing rapidly while MSExchangeISRPC Operations/sec remains fairly stable (Horizontal line)

  • The server is unable to handle the existing workload.
  • Verify hardware components like physical memory, storage, and processor capability
  • Open task manager and look at “All processes”. On an Exchange server, apart from the Store.exe process, nothing should consume memory or CPU no matter what
  • Verify that the system drive (usually C Drive) has atleast 2-5GB free space
  • Decrease the number of users on the server
  • See if online defrag is getting completed on all database’s. Believe me, it plays a huge role when the DB is too large and has a lot of whitespace in it. The best way to identify this is to check eventvwr for events related to Online defragmentation like 700, 701, 702 etc.
  • Run ExMon to see if it is a specific user is a top talker. In ExMon, you can again sort client requests by RPC requests
Microsoft Exchange Rpc Client Access Not Starting Sbs 2011 Remote

MSExchangeISRPC Requests is steadily increasing but MSExchangeISRPC Operations/sec steadily decreases

  • Exchange server is the source of the bottleneck
  • Verify that the system drive (usually C Drive) has atleast 2-5GB free space
  • Open task manager and look at “All processes”. On an Exchange server, apart from the Store.exe process, nothing should consume memory or CPU no matter what
  • Might be caused as something is preventing the Information Store from completing RPC operations, and the associated RPC threads remain allocated permanently
  • More and more threads are allocated; the server has fewer available threads for new operations, so the number of new operations decreases.
  • If the server eventually reaches 500 outstanding RPC requests, new RPC operations will be refused. This is generally caused by either a serious physical resource shortage (memory or disk) or a processing issue within the Information Store or an integrated component (antivirus, journaling, and so on).
  • Run ExMon to see if it is a specific user is a top talker. In ExMon, you can again sort client requests by RPC requests
  • See if online defrag is getting completed on all database’s. Believe me, it plays a huge role when the DB is too large and has a lot of whitespace in it. The best way to identify this is to check eventvwr for events related to Online defragmentation like 700, 701, 702 etc.

If ExMon shows you high RPC requests from one user mailbox, open his mailbox and look for corruption in calendar. Sort the calendar items by “list” and then by “size”. Any calendar item which is more than 8-10MB is not considered as a “calendar” item. Delete it and notify the organizer that you had to delete it since the item got corrupted.

Additionally, look for messages stuck in Outbox for that user. There might be a zip file sitting and looping again and again. Use this:

Get-Mailbox -ResultSize Unlimited | Get-MailboxFolderStatistics |Where-Object {$_.Name -eq “Outbox” -and $_.ItemsInFolder -gt ‘0’ } | Select-Object Identity, FolderType, ItemsinFolder, FolderSize | Export-CSV “C:Outbox.csv”

This is a checklist of RPC counters and their expected values:

Microsoft Exchange Rpc Client Access Not Starting Sbs 2011 Remote Access

CounterValues
MSExchangeISRPC Requests lower than 70
MSExchangeISRPC Operations/secAlways higher than RPC Requests
MSExchangeISRPC Averaged Latency Less than 25 milliseconds
MSExchangeISRPC Num. of Slow Packets Less than 2

If you have a monitoring system in place, it is mandatory to tune it to throw alerts when the counters cross the allowed limit.

Microsoft Exchange Rpc Client Access Not Starting Sbs 2011 Remote Start

Ratish Nair
MVP Exchange
Team@ MSExchange

Keywords: Exchange server performance issues, troubleshoot performance issues with exchange, Exchange RPC request are spiking high, RPC operations not processed on exchange