I need a web application to be able to operate at very high number of requests per second, while keeping a response time pretty low (less than ~70ms).

For testing purposes I booted up 3 c4.xlarge ec2 instances and put them in an elastic load balancer.

I have a method that looks like this.

[HttpPost]
[Route("api/test/rps")]
public IHttpActionResult TestRequestsPerSecond(MediumSizedObject testObj)
{
    return new StatusCodeResult(HttpStatusCode.NoContent, Request);
}

I expected ONE of these instances to be able to handle many thousand requests per second (since it's essentially doing nothing). To my surprise, I'm having trouble handling 300 requests per second without the CPU getting to high, and the requests starting to queue up.

Here's a look at the performance of one of the instances at around 240 requests per second.

Performance Monitor

As you can see I have 19 queued requests already, and the processor is 78%. (Also my context switches is jumping to around 8-9K which seems high).

This is happening across all 3 servers.

Things I've changed on the server

machine.config relevant lines

 <processModel autoConfig="true"
                   maxWorkerThreads="100"
                   maxIoThreads="100"
                   minWorkerThreads="50"
                   minIoThreads="50" />
    <httpRuntime
        minFreeThreads="176"
        minLocalRequestFreeThreads="152"
        />

production web config (possibly) relevant lines

<system.web>
    <sessionState mode="Off" />
    <authentication mode="None" />
    <compilation targetFramework="4.5" />
    <httpRuntime targetFramework="4.5" />
</system.web>
<system.webServer>
    <modules>
      <remove name="FormsAuthentication" />
      <remove name="PassportAuthentication" />
      <remove name="Profile" />
      <remove name="AnonymousIdentification" />
      <remove name="WindowsAuthentication" />
    </modules>
    <handlers>
      <remove name="ExtensionlessUrlHandler-Integrated-4.0" />
      <remove name="OPTIONSVerbHandler" />
      <remove name="TRACEVerbHandler" />
      <add name="ExtensionlessUrlHandler-Integrated-4.0" path="*." verb="*" type="System.Web.Handlers.TransferRequestHandler" preCondition="integratedMode,runtimeVersionv4.0" />
    </handlers>
</system.webServer>

I tried with a default machine.config and it was only a bit worse.

For some reason at about 130 requests per second, we're only at 30% CPU, and nearly no requests get queued.

There has to be some magic setting that I'm missing. Is this par for the course with IIS? Is it possibly WebAPI 2 that's to blame here? (I'm down to test out Nancy or ServiceStack if anyone thinks those would help at all).

Do I need bigger instances?

Any suggestions or things to look for, tweak would be great.

(I had the WebApi Action return a Task to see if it made any difference, but it didn't)

Related posts

Recent Viewed