What Is Response Time Theory

Let’s start by defining what we mean by a Response Time model.

Response Time Model

The Response Time model is a simple way of calculating the overall Queue Length and Queuing Time for a given system for a given set of workload conditions. The Response time Model is generally used for back of the envelope calculations and to get a sense of direction. We wouldn’t recommend using Response Time models to make investment or major architectural design decisions. For more detailed performance modelling when validating architectural decisions or investment decisions we would recommend investing in building thorough Queuing Theory models.

To be able to use the Response Time model at minimum you would need to know two of the three variables mentioned below:

• Uavg - Average System Utilization
• X - System Throughput
• St - Service Time

M is always assumed to be known since it’s the number of CPU’s (also called Servers when referring to performance models) within the system. This is a design assumption and is assumed to be a known quantity.

• M - Number of Servers

Let’s take a look at all of the equations and then look at a real world example:

• U_avg = [ X * S_t ] / M ……………..  [ U_avg = Average Utilization,  X = Throughput, S_t = Service Time, M = Number of Servers ]
• R_t-cpu = St / [ 1 –  (U_avg)^M ]………..  [ Rt = CPU Response Time, U_avg = Average Utilization,  S_t = Service Time, M = Number of Servers ]
• R_t = Q_t + S_t ………………………..  [ S_t = Service Time, R_t = Response Time, Q_t = Queuing Time ]
• Q_t = R_t – S_t………………………  [ R_t = CPU Response Time, Q_t = Queuing Time, S_t= Service Time ]
• Q = X * Q_t …………………………..  [ Q = Queue Length, Q_t = Queuing Time, S_t = Service Time ]

Example: The best way to understand these laws is to look at an example and work our way through the various equations:

Lets assume we observe a system with 4 CPUs for a period of 1 Hour or 3600 seconds out of which the system has been busy for 1000 seconds. For the duration of observation there were 500 successfully completed transactions. Lets use the very basic Response Time theory to do some quick back of the envelope transactions and understand what the Queuing Times would be and how long would the Queue have been on the system.

• T = 3600 seconds [ Total Time of Observation ]
• B = 1000 seconds [ Busy Time ]
• C = 500  [ Completions ]

• M = 4 [ Number of CPUs or Servers ]

• We know from our fundamental equations :

• St = B / C .........[ St = Service Time, B = Busy Time, C = Completions ]
• Therefore : St = 1000 / 500 = 2 seconds

• We also know from our fundamental equations:

• X = C / T ...........[ X = Throughput, C = Completions, T = Total Time of Observation ]
• X = 500 / 3600 = 0.139 TPS

• We also know:
  • U_avg = [ X * S_t] ………[ Uavg = Average Utilization ]
  • Uavg = [ 0.139 * 2 ] / 4 = 0.0695
  • The Average Utilization Per Processor is 0.0695
• We also know :
  • Rt = S_t / [ 1 - (U_avg)^M ] ……. [ Rt = CPU Response Time ]
  • Rt = 2 / [ 1 - (0.0695)⁴]
  • Rt = 2 / [ 1 - 0.00002 ]
  • Rt = 2.000 seconds
• We also know:
  • R_t = Q_t + S_t …….[ Qt = Queuing Time ]
  • Qt = Rt - St
  • Qt = 2.0 - 2.0  [ St = 2s, Rt = 2.0s)
  • Qt = 0 seconds
• We also know:

• Q = X * Q_t [ Q = Queue Length ]
• Q = 0.139 * 0
• Q = 0

Thus, using Response Time theory we have figured out that the actual Queue size Per CPU is 0 for the above set of variables.

We hope this example has illustrated the simplicity and power of Response Time Theory to estimate potential system bottlenecks. As we have mentioned before, only use Response Time models for quick back of the envelope calculations when you need to get a gut feel and direction. For a more detailed system performance model we would encourage you to consider using Erlang C model or Queuing Theory models which we have covered in other sections.

Additional Resources

• Dr. Rajesh’s papers on Performance Testing and Performance Requirements Gathering
• Microsoft : Load Testing of Web Applications
• Microsoft : Patterns & Practices – Performance Testing Guidance
• The Server Side : Performance Engineering – A Practitioners approach to Performance Testing
• Dr. Connie Smith on Performance Engineering
• Dr. Rajesh Mansharamani on Performance Engineerirng
• Wikipedia on Performance Engineering

Hope you’ve enjoyed the content in this section at Practical Performance Analyst and have learnt something new. Please help us grow the community by taking a moment and sharing this content with rest of community using your preferred Social Media Platform (links provided below). We are looking for the bright spark and if you think you have what it takes to build and grow this community reach out to me by Sending us an email.

Trevor Warren is passionate about challenging the status-quo and finding reasons to innovate. Over the past 16 years he has been delivering complex systems, has worked with very large clients across the world and constantly is looking for opportunities to bring about change. Trevor constantly strives to combine his passion for delivering outcomes with his ability to build long lasting professional relationships. You can learn more about the work he does at LinkedIn. You can download a copy of his CV at VisualCV. Visit the Github page for details of the projects he’s been hacking with.