Abstract:
Online applications adoption, and success are driven by a multitude of factors among them the service response time. This is natural as users tend to prefer a faster service than a slower. However, it is challenging to deliver consistently fast response times due to performance variability inherent to the infrastructure running the application; This performance variability causes a fraction of user requests to experience unusual latency called tail latency. In this work, a Linear Regression Based Replica Selection Algorithm is proposed. The regression model helps to estimate how long a specific query is going to take to be serviced, and based on this information, a server with more or less resources is chosen to service the query. Experiments done using data generated by a fleet of buses show that the proposed approach is successful in reducing the higher percentiles latency up to 30 % in some cases while not impacting negatively the throughput.
