Investigating a mystery from our case files Here’s how that all looks in diagram form:
#NEW RELIC TIMESLICE CODE#
Some Lua code running in the nginx process (called the “bouncer”) consults a Redis database to see whether the API key associated with the request has exceeded its worker time budget.The rough sequence goes something like this: This means it ultimately gets timing measurements from nginx. We designed API Overload Protection to operate without the cooperation of the application to which it controls access, in order to guard against malfunctions of the application. If that time becomes excessive, the tool will automatically restrict that account for a period of time. This is an internal, resource-limiting tool we built to keep track of Unicorn worker time used per account. That’s why we created API Overload Protection.
This means that if a single API user can send MTS queries that are expensive enough via API, it can impact MTS, and thus all of MTS’ consumers, including APM’s UI. Here’s a slightly expanded version of what a single API request might look like:īoth the API hosts and the UI hosts rely on our Metric Timeslice Service, which provides access to timeslice data for consumption across all New Relic products (MTS also has its own techniques to prevent overloads). Despite the separation of API and UI traffic, a heavy API user can still break the UI by impacting shared downstream dependencies. Of course, no system is foolproof, so let’s refocus on the separation of API/UI worker pools. With this approach, t he specific thing that prevents API users from consuming all available worker time is that API requests are served by a dedicated pool of hardware resources, meaning that we always have reserved capacity for UI requests. So the normal flow for requests looks like this: To serve static files without tying up a Unicorn worker, we use nginx as a reverse proxy in front of Unicorn. So, to be able to process N requests concurrently, we need to run at least N Unicorn worker processes. Unicorn is completely single-threaded a single process pulls a request from a socket, generates a response, writes it back to the socket, and then waits for the next request to arrive. This simple ‘one request per process at a time’ concurrency model isn’t new or fancy, but it is tried and true, and offers some important benefits, including strong fault isolation (failures or crashes in one worker process are guaranteed to be scoped to only that process). APM uses the Unicorn web server, which spawns a number of worker processes, each of which can handle exactly one request at a time. Specifically, we have developed a number of techniques to limit resource usage and to mitigate the effects of API overloading (note that the techniques discussed here do not apply to the ingest pipeline through which we receive data from our customers).įor example, we separated the pool of hardware for the New Relic APM API from that of its UI.
Since then we’ve made a lot of progress in protecting ourselves from overloading of all our APIs. We originally introduced the New Relic REST API to help our customers access data directly from our products. (Yes, it’s long, but there are pictures! You should at least look at the pictures!) New Relic’s API approach We hope it provides insight into how you can deal with API overuse in your own shop. This post addresses some of the ways New Relic deals with the issue internally, and offers a deep dive into how we dealt with a particularly puzzling API usage issue.
#NEW RELIC TIMESLICE HOW TO#
However, as more and more customers take advantage of these APIs, companies are being forced to grapple with how to appropriately limit API usage to prevent intentional or inadvertent overloading. APIs are an increasingly popular way for businesses to interact with customers and partners without having to build specific features for every interaction.
0 kommentar(er)
