What Is Throttling and How to Work Around It
Throttling is a purposeful action performed by the service provider to slow things down. It's usually in the form of disconnecting connections, rejecting requests, or slowing the transfer of data. The overarching goal of the service provider is often to maintain the performance and quality of their service for their users. In other words, the provider does not want anyone to consume an inordinate amount of resources.
Throttling is usually a function of the number of requests, the frequency of the requests, and the size of the requests that you are making. For instance, acquiring a mailbox that contains emails with large attachments is more likely to trigger throttling than acquiring numerous emails without attachments. Similarly, making requests to the same mailbox simultaneously using different tools would result in a heightened frequency of requests and trigger throttling quickly.
How Do I Know If Throttling Is in Effect?
If the service provider starts throttling, you would see that FEC pauses the acquisition temporarily and resumes it again after a waiting period.
When using Gmail, you might see an error along the lines of " Too many concurrent requests for user" or "Resource has been exhausted (e.g. check quota)" with an error code of 403 or 429. You can find more information about Google's API and Rate limits below:
https://developers.google.com/gmail/api/v1/reference/quota
An Exchange server might issue a response along the lines of " The server cannot service this request right now. Try again later."
These error messages would be recorded in FEC's exception log.
What Does FEC Do About Throttling?
- FEC is efficient with requests. We work hard to avoid repetitive requests and asking for more data than necessary.
- Because FEC uses Gmail API, you can download each message only once as opposed to multiple times for each label. This cuts down the amount of data you need to download and helps with throttling. You can find our Gmail output options here: https://docs.metaspike.com/article/36-gmail-output-options
- Once throttling kicks in, FEC retries items multiple times, reducing the remaining data set over time. The default setting is to automatically retry 10 times. You can change this setting from FEC's preferences.
- FEC determines an optimal amount of time to pause and then resumes the acquisition automatically. In the case of Gmail, FEC attempts to parse Google's responses that tell FEC exactly when to try again.
- You can manually stop and resume an acquisition session at a later time.
After the acquisition is complete, you can also have FEC send you a notification email. The goal is to minimize the babysitting required during long acquisitions.
What Can You Do About Throttling?
There are a few things you can do to minimize the chances of throttling or work around it if it is already in effect:
1. Try to get the specifications right from the beginning and avoid having to acquire the same mailbox multiple times.
2. Do not acquire the same mailbox multiple times using different tools—especially at the same time.
3. If possible, advise the end-user to keep activity in the mailbox to a minimum until the acquisition is complete. The API and rate limits apply to all email clients, not just to FEC.
4. If possible and when appropriate, perform an in-place search before the acquisition to narrow down the data set.
5. When acquiring a large Gmail mailbox, acquire the emails (Gmail) and calendar events (Google Calendar) separately, as two distinct acquisitions.
6. If you are acquiring Drive attachments and their revisions, consider acquiring them separately. You can isolate messages with Drive attachments for a supplemental acquisition using the search query has:drive.
7. Acquisition of Drive revisions and folders is highly likely to trigger throttling. If possible, we recommend using these options in a targeted manner. For instance, an initial acquisition that includes Drive attachments only can be followed up with a more comprehensive acquisition with Drive revisions for a subset of the emails based on directions from the legal team.
8. If FEC is in a waiting pattern due to throttling, do not cancel the acquisition and restart it manually to force FEC to try again. Hammering the mailbox this way would likely exacerbate the issue.
9. When acquiring from an Exchange server or O365, consider using a service account via impersonation.
10. If your target is an on-premises Exchange server, consider (responsibly) adjusting the server's throttling policy.
Change user throttling settings for specific users: Exchange 2013 Help | Microsoft Docs
11. API and rate limits are often calculated at the mailbox level. When acquiring multiple mailboxes, consider switching your targets frequently to give each mailbox time to cool off. You can accomplish this in FEC using batch acquisitions with a smaller retry count such as 2.
12. If you find that throttling is slowing your acquisition down significantly, consider starting with a narrowly-targeted acquisition to get the legal team's review or your investigation started, and following up with broader supplemental acquisitions using FEC's Differential Acquisition feature.