Fix #2723: Retry on HTTP 400 failedPrecondition#2731
Fix #2723: Retry on HTTP 400 failedPrecondition#2731codeXsidd wants to merge 1 commit intogoogleapis:mainfrom
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request enhances the robustness of the HTTP client by implementing a retry mechanism for specific HTTP 400 Bad Request errors. Previously, only certain 403 Forbidden errors triggered retries; now, the system will also retry requests that fail with a 400 status code if the underlying reason is identified as a Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
There was a problem hiding this comment.
Code Review
This pull request adds retry logic for HTTP 400 errors with a failedPrecondition reason, which resolves issue #2723. The changes in googleapiclient/http.py correctly implement this by extending the _should_retry_response function. The accompanying tests in tests/test_http.py validate the new behavior. My review includes a suggestion to refactor the retry logic for better maintainability and a recommendation to enhance test coverage for all supported retry reasons. Overall, this is a good improvement.
| if resp_status == http_client.FORBIDDEN: | ||
| LOGGER.warning('Encountered 403 Forbidden with reason "%s"', reason) | ||
| # Only retry on rate limit related failures. | ||
| if reason in ("userRateLimitExceeded", "rateLimitExceeded"): | ||
| return True | ||
| elif resp_status == http_client.BAD_REQUEST: | ||
| LOGGER.warning('Encountered 400 Bad Request with reason "%s"', reason) | ||
| # Only retry on precondition failures. | ||
| if reason in ("failedPrecondition", "preconditionFailed"): | ||
| return True |
There was a problem hiding this comment.
For better maintainability and to make it easier to add more retryable status codes in the future, consider refactoring this if/elif block to be more data-driven. You can use a dictionary to map status codes to their retryable reasons and log messages.
| if resp_status == http_client.FORBIDDEN: | |
| LOGGER.warning('Encountered 403 Forbidden with reason "%s"', reason) | |
| # Only retry on rate limit related failures. | |
| if reason in ("userRateLimitExceeded", "rateLimitExceeded"): | |
| return True | |
| elif resp_status == http_client.BAD_REQUEST: | |
| LOGGER.warning('Encountered 400 Bad Request with reason "%s"', reason) | |
| # Only retry on precondition failures. | |
| if reason in ("failedPrecondition", "preconditionFailed"): | |
| return True | |
| RETRYABLE_INFO = { | |
| http_client.FORBIDDEN: { | |
| "reasons": ("userRateLimitExceeded", "rateLimitExceeded"), | |
| "message": 'Encountered 403 Forbidden with reason "%s"', | |
| }, | |
| http_client.BAD_REQUEST: { | |
| "reasons": ("failedPrecondition", "preconditionFailed"), | |
| "message": 'Encountered 400 Bad Request with reason "%s"', | |
| }, | |
| } | |
| if resp_status in RETRYABLE_INFO: | |
| info = RETRYABLE_INFO[resp_status] | |
| LOGGER.warning(info["message"], reason) | |
| if reason in info["reasons"]: | |
| return True |
| def test_retry_400_failed_precondition(self): | ||
| num_retries = 2 | ||
| resp_seq = [ | ||
| ({"status": "400"}, FAILED_PRECONDITION_RESPONSE), | ||
| ({"status": "200"}, "{}") | ||
| ] | ||
| http = HttpMockSequence(resp_seq) | ||
| model = JsonModel() | ||
| uri = "https://www.googleapis.com/someapi/v1/collection/?foo=bar" | ||
| method = "POST" | ||
| request = HttpRequest( | ||
| http, | ||
| model.response, | ||
| uri, | ||
| method=method, | ||
| body="{}", | ||
| headers={"content-type": "application/json"}, | ||
| ) | ||
|
|
||
| sleeptimes = [] | ||
| request._sleep = lambda x: sleeptimes.append(x) | ||
| request._rand = lambda: 10 | ||
|
|
||
| request.execute(num_retries=num_retries) | ||
|
|
||
| self.assertEqual(1, len(sleeptimes)) | ||
| self.assertEqual(10 * 2 ** 1, sleeptimes[0]) |
There was a problem hiding this comment.
The new test test_retry_400_failed_precondition is great for covering the failedPrecondition reason. However, the implementation in http.py also handles the preconditionFailed reason. It would be beneficial to add a test case for preconditionFailed as well to ensure full coverage of the new logic.
You could achieve this by parameterizing the test or by adding a separate test method for the preconditionFailed case.
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Fixes #<issue_number_goes_here> 🦕