feat(dbapi): add retry_aborts_internally option to Connection#16498
feat(dbapi): add retry_aborts_internally option to Connection#16498waiho-gumloop wants to merge 1 commit intogoogleapis:mainfrom
Conversation
Add a retry_aborts_internally flag to the DBAPI Connection class and the connect() function. When set to False, aborted transactions raise RetryAborted directly from commit() instead of entering the internal statement-replay retry loop. This aligns with RETRY_ABORTS_INTERNALLY in the Spanner JDBC driver and avoids nested retry loops when the application manages its own transaction retry logic. Fixes googleapis#16491
There was a problem hiding this comment.
Code Review
This pull request introduces the retry_aborts_internally configuration for Spanner DBAPI connections, enabling users to opt-out of automatic internal transaction retries. This is particularly useful for applications that manage their own retry logic to avoid nested loops. The implementation includes a new connection parameter, a property with a setter that prevents changes during active transactions, and updated logic in the commit method. A review comment correctly identifies that the RetryAborted exception is used without being imported in connection.py, which would result in a NameError at runtime.
| self._transaction_helper.retry_transaction() | ||
| self.commit() | ||
| else: | ||
| raise RetryAborted(str(exc)) from exc |
There was a problem hiding this comment.
The RetryAborted exception is used here but it does not appear to be imported in this file. This will cause a NameError at runtime when a transaction is aborted and internal retries are disabled. Please ensure RetryAborted is imported from google.cloud.spanner_dbapi.exceptions at the top of the file.
Summary
Add a
retry_aborts_internallyflag to the DBAPIConnectionclass and theconnect()function. When set toFalse, aborted transactions raiseRetryAborteddirectly fromcommit()instead of entering the internal statement-replay retry loop.Changes
Connection.__init__: Acceptretry_aborts_internallyparameter (defaultTrue)Connection.retry_aborts_internally: Property getter/setter with guard against mid-transaction changesConnection.commit(): Check_retry_aborts_internallybefore entering the replay loop; when disabled, wrapAbortedinRetryAbortedfor PEP 249 complianceconnect(): Pass-throughretry_aborts_internallytoConnectionRationale
Why the internal retry was added
The DBAPI's statement-replay retry was introduced to support Django and other PEP 249 ORMs. These frameworks build transactions incrementally through individual
cursor.execute()calls -- the DBAPI layer sees a sequence of statements but has no callable representing the full transaction. When Spanner aborts a transaction, the only option is to replay all recorded statements and verify checksums to ensure read consistency.Why applications may not want the internal retry
Applications that implement their own transaction retry logic -- wrapping the entire transaction in a callable and re-invoking it with a fresh session on abort -- do not need transparent statement replay. When both layers retry simultaneously, the result is nested retry loops that cause:
In production with 10 concurrent writers, disabling the internal retry reduced abort-to-recovery time from ~18s to ~0.05s and improved success rates from ~55% to 100%.
Precedent
This aligns with existing functionality in other Spanner clients:
RETRY_ABORTS_INTERNALLYconnection propertytrueNewReadWriteStmtBasedTransactionvsReadWriteTransactionretry_aborts_internallyconstructor/connect parameterTrueUsage
Tests
test_retry_aborts_internally_defaults_truetest_retry_aborts_internally_set_falsetest_retry_aborts_internally_settertest_retry_aborts_internally_setter_while_transaction_activetest_commit_retries_internally_when_enabledtest_commit_raises_retry_aborted_when_internal_retry_disabledtest_connect_retry_aborts_internally_defaulttest_connect_retry_aborts_internally_falseFixes #16491