Skip to content

clouddns - client quota failing, unable to handle list in response.#5085

Closed
cycomike wants to merge 3 commits intogoogleapis:masterfrom
cycomike:master
Closed

clouddns - client quota failing, unable to handle list in response.#5085
cycomike wants to merge 3 commits intogoogleapis:masterfrom
cycomike:master

Conversation

@cycomike
Copy link
Copy Markdown

@cycomike cycomike commented Mar 21, 2018

Hi
clouddns client started failing yesterday because API quota response now includes a list where a string is expected.
Added condition to skip value if it's a list.

Original Error:

  File "/usr/lib/python2.7/site-packages/google/cloud/dns/client.py", line 60, in quotas
    for key, value in resp['quota'].items()
  File "/usr/lib/python2.7/site-packages/google/cloud/dns/client.py", line 61, in <dictcomp>
    if key != 'kind'}
TypeError: int() argument must be a string or a number, not 'list'

Caused By:
Addition of the following in the API quota response:
(u'whitelistedKeySpecs', [{u'kind': u'dns#dnsKeySpec', u'algorithm': u'ecdsap256sha256'}, {u'kind': u'dns#dnsKeySpec', u'algorithm': u'ecdsap384sha384'}, {u'kind': u'dns#dnsKeySpec', u'keyLength': 2048, u'algorithm': u'rsasha256'}, {u'kind': u'dns#dnsKeySpec', u'keyLength': 1024, u'keyType': u'zoneSigning', u'algorithm': u'rsasha256'}, {u'kind': u'dns#dnsKeySpec', u'keyLength': 2048, u'algorithm': u'rsasha512'}, {u'kind': u'dns#dnsKeySpec', u'keyLength': 1024, u'keyType': u'zoneSigning', u'algorithm': u'rsasha512'}])

Kind regards, Mike

@googlebot
Copy link
Copy Markdown

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here (e.g. I signed it!) and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers
  • Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the project maintainer to go/cla#troubleshoot.
  • The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
  • The email used to register you as an authorized contributor must also be attached to your GitHub account.

@googlebot googlebot added the cla: no This human has *not* signed the Contributor License Agreement. label Mar 21, 2018
@cycomike
Copy link
Copy Markdown
Author

I signed it

@googlebot
Copy link
Copy Markdown

CLAs look good, thanks!

@googlebot googlebot added cla: yes This human has signed the Contributor License Agreement. and removed cla: no This human has *not* signed the Contributor License Agreement. labels Mar 21, 2018
@tseaver tseaver added the api: dns Issues related to the Cloud DNS API. label Mar 26, 2018
@tseaver
Copy link
Copy Markdown
Contributor

tseaver commented Mar 26, 2018

@cycomike The API docs for projects.get don't show any list values: can you report the keys for which you are seeing a list? Best would be to show the whole response body, with any project-identifying information redacted.

In addition, if the API really has changed to returning lists, then we should likely reflect that as well, which would mean doing a type-sniff to figure out what to do with the whitelistedKeySpecs entries.

@tseaver
Copy link
Copy Markdown
Contributor

tseaver commented Mar 26, 2018

@lukesneeringer @jonparrott Do we have a PoC for the back-end DNS team?

@theacodes
Copy link
Copy Markdown
Contributor

Not that I know of, reaching out internally.

@cycomike
Copy link
Copy Markdown
Author

@tseaver Here is the full response for the quota information which now has the whitelistedKeySpecs entity. Yes you are right that something probably should be done with it.

{u'quota': {u'kind': u'dns#quota', u'rrsetAdditionsPerChange': 1000, u'resourceRecordsPerRrset': 100, u'totalRrdataSizePerChange': 100000, u'dnsKeysPerManagedZone': 4, u'rrsetsPerManagedZone': 10000, u'whitelistedKeySpecs': [{u'kind': u'dns#dnsKeySpec', u'algorithm': u'ecdsap256sha256'}, {u'kind': u'dns#dnsKeySpec', u'algorithm': u'ecdsap384sha384'}, {u'kind': u'dns#dnsKeySpec', u'keyLength': 2048, u'algorithm': u'rsasha256'}, {u'kind': u'dns#dnsKeySpec', u'keyLength': 1024, u'keyType': u'zoneSigning', u'algorithm': u'rsasha256'}, {u'kind': u'dns#dnsKeySpec', u'keyLength': 2048, u'algorithm': u'rsasha512'}, {u'kind': u'dns#dnsKeySpec', u'keyLength': 1024, u'keyType': u'zoneSigning', u'algorithm': u'rsasha512'}], u'rrsetDeletionsPerChange': 1000, u'managedZones': 10000}, u'kind': u'dns#project', u'number': u'1234567', u'id': u'XXXXXXX'}

@seils
Copy link
Copy Markdown

seils commented Apr 25, 2018

Looks like the whitelistedKeySpecs[] addition is documented as part of https://cloud.google.com/dns/api/v1beta2/projects.

@tseaver
Copy link
Copy Markdown
Contributor

tseaver commented Jul 13, 2018

@seils Interesting that the field would be removed in the v1 API docs for projects.

@theacodes Any status on a PoC?

@cycomike cycomike requested a review from theacodes as a code owner August 9, 2018 22:43
@yoshi-automation yoshi-automation added the 🚨 This issue needs some love. label Feb 7, 2019
@tseaver
Copy link
Copy Markdown
Contributor

tseaver commented Feb 19, 2019

@cycomike I'm sorry we've let your patch languish so long. At this point, it seems likely to me that we won't be able to provide proper support for zones created with the whitelistedKeySpecs property under the v1beta2 API.

@tseaver tseaver closed this Feb 19, 2019
@tseaver tseaver added needs work This is a pull request that needs a little love. and removed 🚨 This issue needs some love. labels Feb 19, 2019
@SiddhiVinaykam
Copy link
Copy Markdown

Cloud dns api still throws error on accessing project quotas due to the whitelistedKeySpecs. The error thrown was :-

  key: int(value) for key, value in resp["quota"].items() if key != "kind"
  TypeError:int() argument must be a string, a bytes-like object or a number, not 'list

How to handle this ?

@cycomike
Copy link
Copy Markdown
Author

Cloud dns api still throws error on accessing project quotas due to the whitelistedKeySpecs. The error thrown was :-

  key: int(value) for key, value in resp["quota"].items() if key != "kind"
  TypeError:int() argument must be a string, a bytes-like object or a number, not 'list

How to handle this ?

Hi

The changes were never merged to fix this issue, the patch I created is still sitting in this pull request.
4437bc4

NOTE: It does not do anything with the whitelistedkeys, so if you need that data additional changes would be required.

@tseaver
Copy link
Copy Markdown
Contributor

tseaver commented Jun 12, 2020

Two points about this PR:

  • Adding "correct" support for the whitelistedKeySpecs resource field would be far preferable to just skipping over it. I realize that involves a lot more work. FWIW, I cannot see any reason why that field is under the top-level quotas field: I think the best choice is to adjust the quotas method to skip that field completely (by name, rather than by type), and then add a new whitelisted_key_specs method which returns the values from the nested field, if present.
  • The code for the DNS library has been moved to a separate repository. Any follow-ups should happen there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: dns Issues related to the Cloud DNS API. backend cla: yes This human has signed the Contributor License Agreement. needs work This is a pull request that needs a little love.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants