Skip to content

Commit ffc9bc7

Browse files
author
george
committed
update
1 parent 0525530 commit ffc9bc7

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

82 files changed

+3638
-5827
lines changed

‎.travis.yml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
11
language: python
2-
sudo: false
32

43
before_install:
54
- git clone https://github.com/GoogleCloudPlatform/gcloud-python-wheels

‎CONTRIBUTING.rst‎

Lines changed: 11 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -164,19 +164,23 @@ Running Regression Tests
164164
bamboo-shift-455).
165165
- ``GCLOUD_TESTS_DATASET_ID``: The name of the dataset your tests connect to.
166166
This is typically the same as ``GCLOUD_TESTS_PROJECT_ID``.
167-
- ``GOOGLE_APPLICATION_CREDENTIALS``: The path to a JSON key file;
168-
see ``regression/app_credentials.json.sample`` as an example. Such a file
169-
can be downloaded directly from the developer's console by clicking
170-
"Generate new JSON key". See private key
167+
- ``GCLOUD_TESTS_CLIENT_EMAIL``: The email for the service account you're
168+
authenticating with
169+
- ``GCLOUD_TESTS_KEY_FILE``: The path to an encrypted key file.
170+
See private key
171171
`docs <https://cloud.google.com/storage/docs/authentication#generating-a-private-key>`__
172-
for more details.
172+
for explanation on how to get a private key.
173173

174174
- Examples of these can be found in ``regression/local_test_setup.sample``. We
175175
recommend copying this to ``regression/local_test_setup``, editing the values
176176
and sourcing them into your environment::
177177

178178
$ source regression/local_test_setup
179179

180+
- The ``GCLOUD_TESTS_KEY_FILE`` value should point to a valid path (relative or
181+
absolute) on your system where the key file for your service account can
182+
be found.
183+
180184
- For datastore tests, you'll need to create composite
181185
`indexes <https://cloud.google.com/datastore/docs/tools/indexconfig>`__
182186
with the ``gcloud`` command line
@@ -190,10 +194,8 @@ Running Regression Tests
190194
$ export CLOUDSDK_PYTHON_SITEPACKAGES=1
191195

192196
# Authenticate the gcloud tool with your account.
193-
$ SERVICE_ACCOUNT_EMAIL="some-account@developer.gserviceaccount.com"
194-
$ P12_CREDENTIALS_FILE="path/to/keyfile.p12"
195-
$ gcloud auth activate-service-account $SERVICE_ACCOUNT_EMAIL \
196-
> --key-file=$P12_CREDENTIALS_FILE
197+
$ gcloud auth activate-service-account $GCLOUD_TESTS_CLIENT_EMAIL \
198+
> --key-file=$GCLOUD_TESTS_KEY_FILE
197199

198200
# Create the indexes
199201
$ gcloud preview datastore create-indexes regression/data/ \
@@ -283,33 +285,6 @@ should either be:
283285
``EXTRA_TOX_ENVS``. This value is unencrypted in ``gcloud-python-wheels``
284286
to make ongoing maintenance easier.
285287

286-
Shared Code with External Projects
287-
----------------------------------
288-
289-
In order to enable high-quality HTTP transfer of large data (for Cloud
290-
Storage), we have temporarily included some code from the
291-
`apitools <https://github.com/craigcitro/apitools>`__ library.
292-
293-
We have chosen to partially include it, rather than include it as
294-
a dependency because
295-
296-
- The library is not yet included on PyPI.
297-
- The library's ``protorpc`` dependency is not Python 3 friendly, so
298-
would block us from Python 3 support if fully included.
299-
300-
The included code in lives in the
301-
`_gcloud_vendor <https://github.com/GoogleCloudPlatform/gcloud-python/tree/master/_gcloud_vendor>`__
302-
directory. It is a snapshot of the ``e5a5c36e24926310712d20b93b4cdd02424a81f5``
303-
commit from the main project imported in
304-
``4c27079cf6d7f9814b36cfd16f3402455f768094``. In addition to the raw import,
305-
we have customized (e.g. rewriting imports) for our library:
306-
307-
- ``334961054d875641d150eec4d6938f6f824ea655``
308-
- ``565750ee7d19742b520dd62e2a4ff38325987284``
309-
- ``67b06019549a4db8168ff4c5171c9d701ac94a15``
310-
- ``f4a53ee64fad5f3d7f29a0341e6a72a060edfcc2``
311-
312-
313288
Contributor License Agreements
314289
------------------------------
315290

‎README.rst‎

Lines changed: 63 additions & 97 deletions
Original file line numberDiff line numberDiff line change
@@ -1,38 +1,23 @@
11
Google Cloud Python Client
22
==========================
33

4-
Python idiomatic client for Google Cloud Platform services.
4+
Python idiomatic client for Google Cloud Platform services.
55

6-
|build| |coverage| |pypi|
7-
-------------------------
8-
9-
- `Homepage`_
6+
|build| |coverage|
7+
------------------
108

11-
.. _Homepage: https://googlecloudplatform.github.io/gcloud-python/
9+
- `Homepage <https://googlecloudplatform.github.io/gcloud-python/>`__
1210

1311
This client supports the following Google Cloud Platform services:
1412

15-
- `Google Cloud Datastore`_
16-
- `Google Cloud Storage`_
17-
18-
.. _Google Cloud Datastore: https://cloud.google.com/products/cloud-datastore/
19-
.. _Google Cloud Storage: https://cloud.google.com/products/cloud-storage/
13+
- `Google Cloud
14+
Datastore <https://cloud.google.com/products/cloud-datastore/>`__
15+
- `Google Cloud
16+
Storage <https://cloud.google.com/products/cloud-storage/>`__
2017

21-
If you need support for other Google APIs, check out the
22-
`Google APIs Python Client library`_.
23-
24-
.. _Google APIs Python Client library: https://github.com/google/google-api-python-client
25-
26-
Versioning
27-
----------
28-
29-
This library follows `Semantic Versioning`_.
30-
31-
.. _Semantic Versioning: http://semver.org/
32-
33-
It is currently in major version zero (``0.y.z``), which means that anything
34-
may change at any time and the public API should not be considered
35-
stable.
18+
If you need support for other Google APIs, check out the `Google APIs
19+
Python Client
20+
library <https://github.com/google/google-api-python-client>`__.
3621

3722
Quickstart
3823
----------
@@ -44,60 +29,55 @@ Quickstart
4429
Google Cloud Datastore
4530
----------------------
4631

47-
Google `Cloud Datastore`_ is a fully managed, schemaless database for
48-
storing non-relational data. Cloud Datastore automatically scales with
49-
your users and supports ACID transactions, high availability of reads and
50-
writes, strong consistency for reads and ancestor queries, and eventual
51-
consistency for all other queries.
52-
53-
.. _Cloud Datastore: https://developers.google.com/datastore/
54-
55-
See the `Google Cloud Datastore docs`_ for more details on how to activate
56-
Cloud Datastore for your project.
32+
`Google Cloud Datastore <https://developers.google.com/datastore/>`__ is
33+
a fully managed, schemaless database for storing non-relational data.
34+
Cloud Datastore automatically scales with your users and supports ACID
35+
transactions, high availability of reads and writes, strong consistency
36+
for reads and ancestor queries, and eventual consistency for all other
37+
queries.
5738

58-
.. _Google Cloud Datastore docs: https://developers.google.com/datastore/docs/activate
39+
See the `Google Cloud Datastore
40+
docs <https://developers.google.com/datastore/docs/activate>`__ for more
41+
details on how to activate Cloud Datastore for your project.
5942

60-
See the ``gcloud-python`` API `datastore documentation`_ to learn how to interact
61-
with the Cloud Datastore using this Client Library.
62-
63-
.. _datastore documentation: https://googlecloudplatform.github.io/gcloud-python/datastore-api.html
43+
See `the gcloud-python API
44+
documentation <https://googlecloudplatform.github.io/gcloud-python/datastore-api.html>`__
45+
to learn how to interact with the Cloud Datastore using this Client
46+
Library.
6447

6548
.. code:: python
6649
6750
from gcloud import datastore
68-
datastore.set_default_connection()
69-
datastore.set_default_dataset_id()
51+
dataset = datastore.get_dataset('dataset-id-here',
52+
'long-email@googleapis.com',
53+
'/path/to/private.key')
7054
# Then do other things...
71-
from gcloud.datastore.entity import Entity
72-
from gcloud.datastore.key import Key
73-
from gcloud.datastore.query import Query
74-
query = Query(kind='EntityKind')
75-
entity = Entity(key=Key('EntityKind'))
55+
query = dataset.query().kind('EntityKind')
56+
entity = dataset.entity('EntityKind')
7657
7758
Google Cloud Storage
7859
--------------------
7960

80-
Google `Cloud Storage`_ allows you to store data on Google infrastructure with
81-
very high reliability, performance and availability, and can be used to
82-
distribute large data objects to users via direct download.
83-
84-
.. _Cloud Storage: https://developers.google.com/storage/
61+
`Google Cloud Storage <https://developers.google.com/storage/>`__ allows
62+
you to store data on Google infrastructure with very high reliability,
63+
performance and availability, and can be used to distribute large data
64+
objects to users via direct download.
8565

8666
You need to create a Google Cloud Storage bucket to use this client
87-
library. Follow the steps on the `Google Cloud Storage docs`_
67+
library. Follow the steps on the `Google Cloud Storage
68+
docs <https://developers.google.com/storage/docs/cloud-console#_creatingbuckets>`__
8869
to learn how to create a bucket.
8970

90-
.. _Google Cloud Storage docs: https://developers.google.com/storage/docs/cloud-console#_creatingbuckets
91-
92-
See the ``gcloud-python`` API `storage documentation`_ to learn how to connect
93-
to Cloud Storage using this Client Library.
94-
95-
.. _storage documentation: https://googlecloudplatform.github.io/gcloud-python/storage-api.html
71+
See `the gcloud-python API
72+
documentation <https://googlecloudplatform.github.io/gcloud-python/storage-api.html>`__
73+
to learn how to connect to the Cloud Storage using this Client Library.
9674

9775
.. code:: python
9876
9977
import gcloud.storage
100-
bucket = gcloud.storage.get_bucket('bucket-id-here', 'project-id')
78+
bucket = gcloud.storage.get_bucket('bucket-id-here',
79+
'long-email@googleapis.com',
80+
'/path/to/private.key')
10181
# Then do other things...
10282
key = bucket.get_key('/remote/path/to/file.txt')
10383
print key.get_contents_as_string()
@@ -109,60 +89,46 @@ Contributing
10989

11090
Contributions to this library are always welcome and highly encouraged.
11191

112-
See `CONTRIBUTING`_ for more information on how to get started.
113-
114-
.. _CONTRIBUTING: https://github.com/GoogleCloudPlatform/gcloud-python/blob/master/CONTRIBUTING.rst
92+
See `CONTRIBUTING <CONTRIBUTING.rst>`__ for more information on how to
93+
get started.
11594

11695
Supported Python Versions
11796
-------------------------
11897

11998
We support:
12099

121-
- `Python 2.6`_
122-
- `Python 2.7`_
100+
- `Python 2.6 <https://docs.python.org/2.6/>`__
101+
- `Python 2.7 <https://docs.python.org/2.7/>`__
123102

124103
We plan to support:
125104

126-
- `Python 3.3`_
127-
- `Python 3.4`_
105+
- `Python 3.3 <https://docs.python.org/3.3/>`__
106+
- `Python 3.4 <https://docs.python.org/3.4/>`__
128107

129-
.. _Python 2.6: https://docs.python.org/2.6/
130-
.. _Python 2.7: https://docs.python.org/2.7/
131-
.. _Python 3.3: https://docs.python.org/3.3/
132-
.. _Python 3.4: https://docs.python.org/3.4/
108+
Supported versions can be found in our ``tox.ini``
109+
`config <https://github.com/GoogleCloudPlatform/gcloud-python/blob/master/tox.ini>`__.
133110

134-
Supported versions can be found in our ``tox.ini`` `config`_.
111+
We explicitly decided not to support
112+
`Python 2.5 <https://docs.python.org/2.5/>`__ due to
113+
`decreased usage <https://caremad.io/2013/10/a-look-at-pypi-downloads/>`__ and
114+
lack of continuous integration
115+
`support <http://blog.travis-ci.com/2013-11-18-upcoming-build-environment-updates/>`__.
135116

136-
.. _config: https://github.com/GoogleCloudPlatform/gcloud-python/blob/master/tox.ini
137-
138-
We explicitly decided not to support `Python 2.5`_ due to `decreased usage`_
139-
and lack of continuous integration `support`_.
140-
141-
.. _Python 2.5: https://docs.python.org/2.5/
142-
.. _decreased usage: https://caremad.io/2013/10/a-look-at-pypi-downloads/
143-
.. _support: http://blog.travis-ci.com/2013-11-18-upcoming-build-environment-updates/
144-
145-
We also explicitly decided to support Python 3 beginning with version
146-
3.3. Reasons for this include:
117+
We also explicitly decided to support Python 3 beginning with version 3.3.
118+
Reasons for this include:
147119

148120
- Encouraging use of newest versions of Python 3
149-
- Taking the lead of prominent open-source `projects`_
150-
- `Unicode literal support`_ which allows for a cleaner codebase that
151-
works in both Python 2 and Python 3
152-
153-
.. _projects: http://flask.pocoo.org/docs/0.10/python3/
154-
.. _Unicode literal support: https://www.python.org/dev/peps/pep-0414/
121+
- Taking the lead of prominent open-source
122+
`projects <http://flask.pocoo.org/docs/0.10/python3/>`__
123+
- Unicode literal `support <https://www.python.org/dev/peps/pep-0414>`__ which
124+
allows for a cleaner codebase that works in both Python 2 and Python 3.
155125

156126
License
157127
-------
158128

159-
Apache 2.0 - See `LICENSE`_ for more information.
160-
161-
.. _LICENSE: https://github.com/GoogleCloudPlatform/gcloud-python/blob/master/LICENSE
129+
Apache 2.0 - See `LICENSE <LICENSE>`__ for more information.
162130

163131
.. |build| image:: https://travis-ci.org/GoogleCloudPlatform/gcloud-python.svg?branch=master
164-
:target: https://travis-ci.org/GoogleCloudPlatform/gcloud-python
132+
:target: https://travis-ci.org/GoogleCloudPlatform/gcloud-python
165133
.. |coverage| image:: https://coveralls.io/repos/GoogleCloudPlatform/gcloud-python/badge.png?branch=master
166-
:target: https://coveralls.io/r/GoogleCloudPlatform/gcloud-python?branch=master
167-
.. |pypi| image:: https://img.shields.io/pypi/v/gcloud.svg
168-
:target: https://pypi.python.org/pypi/gcloud
134+
:target: https://coveralls.io/r/GoogleCloudPlatform/gcloud-python?branch=master

‎_gcloud_vendor/apitools/base/py/http_wrapper.py‎

Lines changed: 9 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -6,14 +6,13 @@
66
"""
77

88
import collections
9+
import httplib
910
import logging
1011
import socket
1112
import time
1213
import urlparse
1314

1415
import httplib2
15-
from six.moves import http_client
16-
from six.moves import range
1716

1817
from _gcloud_vendor.apitools.base.py import exceptions
1918
from _gcloud_vendor.apitools.base.py import util
@@ -29,10 +28,10 @@
2928
RESUME_INCOMPLETE = 308
3029
TOO_MANY_REQUESTS = 429
3130
_REDIRECT_STATUS_CODES = (
32-
http_client.MOVED_PERMANENTLY,
33-
http_client.FOUND,
34-
http_client.SEE_OTHER,
35-
http_client.TEMPORARY_REDIRECT,
31+
httplib.MOVED_PERMANENTLY,
32+
httplib.FOUND,
33+
httplib.SEE_OTHER,
34+
httplib.TEMPORARY_REDIRECT,
3635
RESUME_INCOMPLETE,
3736
)
3837

@@ -130,7 +129,7 @@ def MakeRequest(http, http_request, retries=5, redirections=5):
130129
url_scheme = urlparse.urlsplit(http_request.url).scheme
131130
if url_scheme and url_scheme in http.connections:
132131
connection_type = http.connections[url_scheme]
133-
for retry in range(retries + 1):
132+
for retry in xrange(retries + 1):
134133
# Note that the str() calls here are important for working around
135134
# some funny business with message construction and unicode in
136135
# httplib itself. See, eg,
@@ -141,15 +140,15 @@ def MakeRequest(http, http_request, retries=5, redirections=5):
141140
str(http_request.url), method=str(http_request.http_method),
142141
body=http_request.body, headers=http_request.headers,
143142
redirections=redirections, connection_type=connection_type)
144-
except http_client.BadStatusLine as e:
143+
except httplib.BadStatusLine as e:
145144
logging.error('Caught BadStatusLine from httplib, retrying: %s', e)
146145
exc = e
147146
except socket.error as e:
148147
if http_request.http_method != 'GET':
149148
raise
150149
logging.error('Caught socket error, retrying: %s', e)
151150
exc = e
152-
except http_client.IncompleteRead as e:
151+
except httplib.IncompleteRead as e:
153152
if http_request.http_method != 'GET':
154153
raise
155154
logging.error('Caught IncompleteRead error, retrying: %s', e)
@@ -162,7 +161,7 @@ def MakeRequest(http, http_request, retries=5, redirections=5):
162161
break
163162
logging.info('Retrying request to url <%s> after status code %s.',
164163
response.request_url, response.status_code)
165-
elif isinstance(exc, http_client.IncompleteRead):
164+
elif isinstance(exc, httplib.IncompleteRead):
166165
logging.info('Retrying request to url <%s> after incomplete read.',
167166
str(http_request.url))
168167
else:

0 commit comments

Comments
 (0)