Skip to content
This repository was archived by the owner on Oct 1, 2025. It is now read-only.

Commit 027c0f0

Browse files
committed
add Blobstore & GCS samples
1 parent 1c48980 commit 027c0f0

16 files changed

Lines changed: 687 additions & 3 deletions

File tree

README.md

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,8 @@ Datastore | 2 [⇒ 3 [⇒ 6]] | Moving off App Engine `ndb` makes your app
8585
Memcache | [12 ⇒] 13 | Moving off App Engine `memcache` makes your apps more portable, so the **Module 13** Cloud Memorystore (for Redis) migration is _recommended_ for those using `memcache`. Those unfamiliar with `memcache` should do **Module 12** first to add its usage to the sample app.
8686
Cloud Functions | 11 | Cloud Functions does not support Python 2, so after the Module 1 migration, you need to upgrade your app to Python 3 before attempting this migration, recommended if you have a very small App Engine app, or it has only one function/feature.
8787
Cloud Run | 4 or 5 | **Module 4** covers migrating to Cloud Run with Docker. Those unfamiliar with containers or do not wish to create/maintain a `Dockerfile` should do **Module 5**. Those doing **Module 4** will get additional information about Cloud Run in **Module 5** not covered in **Module 4**.
88+
Blobstore | [15 ⇒] 16 | Moving off App Engine `blobstore` makes your apps more portable, so the **Module 16** Cloud Storage migration is _recommended_ for those using `blobstore`. Those unfamiliar with `blobstore` should do **Module 15** first to add its usage to the sample app.
89+
general migration | [6 ⇒] 10 [⇒ 14] | This series is more generic and not targeting a specific feature migration, but rather if you need to migrate your App Engine apps from one running project to another. It starts with **Module 6** if you need to migrate your code, say from Datastore to Firestore. **Module 10** is if you need to migrate your data from one project to another, and finally, **Module 14** is after you're done migrating your app, your data, or both, and need to migrate a running service on one GCP project to another.
8890

8991

9092
## Migration modules
@@ -108,8 +110,11 @@ Module | Topic | Video | Codelab | START here | FINISH here
108110
9|Migrate to Python 3, Cloud Datastore & Cloud Tasks v2| _TBD_ | _TBD_ | Module 8 [code](/mod8-cloudtasks) (2.x) | Module 9 [code](/mod9-py3dstasks)
109111
10|Migrate Datastore/Firestore data to another project| _TBD_ | _N/A_ | _N/A_ | _TBD_
110112
11|Migrate to Cloud Functions| _TBD_ | [link](https://codelabs.developers.google.com/codelabs/cloud-gae-python-migrate-11-functions?utm_source=codelabs&utm_medium=et&utm_campaign=CDR_wes_aap-serverless_mgrcloudfuncs_sms_202006&utm_content=-) | Module 2 [code](/mod2b-cloudndb) (3.x) | Module 11 [code](/mod11-functions) (3.x)
111-
12|Add App Engine `memcache`| _TBD_ | [link](https://codelabs.developers.google.com/codelabs/cloud-gae-python-migrate-12-memcache?utm_source=codelabs&utm_medium=et&utm_campaign=CDR_wes_aap-serverless_mgrcloudfuncs_sms_202006&utm_content=-) | Module 1 [code](/mod1-flask) (2.x) | Module 12 [code](/mod12-memcache) (2.x) & [code](/mod12b-memcache) (3.x)
113+
12|Add App Engine `memcache`| _TBD_ | [link](https://codelabs.developers.google.com/codelabs/cloud-gae-python-migrate-12-memcache?utm_source=codelabs&utm_medium=et&utm_campaign=CDR_wes_aap-serverless_mgrmemcache_sms_202006&utm_content=-) | Module 1 [code](/mod1-flask) (2.x) | Module 12 [code](/mod12-memcache) (2.x) & [code](/mod12b-memcache) (3.x)
112114
13|Migrate to Cloud Memorystore| _TBD_ | _TBD_ | Module 12 [code](/mod12-memcache) (2.x) & [code](/mod12b-memcache) (3.x) | Module 13 [code](/mod13a-memorystore) (2.x) & [code](/mod13b-memorystore) (3.x)
115+
14|Migrate service between projects| _TBD_ | _TBD_ | _TBD_ | _TBD_
116+
15|Add App Engine `blobstore`| _TBD_ | _TBD_ | Module 0 [code](/mod0-baseline) (2.x) | Module 15 [code](/mod15-blobstore) (2.x)
117+
16|Migrate to Cloud Storage| _TBD_ | _TBD_ | Module 15 [code](/mod15-blobstore) (2.x) | Module 16 [code](/mod16a-cloudstorage) (2.x)
113118

114119

115120
### Table of contents
@@ -218,7 +223,7 @@ If there is a logical codelab to do immediately after completing one, they will
218223

219224
- [Module 11 codelab](https://codelabs.developers.google.com/codelabs/cloud-gae-python-migrate-11-functions?utm_source=codelabs&utm_medium=et&utm_campaign=CDR_wes_aap-serverless_mgrcloudfuncs_sms_202006&utm_content=-): **Migrate from App Engine to Cloud Functions**
220225
- **Optional** migration
221-
- Recommende for small apps or for breaking up large apps into multiple microservices
226+
- Recommended for small apps or for breaking up large apps into multiple microservices
222227
- Python 3 only
223228
- START: [Module 2 code - Cloud NDB](/mod2b-cloudndb) (3.x)
224229
- FINISH: [Module 11 code - Cloud Functions](/mod11-functions) (3.x)

mod13b-memorystore/main.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
ds_client = ndb.Client()
2323
HOUR = 3600
2424
REDIS_HOST = os.environ.get('REDIS_HOST', 'localhost')
25-
REDIS_PORT = os.environ.get('REDIS_PORT', 6379)
25+
REDIS_PORT = os.environ.get('REDIS_PORT', '6379')
2626
REDIS = redis.Redis(host=REDIS_HOST, port=REDIS_PORT)
2727

2828
class Visit(ndb.Model):

mod15-blobstore/.gcloudignore

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# This file specifies files that are *not* uploaded to Google Cloud Platform
2+
# using gcloud. It follows the same syntax as .gitignore, with the addition of
3+
# "#!include" directives (which insert the entries of the given .gitignore-style
4+
# file at that point).
5+
#
6+
# For more information, run:
7+
# $ gcloud topic gcloudignore
8+
#
9+
.gcloudignore
10+
11+
# Source code control files
12+
.git/
13+
.gitignore
14+
.hgignore
15+
.hg/
16+
17+
# README/text files
18+
LICENSE
19+
*.md
20+
21+
# Tests/results (not in .gitignore)
22+
noxfile.py
23+
test_translate.py
24+
pylintrc
25+
pylintrc.test
26+
27+
# most of .gitignore (except `lib`)
28+
#
29+
# Python
30+
*.py[cod]
31+
__pycache__/
32+
/setup.cfg
33+
34+
# C extensions
35+
*.so
36+
37+
# Packages
38+
*.egg
39+
*.egg-info
40+
dist
41+
build
42+
eggs
43+
.eggs
44+
parts
45+
bin
46+
var
47+
sdist
48+
develop-eggs
49+
.installed.cfg
50+
lib64
51+
*.tgz
52+
53+
# Installer logs
54+
pip-log.txt
55+
56+
# Tests/results
57+
.nox/
58+
.pytest_cache/
59+
.cache
60+
.pytype
61+
.coverage
62+
coverage.xml
63+
*sponge_log.xml
64+
system_tests/local_test_setup
65+
66+
# Mac
67+
.DS_Store
68+
69+
# IDEs/editors
70+
*.sw[op]
71+
*~
72+
.vscode
73+
.idea
74+
75+
# Built documentation
76+
docs/_build
77+
docs.metadata
78+
79+
# Virtual environment
80+
env/

mod15-blobstore/README.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Module 15 - Add usage of App Engine `blobstore` to baseline sample app (Module 0)
2+
3+
This repo folder is the corresponding code to the [Module 14 codelab](http://g.co/codelabs/pae-migrate-blobstore). The tutorial STARTs with the Python 2 code in the [Module 0 repo folder](/mod0-baseline) and leads developers through adding use of App Engine `blobstore`. Unlike other sample apps, this does not use the default Django templating system, but instead, uses Jinja2, which is supported in `webapp2_extras`.
4+
5+
Blobstore evolved into [Google Cloud Storage](https://cloud.google.com/storage), and all blobs/files created using the Blobstore API go into the default Cloud Storage bucket for your Cloud project, which is the project's ID, meaning it's your `appspot` domain name, e.g., for project `my-project`, your default bucket would be `my-project.appspot.com`. It is programmatically accessible via `google.appengine.api.app_identity.get_default_gcs_bucket_name()`.
6+
7+
The primary application file [`main.py`](main.py) writes files directly to the default bucket. If you want to customize the GCS location where App Engine writes files, see the alternative [`main-gcs.py`](main-gcs.py) file. It uses `google.appengine.api.app_identity.get_default_gcs_bucket_name()` along with the `gs_bucket_name` parameter when calling `google.appengine.ext.blobstore.create_upload_url()`.
8+
9+
Unlike some of the other migrations, Blobstore usage depends on the `webapp` and `webapp2`, so this migration must start at Module 0 rather than Module 1. One update however, is that this sample does use the [Jinja2 templating system](https://jinja.palletsprojects.com) rather than the default Django templates used in Module 0. It is supported in `webapp2` via the `webapp2_extras` package.

mod15-blobstore/app.yaml

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Copyright 2022 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
runtime: python27
16+
threadsafe: yes
17+
api_version: 1
18+
19+
handlers:
20+
- url: /.*
21+
script: main.app
22+
23+
libraries:
24+
- name: jinja2
25+
version: latest

mod15-blobstore/main-gcs.py

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# Copyright 2022 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
import webapp2
16+
from webapp2_extras import jinja2
17+
from google.appengine.api import app_identity
18+
from google.appengine.ext import blobstore, ndb
19+
from google.appengine.ext.webapp import blobstore_handlers
20+
21+
BUCKET = app_identity.get_default_gcs_bucket_name()
22+
23+
24+
class BaseHandler(webapp2.RequestHandler):
25+
'Derived request handler mixing-in Jinja2 support'
26+
@webapp2.cached_property
27+
def jinja2(self):
28+
return jinja2.get_jinja2(app=self.app)
29+
30+
def render_response(self, _template, **context):
31+
self.response.write(self.jinja2.render_template(_template, **context))
32+
33+
34+
class Visit(ndb.Model):
35+
'Visit entity registers visitor IP address & timestamp'
36+
visitor = ndb.StringProperty()
37+
timestamp = ndb.DateTimeProperty(auto_now_add=True)
38+
file_blob = ndb.BlobKeyProperty()
39+
40+
41+
def store_visit(remote_addr, user_agent, upload_key):
42+
'create new Visit entity in Datastore'
43+
Visit(visitor='{}: {}'.format(remote_addr, user_agent),
44+
file_blob=upload_key).put()
45+
46+
47+
def fetch_visits(limit):
48+
'get most recent visits'
49+
return Visit.query().order(-Visit.timestamp).fetch(limit)
50+
51+
52+
class UploadHandler(blobstore_handlers.BlobstoreUploadHandler):
53+
'Upload blob (POST) handler'
54+
def post(self):
55+
uploads = self.get_uploads()
56+
blob_id = uploads[0].key() if uploads else None
57+
store_visit(self.request.remote_addr, self.request.user_agent, blob_id)
58+
self.redirect('/', code=307)
59+
60+
61+
class ViewBlobHandler(blobstore_handlers.BlobstoreDownloadHandler):
62+
'view uploaded blob (GET) handler'
63+
def get(self, blob_key):
64+
self.send_blob(blob_key) if blobstore.get(blob_key) else self.error(404)
65+
66+
67+
class MainHandler(BaseHandler):
68+
'main application (GET/POST) handler'
69+
def get(self):
70+
self.render_response('index.html',
71+
upload_url=blobstore.create_upload_url('/upload',
72+
gs_bucket_name=BUCKET))
73+
74+
def post(self):
75+
visits = fetch_visits(10)
76+
self.render_response('index.html', visits=visits)
77+
78+
79+
app = webapp2.WSGIApplication([
80+
('/', MainHandler),
81+
('/upload', UploadHandler),
82+
('/view/([^/]+)?', ViewBlobHandler),
83+
], debug=True)

mod15-blobstore/main.py

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# Copyright 2022 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
import webapp2
16+
from webapp2_extras import jinja2
17+
from google.appengine.ext import blobstore, ndb
18+
from google.appengine.ext.webapp import blobstore_handlers
19+
20+
21+
class BaseHandler(webapp2.RequestHandler):
22+
'Derived request handler mixing-in Jinja2 support'
23+
@webapp2.cached_property
24+
def jinja2(self):
25+
return jinja2.get_jinja2(app=self.app)
26+
27+
def render_response(self, _template, **context):
28+
self.response.write(self.jinja2.render_template(_template, **context))
29+
30+
31+
class Visit(ndb.Model):
32+
'Visit entity registers visitor IP address & timestamp'
33+
visitor = ndb.StringProperty()
34+
timestamp = ndb.DateTimeProperty(auto_now_add=True)
35+
file_blob = ndb.BlobKeyProperty()
36+
37+
38+
def store_visit(remote_addr, user_agent, upload_key):
39+
'create new Visit entity in Datastore'
40+
Visit(visitor='{}: {}'.format(remote_addr, user_agent),
41+
file_blob=upload_key).put()
42+
43+
44+
def fetch_visits(limit):
45+
'get most recent visits'
46+
return Visit.query().order(-Visit.timestamp).fetch(limit)
47+
48+
49+
class UploadHandler(blobstore_handlers.BlobstoreUploadHandler):
50+
'Upload blob (POST) handler'
51+
def post(self):
52+
uploads = self.get_uploads()
53+
blob_id = uploads[0].key() if uploads else None
54+
store_visit(self.request.remote_addr, self.request.user_agent, blob_id)
55+
self.redirect('/', code=307)
56+
57+
58+
class ViewBlobHandler(blobstore_handlers.BlobstoreDownloadHandler):
59+
'view uploaded blob (GET) handler'
60+
def get(self, blob_key):
61+
self.send_blob(blob_key) if blobstore.get(blob_key) else self.error(404)
62+
63+
64+
class MainHandler(BaseHandler):
65+
'main application (GET/POST) handler'
66+
def get(self):
67+
self.render_response('index.html',
68+
upload_url=blobstore.create_upload_url('/upload'))
69+
70+
def post(self):
71+
visits = fetch_visits(10)
72+
self.render_response('index.html', visits=visits)
73+
74+
75+
app = webapp2.WSGIApplication([
76+
('/', MainHandler),
77+
('/upload', UploadHandler),
78+
('/view/([^/]+)?', ViewBlobHandler),
79+
], debug=True)
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
<!doctype html>
2+
<html>
3+
<head>
4+
<title>VisitMe Example</title>
5+
<body>
6+
7+
<h1>VisitMe example</h1>
8+
{% if upload_url %}
9+
10+
<h3>Welcome... upload a file? (optional)</h3>
11+
<form action="{{ upload_url }}" method="POST" enctype="multipart/form-data">
12+
<input type="file" name="file"><p></p>
13+
<input type="submit"> <input type="submit" value="Skip">
14+
</form>
15+
16+
{% else %}
17+
18+
<h3>Last 10 visits</h3>
19+
<ul>
20+
{% for visit in visits %}
21+
<li>{{ visit.timestamp.ctime() }}
22+
<i><code>
23+
{% if visit.file_blob %}
24+
(<a href="/view/{{ visit.file_blob }}" target="_blank">view</a>)
25+
{% else %}
26+
(none)
27+
{% endif %}
28+
</code></i>
29+
from {{ visit.visitor }}
30+
</li>
31+
{% endfor %}
32+
</ul>
33+
34+
{% endif %}
35+
36+
</body>
37+
</html>

0 commit comments

Comments
 (0)