0

I am working on some code that was recently upgrade from python 2 to python 3 that is part of google app engine.

Can I ask for some assistance tracing this? I do not know if this is because something was missing during the conversion or due to the fact sync is a seperate microservice from app which appears to have no issue using the datastore

sync.yaml:
  service: sync
  instance_class: F2
  automatic_scaling:
    max_instances: 1
  runtime: python312
  app_engine_apis: true
  entrypoint: gunicorn -b :$PORT sync:app 

  #inbound_services:
  #- warmup

  #libraries:
  #- name: jinja2
  #  version: latest
  #- name: ssl
  #  version: latest

  # taskqueue and cron tasks can access admin urls

  handlers:
  - url: /.*
    script: sync.app
    secure: always
    redirect_http_response_code: 301

  env_variables:
    MEMCACHE_USE_CROSS_COMPATIBLE_PROTOCOL: "True"
    NDB_USE_CROSS_COMPATIBLE_PICKLE_PROTOCOL: "True"
    DEFERRED_USE_CROSS_COMPATIBLE_PICKLE_PROTOCOL: "True"
    CURRENT_VERSION_TIMESTAMP: "1677721600"

sync.py:
collection_dbs, collection_cursor = model.Collection.get_dbs(
  order='name'
)  

collection.py:
from google.cloud import ndb
from google.appengine.api import memcache

from NdbSearchableBase import SearchableModel

from api import fields

import model
import util

import datetime


class Collection(SearchableModel, model.Base):
  slug = ndb.StringProperty(required=True, verbose_name=u'Collection URL')
  name = ndb.StringProperty(required=True, verbose_name=u'Collection Name')
  short_desc = ndb.TextProperty(verbose_name=u'Short Description')`

base.py
@classmethod
def get_dbs(cls, query=None, ancestor=None, order=None, limit=None, cursor=None, **kwargs):
  args = parser.parse({
    'cursor': wf.Str(missing=None),
    'limit': wf.Int(missing=None, validate=validate.Range(min=-1)),
    'order': wf.Str(missing=None),
  })
  return util.get_dbs(
    query or cls.query(ancestor=ancestor),
    limit=limit or args['limit'],
    cursor=cursor or args['cursor'],
    order=order or args['order'],
    **kwargs
  )

util.py
def get_dbs(
        query, order=None, limit=None, cursor=None, prev_cursor=False,
        keys_only=None, **filters
):
  model_class = ndb.Model._kind_map[query.kind]
  query_prev = query
  if order:
    for o in order.split(','):
      if o.startswith('-'):
        query = query.order(-model_class._properties[o[1:]])
        if prev_cursor:
          query_prev = query_prev.order(model_class._properties[o[1:]])
      else:
        query = query.order(model_class._properties[o])
        if prev_cursor:
          query_prev = query_prev.order(-model_class._properties[o])

  for prop, value in filters.items():
    if value is None:
      continue
    for val in value if isinstance(value, list) else [value]:
      query = query.filter(model_class._properties[prop] == val)
      if prev_cursor:
        query_prev = query_prev.filter(model_class._properties[prop] == val)

  limit = limit or config.DEFAULT_DB_LIMIT
  if limit == -1:
    return list(query.fetch(keys_only=keys_only)), {'next': None, 'prev': None}

  cursor = Cursor.from_websafe_string(cursor) if cursor else None
  model_dbs, next_cursor, more = query.fetch_page(
    limit, start_cursor=cursor, keys_only=keys_only,
  )
  next_cursor = next_cursor.to_websafe_string() if more else None
  if not prev_cursor:
    return list(model_dbs), {'next': next_cursor, 'prev': None}
  model_dbs_prev, prev_cursor, prev_more = query_prev.fetch_page(
    limit, start_cursor=cursor.reversed() if cursor else None, keys_only=True
  )
  prev_cursor = prev_cursor.reversed().to_websafe_string() \
    if prev_cursor and cursor else None
  return list(model_dbs), {'next': next_cursor, 'prev': prev_cursor}

2025-05-09 16:39:19 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/tasklets.py", line 319, in _advance_tasklet 2025-05-09 16:39:19 sync[20250425t195045] yielded = self.generator.throw(type(error), error, traceback) 2025-05-09 16:39:19 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:19 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/_datastore_query.py", line 116, in fetch 2025-05-09 16:39:19 sync[20250425t195045] while (yield results.has_next_async()): 2025-05-09 16:39:19 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:19 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/tasklets.py", line 319, in _advance_tasklet 2025-05-09 16:39:19 sync[20250425t195045] yielded = self.generator.throw(type(error), error, traceback) 2025-05-09 16:39:19 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:19 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/_datastore_query.py", line 343, in has_next_async 2025-05-09 16:39:19 sync[20250425t195045] yield self._next_batch() # First time 2025-05-09 16:39:19 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:19 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/tasklets.py", line 319, in _advance_tasklet 2025-05-09 16:39:19 sync[20250425t195045] yielded = self.generator.throw(type(error), error, traceback) 2025-05-09 16:39:19 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:19 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/_datastore_query.py", line 373, in _next_batch 2025-05-09 16:39:19 sync[20250425t195045] response = yield _datastore_run_query(query) 2025-05-09 16:39:19 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:19 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/tasklets.py", line 319, in _advance_tasklet 2025-05-09 16:39:19 sync[20250425t195045] yielded = self.generator.throw(type(error), error, traceback) 2025-05-09 16:39:19 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:19 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/_datastore_query.py", line 1030, in _datastore_run_query 2025-05-09 16:39:19 sync[20250425t195045] response = yield _datastore_api.make_call( 2025-05-09 16:39:19 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:19 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/tasklets.py", line 323, in _advance_tasklet 2025-05-09 16:39:19 sync[20250425t195045] yielded = self.generator.send(send_value) 2025-05-09 16:39:19 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:19 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/_retry.py", line 112, in retry_wrapper 2025-05-09 16:39:19 sync[20250425t195045] raise core_exceptions.RetryError( 2025-05-09 16:39:20 sync[20250425t195045] google.api_core.exceptions.RetryError: Maximum number of 3 retries exceeded while calling <function make_call..rpc_call at 0x3ebfd2fc2700>, last exception: 503 Getting metadata from plugin failed with error: '_AppEnginePoolManager' object has no attribute 'connection_from_host' 2025-05-09 16:39:22 sync[20250425t195045] [2025-05-09 16:39:22,717] ERROR in app: Exception on /keepalive/ [GET] 2025-05-09 16:39:22 sync[20250425t195045] Traceback (most recent call last): File "/layers/google.python.pip/pip/lib/python3.12/site-packages/flask/app.py", line 2529, in wsgi_app response = self.full_dispatch_request() 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/flask/app.py", line 1825, in full_dispatch_request 2025-05-09 16:39:22 sync[20250425t195045] rv = self.handle_user_exception(e) 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/flask/app.py", line 1823, in full_dispatch_request 2025-05-09 16:39:22 sync[20250425t195045] rv = self.dispatch_request() 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/flask/app.py", line 1799, in dispatch_request 2025-05-09 16:39:22 sync[20250425t195045] return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/workspace/sync.py", line 224, in keepalive 2025-05-09 16:39:22 sync[20250425t195045] collection_dbs, collection_cursor = model.Collection.get_dbs( 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/workspace/model/base.py", line 33, in get_dbs 2025-05-09 16:39:22 sync[20250425t195045] return util.get_dbs( 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/workspace/util.py", line 196, in get_dbs 2025-05-09 16:39:22 sync[20250425t195045] model_dbs, next_cursor, more = query.fetch_page( 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/query.py", line 1201, in wrapper 2025-05-09 16:39:22 sync[20250425t195045] return wrapped(self, *dummy_args, _options=query_options) 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/utils.py", line 118, in wrapper 2025-05-09 16:39:22 sync[20250425t195045] return wrapped(*args, **new_kwargs) 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/utils.py", line 150, in positional_wrapper 2025-05-09 16:39:22 sync[20250425t195045] return wrapped(*args, **kwds) 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/query.py", line 2267, in fetch_page 2025-05-09 16:39:22 sync[20250425t195045] return self.fetch_page_async(None, _options=kwargs["_options"]).result() 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/tasklets.py", line 210, in result 2025-05-09 16:39:22 sync[20250425t195045] self.check_success() 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/tasklets.py", line 157, in check_success 2025-05-09 16:39:22 sync[20250425t195045] raise self._exception 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/tasklets.py", line 319, in _advance_tasklet 2025-05-09 16:39:22 sync[20250425t195045] yielded = self.generator.throw(type(error), error, traceback) 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/query.py", line 2310, in fetch_page_async 2025-05-09 16:39:22 sync[20250425t195045] while (yield iterator.has_next_async()): 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/tasklets.py", line 319, in _advance_tasklet 2025-05-09 16:39:22 sync[20250425t195045] yielded = self.generator.throw(type(error), error, traceback) 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/_datastore_query.py", line 343, in has_next_async 2025-05-09 16:39:22 sync[20250425t195045] yield self._next_batch() # First time 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/tasklets.py", line 319, in _advance_tasklet 2025-05-09 16:39:22 sync[20250425t195045] yielded = self.generator.throw(type(error), error, traceback) 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/_datastore_query.py", line 373, in _next_batch 2025-05-09 16:39:22 sync[20250425t195045] response = yield _datastore_run_query(query) 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/tasklets.py", line 319, in _advance_tasklet 2025-05-09 16:39:22 sync[20250425t195045] yielded = self.generator.throw(type(error), error, traceback) 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/_datastore_query.py", line 1030, in _datastore_run_query 2025-05-09 16:39:22 sync[20250425t195045] response = yield _datastore_api.make_call( 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/tasklets.py", line 323, in _advance_tasklet 2025-05-09 16:39:22 sync[20250425t195045] yielded = self.generator.send(send_value) 2025-05-09 16:39:22 sync[20250425t195045] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-05-09 16:39:22 sync[20250425t195045] File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/cloud/ndb/_retry.py", line 112, in retry_wrapper 2025-05-09 16:39:22 sync[20250425t195045] raise core_exceptions.RetryError( 2025-05-09 16:39:23 sync[20250425t195045] google.api_core.exceptions.RetryError: Maximum number of 3 retries exceeded while calling <function make_call..rpc_call at 0x3ebff8fff1a0>, last exception: 503 Getting metadata from plugin failed with error: '_AppEnginePoolManager' object has no attribute 'connection_from_host'

Improve this question
6
  • What exactly are you trying to do (what is your code supposed to be doing)? Where in your code are you having errors (try to narrow it down maybe by adding print statements so you know where your code stops running) Commented May 12 at 0:44
  • Thank you for the reply. I don't understand the lower layers of ndb. The error bubbles up through this call. These call support enumeration of two google data store entities. cursor = Cursor.from_websafe_string(cursor) if cursor else None line: 196 model_dbs, next_cursor, more = query.fetch_page( limit, start_cursor=cursor, keys_only=keys_only, ) Commented May 12 at 1:39
  • Are you saying - you want to run a paginated query (using cursors) and that's where you're getting an error? Commented May 12 at 13:31
  • Yes, that is my understanding of the execution path. I don't understand the exception that is being produced. This was working with python 2. Is there documentation available to help me understand? Commented May 12 at 14:01
  • See my response (it has sample code) to one of your other questions Commented May 14 at 4:46

1 Answer 1

Reset to default
0

In your sync.py try wrapping all ndb operations inside ndb.Client().context() like:

from google.cloud import ndb
client = ndb.Client()
with client.context():
    collection_dbs, collection_cursor = model.Collection.get_dbs( order='name' )

You can refer to this documentation on Python 3 version of ndb client library.

Also, make sure that your service account has the proper permissions to query Datastore.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Did this work? Any suggestions or confirmation to my solution would be appreciated.
Thanks, I'll try tommorrow. This format helps me digest information. Rereading I can see some of my assertions were misguided. I am suspecting this line below doesn't work in python3. I'm hoping there is a dropin replacement. I do not think it was being reached from the other microservice cursor = Cursor.from_websafe_string(cursor) if cursor else None model_dbs, next_cursor, more = query.fetch_page( limit, start_cursor=cursor, keys_only=keys_only, )
the thread already has context the simple call to get_dbs yields "last exception: 503 Getting metadata from plugin failed with error: '_AppEnginePoolManager' object has no attribute 'connection_from_host'"
"context activation" per googleapis.dev/python/python-ndb/latest

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.