How to get Registry().settings during Pyramid app startup time?

I am used to develop web applications on Django and gunicorn.

In case of Django, any application modules in a Django application can get deployment settings through django.conf.settings. The "settings.py" is written in Python, so that any arbitrary settings and pre-processing can be defined dynamically.

In case of gunicorn, it has three configuration places in order of precedence, and one settings registry class instance combines those.(But usually these settings are used only for gunicorn not application.)

1. Command line parameters.
2. Configuration file. (like Django, written in Python which can have any arbitrary settings dynamically.)
3. Paster application settings.

In case of Pyramid, according to Pyramid documentation, deployment settings may be usually put into pyramid.registry.Registry().settings. But it seems to be accessed only when a pyramid.router.Router() instances exists. That is pyramid.threadlocal.get_current_registry().settings returns None, during the startup process in an application "main.py".

For example, I usually define some business logic in SQLAlchemy model modules, which requires deployment settings as follows.

myapp/models.py

from sqlalchemy import Table, Column, Types
from sqlalchemy.orm import mapper
from pyramid.threadlocal import get_current_registry
from myapp.db import session, metadata

settings = get_current_registry().settings

mytable = Table('mytable', metadata,
    Column('id', Types.INTEGER, primary_key=True,)
    (other columns)...
)

class MyModel(object):
    query = session.query_property()
    external_api_endpoint = settings['external_api_uri']
    timezone = settings['timezone']

    def get_api_result(self):
        (interact with external api ...)

mapper(MyModel, mytable)

But, "settings['external_api_endpoint']" raises a TypeError exception because the "settings" is None.

I thought two solutions.

• Define a callable which accepts "config" argument in "models.py" and "main.py" calls it with a Configurator() instance.

  myapp/models.py

  from sqlalchemy import Table, Column, Types
  from sqlalchemy.orm import mapper
  from myapp.db import session, metadata
  
  _g = globals()
  def initialize(config):
      settings = config.get_settings()
      mytable = Table('mytable', metadata,
          Column('id', Types.INTEGER, rimary_key = True,)
          (other columns ...)
      )
      class MyModel(object):
          query = session.query_property()
          external_api_endpoint = settings['external_api_endpoint']
  
          def get_api_result(self):
              (interact with external api)...
  
      mapper(MyModel, mytable)
      _g['MyModel'] = MyModel
      _g['mytable'] = mytable
  

• Or, put an empty module "app/settings.py", and put setting into it later.

  myapp/__init__.py

  from pyramid.config import Configurator
  from .resources import RootResource
  
  def main(global_config, **settings):
      config = Configurator(
          settings = settings,
          root_factory = RootResource,
      )
      import myapp.settings
      myapp.setting.settings = config.get_settings()
      (other configurations ...)
      return config.make_wsgi_app()
  

Both and other solutions meet the requirements, but I feel troublesome. What I want is the followings.

• development.ini

  defines rough settings because development.ini can have only string type constants.

  [app:myapp]
  use = egg:myapp
  env = dev0
  api_signature = xxxxxx
  

• myapp/settings.py

  defines detail settings based on development.ini, beacause any arbitrary variables(types) can be set.

  import datetime, urllib
  from pytz import timezone
  from pyramid.threadlocal import get_current_registry
  
  pyramid_settings = get_current_registry().settings
  
  if pyramid_settings['env'] == 'production':
      api_endpoint_uri = 'http://api.external.com/?{0}'
      timezone = timezone('US/Eastern')
  elif pyramid_settings['env'] == 'dev0':
      api_endpoint_uri = 'http://sandbox0.external.com/?{0}'
      timezone = timezone('Australia/Sydney')
  elif pyramid_settings['env'] == 'dev1':
      api_endpoint_uri = 'http://sandbox1.external.com/?{0}'
      timezone = timezone('JP/Tokyo')
  api_endpoint_uri = api_endpoint_uri.format(urllib.urlencode({'signature':pyramid_settings['api_signature']}))
  

Then, other modules can get arbitrary deployment settings through "import myapp.settings". Or, if Registry().settings is preferable than "settings.py", **settings kwargs and "settings.py" may be combined and registered into Registry().settings during "main.py" startup process.

Anyway, how to get the settings dictionay during startup time ? Or, Pyramid gently forces us to put every code which requires deployment settings in "views" callables which can get settings dictionary anytime through request.registry.settings ?

EDIT

Thanks, Michael and Chris.

I at last understand why Pyramid uses threadlocal variables(registry and request), in particular registry object for more than one Pyramid applications.

In my opinion, however, deployment settings usually affect business logics that may define application-specific somethings. Those logics are usually put in one or more Python modules that may be other than "app/init.py" or "app/views.py" that can easily get access to Config() or Registry(). Those Python modules are normally "global" at Python process level.

That is, even when more than one Pyramid applications coexist, despite their own threadlocal variables, they have to share those "global" Python modules that may contain applicatin-specific somethings at Pyth开发者_开发技巧on process level.

Of cause, every those modules can have "initialize()" callalbe which is called with a Configurator() by the application "main" callable, or passing Registory() or Request() object through so long series of function calls can meet usual requirements. But, I guess Pyramid beginers (like me) or developers who has "large application or so many settings" may feel troublesome, although that is Pyramid design.

So, I think, Registry().settings should have only real "thread-local" variables, and should not have normal business-logic settings. Responsibility for segregation of multiple application-specific module, classes, callables variables etc. should be taken by developer. As of now, from my viewpoint, I will take Chris's answer. Or in "main" callable, do "execfile('settings.py', settings, settings)" and put it in some "global" space.

Another option, if you enjoy global configuration via Python, create a settings.py file. If it needs values from the ini file, parse the ini file and grab them out (at module scope, so it runs at import time):

from paste.deploy.loadwsgi import appconfig
config = appconfig('config:development.ini', 'myapp', relative_to='.')

if config['env'] == 'production':
    api_endpoint_uri = 'http://api.external.com/?{0}'
    timezone = timezone('US/Eastern')
# .. and so on ...

'config:development.ini' is the name of the ini file (prefixed with 'config:'). 'myapp' is the section name in the config file representing your app (e.g. [app:myapp]). "relative_to" is the directory name in which the config file can be found.

The pattern that I use is to pass the Configurator to modules that need to be initialized. Pyramid doesn't use any global variables because a design goal is to be able to run multiple instances of Pyramid in the same process. The threadlocals are global, but they are local to the current request, so different Pyramid apps can push to them at the same time from different threads.

With this in mind, if you do want a global settings dictionary you'll have to take care of that yourself. You could even push the registry onto the threadlocal manager yourself by calling config.begin().

I think the major thing to take away here is that you shouldn't be calling get_current_registry() at the module level, because at the time of import you aren't really guaranteed that the threadlocals are initialized, however in your init_model() function if you call get_current_registry(), you'd be fine if you previously called config.begin().

Sorry this is a little convoluted, but it's a common question and the best answer is: pass the configurator to your submodules that need it and allow them to add stuff to the registry/settings objects for use later.

Pyramid uses static configration by PasteDeploy, unlike Django. Your [EDIT] part is a nice solution, I think Pyramid community should consider such usage.