Merge remote-tracking branch 'upstream/master' into issue_198

CHANGES

@@ -40,6 +40,7 @@ Release date to be announced, codename to be selected
 - Added `teardown_request` decorator, for functions that should run at the end
   of a request regardless of whether an exception occurred.
 - Implemented :func:`flask.has_request_context`
+- Added :func:`safe_join`
 
 Version 0.6.1
 -------------

docs/api.rst

@@ -244,6 +244,8 @@ Useful Functions and Classes
 
 .. autofunction:: send_from_directory
 
+.. autofunction:: safe_join
+
 .. autofunction:: escape
 
 .. autoclass:: Markup

docs/deploying/others.rst

@@ -72,7 +72,7 @@ problematic values in the WSGI environment usually are `REMOTE_ADDR` and
 but you might want to write your own WSGI middleware for specific setups.
 
 The most common setup invokes the host being set from `X-Forwarded-Host`
-and the remote address from `X-Forward-For`::
+and the remote address from `X-Forwarded-For`::
 
     from werkzeug.contrib.fixers import ProxyFix
     app.wsgi_app = ProxyFix(app.wsgi_app)

docs/extensiondev.rst

@@ -137,7 +137,6 @@ Now this is where your extension code goes.  But how exactly should such
 an extension look like?  What are the best practices?  Continue reading
 for some insight.
 
-
 Initializing Extensions
 -----------------------
 
@@ -165,8 +164,8 @@ classes:
     a remote application that uses OAuth.
 
 What to use depends on what you have in mind.  For the SQLite 3 extension
-we will need to use the class based approach because we have to use a
+we will use the class based approach because it will provide users with a
-controller object that can be used to connect to the database.
+manager object that handles opening and closing database connections.
 
 The Extension Code
 ------------------
@@ -175,87 +174,124 @@ Here's the contents of the `flaskext/sqlite3.py` for copy/paste::
 
     from __future__ import absolute_import
     import sqlite3
-    from flask import g
+
+    from flask import _request_ctx_stack
 
     class SQLite3(object):
-    
+
         def __init__(self, app):
             self.app = app
             self.app.config.setdefault('SQLITE3_DATABASE', ':memory:')
-
-            self.app.before_request(self.before_request)
             self.app.after_request(self.after_request)
+            self.app.before_request(self.before_request)
 
         def connect(self):
             return sqlite3.connect(self.app.config['SQLITE3_DATABASE'])
 
         def before_request(self):
-            g.sqlite3_db = self.connect()
+            ctx = _request_ctx_stack.top
+            ctx.sqlite3_db = self.connect()
 
         def after_request(self, response):
-            g.sqlite3_db.close()
+            ctx = _request_ctx_stack.top
+            ctx.sqlite3_db.close()
             return response
 
-So here's what the lines of code do:
+        def get_db(self):
+            ctx = _request_ctx_stack.top
+            if ctx is not None:
+                return ctx.sqlite3_db
 
-1.  the ``__future__`` import is necessary to activate absolute imports.
+So here's what these lines of code do:
-    This is needed because otherwise we could not call our module
-    `sqlite3.py` and import the top-level `sqlite3` module which actually
-    implements the connection to SQLite.
-2.  We create a class for our extension that sets a default configuration
-    for the SQLite 3 database if it's not there (:meth:`dict.setdefault`)
-    and connects two functions as before and after request handlers.
-3.  Then it implements a `connect` function that returns a new database
-    connection and the two handlers.
 
-So why did we decide on a class based approach here?  Because using that
+1.  The ``__future__`` import is necessary to activate absolute imports.
+    Otherwise we could not call our module `sqlite3.py` and import the
+    top-level `sqlite3` module which actually implements the connection to
+    SQLite.
+2.  We create a class for our extension that requires a supplied `app` object,
+    sets a configuration for the database if it's not there
+    (:meth:`dict.setdefault`), and attaches `before_request` and
+    `after_request` handlers.
+3.  Next, we define a `connect` function that opens a database connection.
+4.  Then we set up the request handlers we bound to the app above.  Note here
+    that we're attaching our database connection to the top request context via
+    `_request_ctx_stack.top`. Extensions should use the top context and not the
+    `g` object to store things like database connections.
+5.  Finally, we add a `get_db` function that simplifies access to the context's
+    database.
+
+So why did we decide on a class based approach here?  Because using our
 extension looks something like this::
 
-    from flask import Flask, g
+    from flask import Flask
     from flaskext.sqlite3 import SQLite3
 
     app = Flask(__name__)
     app.config.from_pyfile('the-config.cfg')
-    db = SQLite(app)
+    manager = SQLite3(app)
+    db = manager.get_db()
 
-Either way you can use the database from the views like this::
+You can then use the database from views like this::
 
     @app.route('/')
     def show_all():
-        cur = g.sqlite3_db.cursor()
+        cur = db.cursor()
         cur.execute(...)
 
-But how would you open a database connection from outside a view function?
+Opening a database connection from outside a view function is simple.
-This is where the `db` object now comes into play:
 
 >>> from yourapplication import db
->>> con = db.connect()
+>>> cur = db.cursor()
->>> cur = con.cursor()
+>>> cur.execute(...)
 
-If you don't need that, you can go with initialization functions.
+Adding an `init_app` Function
+-----------------------------
 
-Initialization Functions
+In practice, you'll almost always want to permit users to initialize your
-------------------------
+extension and provide an app object after the fact. This can help avoid
+circular import problems when a user is breaking their app into multiple files.
+Our extension could add an `init_app` function as follows::
 
-Here's what the module would look like with initialization functions::
+    class SQLite3(object):
 
-    from __future__ import absolute_import
+        def __init__(self, app=None):
-    import sqlite3
+            if app is not None:
-    from flask import g
+                self.app = app
+                self.init_app(self.app)
+            else:
+                self.app = None
 
-    def init_sqlite3(app):
+        def init_app(self, app):
-        app = app
+            self.app = app
-        app.config.setdefault('SQLITE3_DATABASE', ':memory:')
+            self.app.config.setdefault('SQLITE3_DATABASE', ':memory:')
+            self.app.after_request(self.after_request)
+            self.app.before_request(self.before_request)
 
-        @app.before_request
+        def connect(self):
-        def before_request():
+            return sqlite3.connect(app.config['SQLITE3_DATABASE'])
-            g.sqlite3_db = sqlite3.connect(self.app.config['SQLITE3_DATABASE'])
 
-        @app.after_request
+        def before_request(self):
-        def after_request(response):
+            ctx = _request_ctx_stack.top
-            g.sqlite3_db.close()
+            ctx.sqlite3_db = self.connect()
+
+        def after_request(self, response):
+            ctx = _request_ctx_stack.top
+            ctx.sqlite3_db.close()
             return response
 
+        def get_db(self):
+            ctx = _request_ctx_stack.top
+            if ctx is not None:
+                return ctx.sqlite3_db
+
+The user could then initialize the extension in one file::
+
+    manager = SQLite3()
+
+and bind their app to the extension in another file::
+
+    manager.init_app(app)
+
 Learn from Others
 -----------------
 
@@ -276,7 +312,6 @@ designing the API.
 The best Flask extensions are extensions that share common idioms for the
 API.  And this can only work if collaboration happens early.
 
-
 Approved Extensions
 -------------------
 

docs/patterns/appdispatch.rst

@@ -0,0 +1,154 @@
+.. _app-dispatch:
+
+Application Dispatching
+=======================
+
+Application dispatching is the process of combining multiple Flask
+applications on the WSGI level.  You can not only combine Flask
+applications into something larger but any WSGI application.  This would
+even allow you to run a Django and a Flask application in the same
+interpreter side by side if you want.  The usefulness of this depends on
+how the applications work internally.
+
+The fundamental difference from the :ref:`module approach
+<larger-applications>` is that in this case you are running the same or
+different Flask applications that are entirely isolated from each other.
+They run different configurations and are dispatched on the WSGI level.
+
+Combining Applications
+----------------------
+
+If you have entirely separated applications and you want them to work next
+to each other in the same Python interpreter process you can take
+advantage of the :class:`werkzeug.wsgi.DispatcherMiddleware`.  The idea
+here is that each Flask application is a valid WSGI application and they
+are combined by the dispatcher middleware into a larger one that
+dispatched based on prefix.
+
+For example you could have your main application run on `/` and your
+backend interface on `/admin`::
+
+    from werkzeug.wsgi import DispatcherMiddleware
+    from frontend_app import application as frontend
+    from backend_app import application as backend
+
+    application = DispatcherMiddleware(frontend, {
+        '/backend':     backend
+    })
+
+
+Dispatch by Subdomain
+---------------------
+
+Sometimes you might want to use multiple instances of the same application
+with different configurations.  Assuming the application is created inside
+a function and you can call that function to instanciate it, that is
+really easy to implement.  In order to develop your application to support
+creating new instances in functions have a look at the
+:ref:`app-factories` pattern.
+
+A very common example would be creating applications per subdomain.  For
+instance you configure your webserver to dispatch all requests for all
+subdomains to your application and you then use the subdomain information
+to create user-specific instances.  Once you have your server set up to
+listen on all subdomains you can use a very simple WSGI application to do
+the dynamic application creation.
+
+The perfect level for abstraction in that regard is the WSGI layer.  You
+write your own WSGI application that looks at the request that comes and
+and delegates it to your Flask application.  If that application does not
+exist yet, it is dynamically created and remembered::
+
+    from threading import Lock
+
+    class SubdomainDispatcher(object):
+
+        def __init__(self, domain, create_app):
+            self.domain = domain
+            self.create_app = create_app
+            self.lock = Lock()
+            self.instances = {}
+
+        def get_application(self, host):
+            host = host.split(':')[0]
+            assert host.endswith(self.domain), 'Configuration error'
+            subdomain = host[:-len(self.domain)].rstrip('.')
+            with self.lock:
+                app = self.instances.get(subdomain)
+                if app is None:
+                    app = self.create_app(subdomain)
+                    self.instances[subdomain] = app
+                return app
+
+        def __call__(self, environ, start_response):
+            app = self.get_application(environ['HTTP_HOST'])
+            return app(environ, start_response)
+
+
+This dispatcher can then be used like this::
+
+    from myapplication import create_app, get_user_for_subdomain
+    from werkzeug.exceptions import NotFound
+
+    def make_app(subdomain):
+        user = get_user_for_subdomain(subdomain)
+        if user is None:
+            # if there is no user for that subdomain we still have
+            # to return a WSGI application that handles that request.
+            # We can then just return the NotFound() exception as
+            # application which will render a default 404 page.
+            # You might also redirect the user to the main page then
+            return NotFound()
+
+        # otherwise create the application for the specific user
+        return create_app(user)
+
+    application = SubdomainDispatcher('example.com', make_app)
+
+
+Dispatch by Path
+----------------
+
+Dispatching by a path on the URL is very similar.  Instead of looking at
+the `Host` header to figure out the subdomain one simply looks at the
+request path up to the first slash::
+
+    from threading import Lock
+    from werkzeug.wsgi import pop_path_info, peek_path_info
+
+    class PathDispatcher(object):
+
+        def __init__(self, default_app, create_app):
+            self.default_app = default_app
+            self.create_app = create_app
+            self.lock = Lock()
+            self.instances = {}
+
+        def get_application(self, prefix):
+            with self.lock:
+                app = self.instances.get(prefix)
+                if app is None:
+                    app = self.create_app(prefix)
+                    if app is not None:
+                        self.instances[prefix] = app
+                return app
+
+        def __call__(self, environ, start_response):
+            app = self.get_application(peek_path_info(environ))
+            if app is not None:
+                pop_path_info(environ)
+            else:
+                app = self.default_app
+            return app(environ, start_response)
+
+The big difference between this and the subdomain one is that this one
+falls back to another application if the creator function returns `None`::
+
+    from myapplication import create_app, default_app, get_user_for_prefix
+
+    def make_app(prefix):
+        user = get_user_for_prefix(prefix)
+        if user is not None:
+            return create_app(user)
+
+    application = PathDispatcher('example.com', default_app, make_app)

docs/patterns/index.rst

@@ -18,6 +18,7 @@ Snippet Archives <http://flask.pocoo.org/snippets/>`_.
 
    packages
    appfactories
+   appdispatch
    distribute
    fabric
    sqlite3

docs/tutorial/dbcon.rst

@@ -24,7 +24,7 @@ db connection in the interactive debugger::
         return response
 
 If you want to guarantee that the connection is always closed in debug mode, you
-can close it in a function decorated with :meth:`~flask.Flask.teardown_request`:
+can close it in a function decorated with :meth:`~flask.Flask.teardown_request`::
 
     @app.before_request
     def before_request():

docs/tutorial/setup.rst

@@ -70,7 +70,14 @@ server if we want to run that file as a standalone application::
         app.run()
 
 With that out of the way you should be able to start up the application
-without problems.  When you head over to the server you will get an 404
+without problems.  Do this with the following command::
+
+   python flaskr.py
+
+You will see a message telling you that server has started along with
+the address at which you can access it.
+
+When you head over to the server in your browser you will get an 404
 page not found error because we don't have any views yet.  But we will
 focus on that a little later.  First we should get the database working.
 

flask/__init__.py

@@ -19,7 +19,7 @@ from .app import Flask, Request, Response
 from .config import Config
 from .helpers import url_for, jsonify, json_available, flash, \
     send_file, send_from_directory, get_flashed_messages, \
-    get_template_attribute, make_response
+    get_template_attribute, make_response, safe_join
 from .globals import current_app, g, request, session, _request_ctx_stack
 from .ctx import has_request_context
 from .module import Module

flask/app.py

@@ -713,12 +713,17 @@ class Flask(_PackageBoundObject):
         return f
 
     def after_request(self, f):
-        """Register a function to be run after each request."""
+        """Register a function to be run after each request. Your function
+        must take one parameter, a :attr:`response_class` object and return
+        a new response object or the same (see :meth:`process_response`).
+        """
         self.after_request_funcs.setdefault(None, []).append(f)
         return f
 
     def teardown_request(self, f):
-        """Register a function to be run at the end of each request, regardless of whether there was an exception or not."""
+        """Register a function to be run at the end of each request,
+        regardless of whether there was an exception or not.
+        """
         self.teardown_request_funcs.setdefault(None, []).append(f)
         return f
 

flask/helpers.py

@@ -166,7 +166,8 @@ def url_for(endpoint, **values):
     ==================== ======================= =============================
 
     Variable arguments that are unknown to the target endpoint are appended
-    to the generated URL as query arguments.
+    to the generated URL as query arguments.  If the value of a query argument
+    is `None`, the whole pair is skipped.
 
     For more information, head over to the :ref:`Quickstart <url-building>`.
 
@@ -321,7 +322,7 @@ def send_file(filename_or_fp, mimetype=None, as_attachment=False,
         if not attachment_filename and not mimetype \
            and isinstance(filename, basestring):
             warn(DeprecationWarning('The filename support for file objects '
-                'passed to send_file is not deprecated.  Pass an '
+                'passed to send_file is now deprecated.  Pass an '
                 'attach_filename if you want mimetypes to be guessed.'),
                 stacklevel=2)
         if add_etags:
@@ -388,6 +389,31 @@ def send_file(filename_or_fp, mimetype=None, as_attachment=False,
     return rv
 
 
+def safe_join(directory, filename):
+    """Safely join `directory` and `filename`.
+
+    Example usage::
+
+        @app.route('/wiki/<path:filename>')
+        def wiki_page(filename):
+            filename = safe_join(app.config['WIKI_FOLDER'], filename)
+            with open(filename, 'rb') as fd:
+                content = fd.read() # Read and process the file content...
+
+    :param directory: the base directory.
+    :param filename: the untrusted filename relative to that directory.
+    :raises: :class:`~werkzeug.exceptions.NotFound` if the retsulting path
+             would fall out of `directory`.
+    """
+    filename = posixpath.normpath(filename)
+    for sep in _os_alt_seps:
+        if sep in filename:
+            raise NotFound()
+    if os.path.isabs(filename) or filename.startswith('../'):
+        raise NotFound()
+    return os.path.join(directory, filename)
+
+
 def send_from_directory(directory, filename, **options):
     """Send a file from a given directory with :func:`send_file`.  This
     is a secure way to quickly expose static files from an upload folder
@@ -415,13 +441,7 @@ def send_from_directory(directory, filename, **options):
     :param options: optional keyword arguments that are directly
                     forwarded to :func:`send_file`.
     """
-    filename = posixpath.normpath(filename)
+    filename = safe_join(directory, filename)
-    for sep in _os_alt_seps:
-        if sep in filename:
-            raise NotFound()
-    if os.path.isabs(filename) or filename.startswith('../'):
-        raise NotFound()
-    filename = os.path.join(directory, filename)
     if not os.path.isfile(filename):
         raise NotFound()
     return send_file(filename, conditional=True, **options)

tests/flask_tests.py

@@ -47,6 +47,10 @@ SECRET_KEY = 'devkey'
 def catch_warnings():
     """Catch warnings in a with block in a list"""
     import warnings
+
+    # make sure deprecation warnings are active in tests
+    warnings.simplefilter('default', category=DeprecationWarning)
+
     filters = warnings.filters
     warnings.filters = filters[:]
     old_showwarning = warnings.showwarning
@@ -638,6 +642,7 @@ class BasicFunctionalityTestCase(unittest.TestCase):
         app.config.update(
             SERVER_NAME='localhost.localdomain:5000'
         )
+
         @app.route('/')
         def index():
             return None