maya-chen/flask
1
0
Fork 0
forked from orbit-oss/flask
Code Pull requests Activity

Merge branch '2.2.x'

Browse source
This commit is contained in:
David Lord 2023-02-10 15:06:38 -08:00
commit a18ae3d752
No known key found for this signature in database
GPG key ID: 7A1C87E3F5BC42A8
15 changed files with 764 additions and 105 deletions
Download patch file Download diff file Expand all files Collapse all files

1
docs/index.rst
View file

@ -48,6 +48,7 @@ community-maintained extensions to add even more functionality.
config
signals
views
lifecycle
appcontext
reqcontext
blueprints

168
docs/lifecycle.rst Normal file
View file

@ -0,0 +1,168 @@
Application Structure and Lifecycle
===================================
Flask makes it pretty easy to write a web application. But there are quite a few
different parts to an application and to each request it handles. Knowing what happens
during application setup, serving, and handling requests will help you know what's
possible in Flask and how to structure your application.
Application Setup
-----------------
The first step in creating a Flask application is creating the application object. Each
Flask application is an instance of the :class:`.Flask` class, which collects all
configuration, extensions, and views.
.. code-block:: python
from flask import Flask
app = Flask(__name__)
app.config.from_mapping(
SECRET_KEY="dev",
)
app.config.from_prefixed_env()
@app.route("/")
def index():
return "Hello, World!"
This is known as the "application setup phase", it's the code you write that's outside
any view functions or other handlers. It can be split up between different modules and
sub-packages, but all code that you want to be part of your application must be imported
in order for it to be registered.
All application setup must be completed before you start serving your application and
handling requests. This is because WSGI servers divide work between multiple workers, or
can be distributed across multiple machines. If the configuration changed in one worker,
there's no way for Flask to ensure consistency between other workers.
Flask tries to help developers catch some of these setup ordering issues by showing an
error if setup-related methods are called after requests are handled. In that case
you'll see this error:
The setup method 'route' can no longer be called on the application. It has already
handled its first request, any changes will not be applied consistently.
Make sure all imports, decorators, functions, etc. needed to set up the application
are done before running it.
However, it is not possible for Flask to detect all cases of out-of-order setup. In
general, don't do anything to modify the ``Flask`` app object and ``Blueprint`` objects
from within view functions that run during requests. This includes:
- Adding routes, view functions, and other request handlers with ``@app.route``,
``@app.errorhandler``, ``@app.before_request``, etc.
- Registering blueprints.
- Loading configuration with ``app.config``.
- Setting up the Jinja template environment with ``app.jinja_env``.
- Setting a session interface, instead of the default itsdangerous cookie.
- Setting a JSON provider with ``app.json``, instead of the default provider.
- Creating and initializing Flask extensions.
Serving the Application
-----------------------
Flask is a WSGI application framework. The other half of WSGI is the WSGI server. During
development, Flask, through Werkzeug, provides a development WSGI server with the
``flask run`` CLI command. When you are done with development, use a production server
to serve your application, see :doc:`deploying/index`.
Regardless of what server you're using, it will follow the :pep:`3333` WSGI spec. The
WSGI server will be told how to access your Flask application object, which is the WSGI
application. Then it will start listening for HTTP requests, translate the request data
into a WSGI environ, and call the WSGI application with that data. The WSGI application
will return data that is translated into an HTTP response.
#. Browser or other client makes HTTP request.
#. WSGI server receives request.
#. WSGI server converts HTTP data to WSGI ``environ`` dict.
#. WSGI server calls WSGI application with the ``environ``.
#. Flask, the WSGI application, does all its internal processing to route the request
to a view function, handle errors, etc.
#. Flask translates View function return into WSGI response data, passes it to WSGI
server.
#. WSGI server creates and send an HTTP response.
#. Client receives the HTTP response.
Middleware
~~~~~~~~~~
The WSGI application above is a callable that behaves in a certain way. Middleware
is a WSGI application that wraps another WSGI application. It's a similar concept to
Python decorators. The outermost middleware will be called by the server. It can modify
the data passed to it, then call the WSGI application (or further middleware) that it
wraps, and so on. And it can take the return value of that call and modify it further.
From the WSGI server's perspective, there is one WSGI application, the one it calls
directly. Typically, Flask is the "real" application at the end of the chain of
middleware. But even Flask can call further WSGI applications, although that's an
advanced, uncommon use case.
A common middleware you'll see used with Flask is Werkzeug's
:class:`~werkzeug.middleware.proxy_fix.ProxyFix`, which modifies the request to look
like it came directly from a client even if it passed through HTTP proxies on the way.
There are other middleware that can handle serving static files, authentication, etc.
How a Request is Handled
------------------------
For us, the interesting part of the steps above is when Flask gets called by the WSGI
server (or middleware). At that point, it will do quite a lot to handle the request and
generate the response. At the most basic, it will match the URL to a view function, call
the view function, and pass the return value back to the server. But there are many more
parts that you can use to customize its behavior.
#. WSGI server calls the Flask object, which calls :meth:`.Flask.wsgi_app`.
#. A :class:`.RequestContext` object is created. This converts the WSGI ``environ``
dict into a :class:`.Request` object. It also creates an :class:`AppContext` object.
#. The :doc:`app context <appcontext>` is pushed, which makes :data:`.current_app` and
:data:`.g` available.
#. The :data:`.appcontext_pushed` signal is sent.
#. The :doc:`request context <reqcontext>` is pushed, which makes :attr:`.request` and
:class:`.session` available.
#. The session is opened, loading any existing session data using the app's
:attr:`~.Flask.session_interface`, an instance of :class:`.SessionInterface`.
#. The URL is matched against the URL rules registered with the :meth:`~.Flask.route`
decorator during application setup. If there is no match, the error - usually a 404,
405, or redirect - is stored to be handled later.
#. The :data:`.request_started` signal is sent.
#. Any :meth:`~.Flask.url_value_preprocessor` decorated functions are called.
#. Any :meth:`~.Flask.before_request` decorated functions are called. If any of
these function returns a value it is treated as the response immediately.
#. If the URL didn't match a route a few steps ago, that error is raised now.
#. The :meth:`~.Flask.route` decorated view function associated with the matched URL
is called and returns a value to be used as the response.
#. If any step so far raised an exception, and there is an :meth:`~.Flask.errorhandler`
decorated function that matches the exception class or HTTP error code, it is
called to handle the error and return a response.
#. Whatever returned a response value - a before request function, the view, or an
error handler, that value is converted to a :class:`.Response` object.
#. Any :func:`~.after_this_request` decorated functions are called, then cleared.
#. Any :meth:`~.Flask.after_request` decorated functions are called, which can modify
the response object.
#. The session is saved, persisting any modified session data using the app's
:attr:`~.Flask.session_interface`.
#. The :data:`.request_finished` signal is sent.
#. If any step so far raised an exception, and it was not handled by an error handler
function, it is handled now. HTTP exceptions are treated as responses with their
corresponding status code, other exceptions are converted to a generic 500 response.
The :data:`.got_request_exception` signal is sent.
#. The response object's status, headers, and body are returned to the WSGI server.
#. Any :meth:`~.Flask.teardown_request` decorated functions are called.
#. The :data:`.request_tearing_down` signal is sent.
#. The request context is popped, :attr:`.request` and :class:`.session` are no longer
available.
#. Any :meth:`~.Flask.teardown_appcontext` decorated functions are called.
#. The :data:`.appcontext_tearing_down` signal is sent.
#. The app context is popped, :data:`.current_app` and :data:`.g` are no longer
available.
#. The :data:`.appcontext_popped` signal is sent.
There are even more decorators and customization points than this, but that aren't part
of every request lifecycle. They're more specific to certain things you might use during
a request, such as templates, building URLs, or handling JSON data. See the rest of this
documentation, as well as the :doc:`api` to explore further.

277
docs/patterns/celery.rst
View file

@ -1,105 +1,242 @@
Celery Background Tasks
=======================
Background Tasks with Celery
============================
If your application has a long running task, such as processing some uploaded
data or sending email, you don't want to wait for it to finish during a
request. Instead, use a task queue to send the necessary data to another
process that will run the task in the background while the request returns
immediately.
If your application has a long running task, such as processing some uploaded data or
sending email, you don't want to wait for it to finish during a request. Instead, use a
task queue to send the necessary data to another process that will run the task in the
background while the request returns immediately.
`Celery`_ is a powerful task queue that can be used for simple background tasks as well
as complex multi-stage programs and schedules. This guide will show you how to configure
Celery using Flask. Read Celery's `First Steps with Celery`_ guide to learn how to use
Celery itself.
.. _Celery: https://celery.readthedocs.io
.. _First Steps with Celery: https://celery.readthedocs.io/en/latest/getting-started/first-steps-with-celery.html
The Flask repository contains `an example <https://github.com/pallets/flask/tree/main/examples/celery>`_
based on the information on this page, which also shows how to use JavaScript to submit
tasks and poll for progress and results.
Celery is a powerful task queue that can be used for simple background tasks
as well as complex multi-stage programs and schedules. This guide will show you
how to configure Celery using Flask, but assumes you've already read the
`First Steps with Celery <https://celery.readthedocs.io/en/latest/getting-started/first-steps-with-celery.html>`_
guide in the Celery documentation.
Install
-------
Celery is a separate Python package. Install it from PyPI using pip::
Install Celery from PyPI, for example using pip:
.. code-block:: text
$ pip install celery
Configure
---------
The first thing you need is a Celery instance, this is called the celery
application. It serves the same purpose as the :class:`~flask.Flask`
object in Flask, just for Celery. Since this instance is used as the
entry-point for everything you want to do in Celery, like creating tasks
and managing workers, it must be possible for other modules to import it.
Integrate Celery with Flask
---------------------------
For instance you can place this in a ``tasks`` module. While you can use
Celery without any reconfiguration with Flask, it becomes a bit nicer by
subclassing tasks and adding support for Flask's application contexts and
hooking it up with the Flask configuration.
You can use Celery without any integration with Flask, but it's convenient to configure
it through Flask's config, and to let tasks access the Flask application.
This is all that is necessary to integrate Celery with Flask:
Celery uses similar ideas to Flask, with a ``Celery`` app object that has configuration
and registers tasks. While creating a Flask app, use the following code to create and
configure a Celery app as well.
.. code-block:: python
from celery import Celery
from celery import Celery, Task
def make_celery(app):
celery = Celery(app.import_name)
celery.conf.update(app.config["CELERY_CONFIG"])
class ContextTask(celery.Task):
def __call__(self, *args, **kwargs):
def celery_init_app(app: Flask) -> Celery:
class FlaskTask(Task):
def __call__(self, *args: object, **kwargs: object) -> object:
with app.app_context():
return self.run(*args, **kwargs)
celery.Task = ContextTask
return celery
celery_app = Celery(app.name, task_cls=FlaskTask)
celery_app.config_from_object(app.config["CELERY"])
celery_app.set_default()
app.extensions["celery"] = celery_app
return celery_app
The function creates a new Celery object, configures it with the broker
from the application config, updates the rest of the Celery config from
the Flask config and then creates a subclass of the task that wraps the
task execution in an application context.
This creates and returns a ``Celery`` app object. Celery `configuration`_ is taken from
the ``CELERY`` key in the Flask configuration. The Celery app is set as the default, so
that it is seen during each request. The ``Task`` subclass automatically runs task
functions with a Flask app context active, so that services like your database
connections are available.
.. note::
Celery 5.x deprecated uppercase configuration keys, and 6.x will
remove them. See their official `migration guide`_.
.. _configuration: https://celery.readthedocs.io/en/stable/userguide/configuration.html
.. _migration guide: https://docs.celeryproject.org/en/stable/userguide/configuration.html#conf-old-settings-map.
Here's a basic ``example.py`` that configures Celery to use Redis for communication. We
enable a result backend, but ignore results by default. This allows us to store results
only for tasks where we care about the result.
An example task
---------------
Let's write a task that adds two numbers together and returns the result. We
configure Celery's broker and backend to use Redis, create a ``celery``
application using the factory from above, and then use it to define the task. ::
.. code-block:: python
from flask import Flask
flask_app = Flask(__name__)
flask_app.config.update(CELERY_CONFIG={
'broker_url': 'redis://localhost:6379',
'result_backend': 'redis://localhost:6379',
})
celery = make_celery(flask_app)
app = Flask(__name__)
app.config.from_mapping(
CELERY=dict(
broker_url="redis://localhost",
result_backend="redis://localhost",
task_ignore_result=True,
),
)
celery_app = celery_init_app(app)
@celery.task()
def add_together(a, b):
Point the ``celery worker`` command at this and it will find the ``celery_app`` object.
.. code-block:: text
$ celery -A example worker --loglevel INFO
You can also run the ``celery beat`` command to run tasks on a schedule. See Celery's
docs for more information about defining schedules.
.. code-block:: text
$ celery -A example beat --loglevel INFO
Application Factory
-------------------
When using the Flask application factory pattern, call the ``celery_init_app`` function
inside the factory. It sets ``app.extensions["celery"]`` to the Celery app object, which
can be used to get the Celery app from the Flask app returned by the factory.
.. code-block:: python
def create_app() -> Flask:
app = Flask(__name__)
app.config.from_mapping(
CELERY=dict(
broker_url="redis://localhost",
result_backend="redis://localhost",
task_ignore_result=True,
),
)
app.config.from_prefixed_env()
celery_init_app(app)
return app
To use ``celery`` commands, Celery needs an app object, but that's no longer directly
available. Create a ``make_celery.py`` file that calls the Flask app factory and gets
the Celery app from the returned Flask app.
.. code-block:: python
from example import create_app
flask_app = create_app()
celery_app = flask_app.extensions["celery"]
Point the ``celery`` command to this file.
.. code-block:: text
$ celery -A make_celery worker --loglevel INFO
$ celery -A make_celery beat --loglevel INFO
Defining Tasks
--------------
Using ``@celery_app.task`` to decorate task functions requires access to the
``celery_app`` object, which won't be available when using the factory pattern. It also
means that the decorated tasks are tied to the specific Flask and Celery app instances,
which could be an issue during testing if you change configuration for a test.
Instead, use Celery's ``@shared_task`` decorator. This creates task objects that will
access whatever the "current app" is, which is a similar concept to Flask's blueprints
and app context. This is why we called ``celery_app.set_default()`` above.
Here's an example task that adds two numbers together and returns the result.
.. code-block:: python
from celery import shared_task
@shared_task(ignore_result=False)
def add_together(a: int, b: int) -> int:
return a + b
This task can now be called in the background::
Earlier, we configured Celery to ignore task results by default. Since we want to know
the return value of this task, we set ``ignore_result=False``. On the other hand, a task
that didn't need a result, such as sending an email, wouldn't set this.
result = add_together.delay(23, 42)
result.wait() # 65
Run a worker
------------
Calling Tasks
-------------
If you jumped in and already executed the above code you will be
disappointed to learn that ``.wait()`` will never actually return.
That's because you also need to run a Celery worker to receive and execute the
task. ::
The decorated function becomes a task object with methods to call it in the background.
The simplest way is to use the ``delay(*args, **kwargs)`` method. See Celery's docs for
more methods.
$ celery -A your_application.celery worker
A Celery worker must be running to run the task. Starting a worker is shown in the
previous sections.
The ``your_application`` string has to point to your application's package
or module that creates the ``celery`` object.
.. code-block:: python
Now that the worker is running, ``wait`` will return the result once the task
is finished.
from flask import request
@app.post("/add")
def start_add() -> dict[str, object]:
a = request.form.get("a", type=int)
b = request.form.get("b", type=int)
result = add_together.delay(a, b)
return {"result_id": result.id}
The route doesn't get the task's result immediately. That would defeat the purpose by
blocking the response. Instead, we return the running task's result id, which we can use
later to get the result.
Getting Results
---------------
To fetch the result of the task we started above, we'll add another route that takes the
result id we returned before. We return whether the task is finished (ready), whether it
finished successfully, and what the return value (or error) was if it is finished.
.. code-block:: python
from celery.result import AsyncResult
@app.get("/result/<id>")
def task_result(id: str) -> dict[str, object]:
result = AsyncResult(id)
return {
"ready": result.ready(),
"successful": result.successful(),
"value": result.result if result.ready() else None,
}
Now you can start the task using the first route, then poll for the result using the
second route. This keeps the Flask request workers from being blocked waiting for tasks
to finish.
The Flask repository contains `an example <https://github.com/pallets/flask/tree/main/examples/celery>`_
using JavaScript to submit tasks and poll for progress and results.
Passing Data to Tasks
---------------------
The "add" task above took two integers as arguments. To pass arguments to tasks, Celery
has to serialize them to a format that it can pass to other processes. Therefore,
passing complex objects is not recommended. For example, it would be impossible to pass
a SQLAlchemy model object, since that object is probably not serializable and is tied to
the session that queried it.
Pass the minimal amount of data necessary to fetch or recreate any complex data within
the task. Consider a task that will run when the logged in user asks for an archive of
their data. The Flask request knows the logged in user, and has the user object queried
from the database. It got that by querying the database for a given id, so the task can
do the same thing. Pass the user's id rather than the user object.
.. code-block:: python
@shared_task
def generate_user_archive(user_id: str) -> None:
user = db.session.get(User, user_id)
...
generate_user_archive.delay(current_user.id)

2
docs/patterns/packages.rst
View file

@ -67,6 +67,8 @@ To use the ``flask`` command and run your application you need to set
the ``--app`` option that tells Flask where to find the application
instance:
.. code-block:: text
$ flask --app yourapplication run
What did we gain from this? Now we can restructure the application a bit

5
docs/reqcontext.rst
View file

@ -69,11 +69,12 @@ everything that runs in the block will have access to :data:`request`,
populated with your test data. ::
def generate_report(year):
format = request.args.get('format')
format = request.args.get("format")
...
with app.test_request_context(
'/make_report/2017', data={'format': 'short'}):
"/make_report/2017", query_string={"format": "short"}
):
generate_report()
If you see that error somewhere else in your code not related to