explain when to use a task queue
remove deprecated abstract attr from celery add explanation of example task [ci skip]
This commit is contained in:
parent
f13e3fc352
commit
d5a88bf0d3
1 changed files with 43 additions and 35 deletions
|
|
@ -1,24 +1,27 @@
|
|||
Celery Based Background Tasks
|
||||
=============================
|
||||
Celery Background Tasks
|
||||
=======================
|
||||
|
||||
Celery is a task queue for Python with batteries included. It used to
|
||||
have a Flask integration but it became unnecessary after some
|
||||
restructuring of the internals of Celery with Version 3. This guide fills
|
||||
in the blanks in how to properly use Celery with Flask but assumes that
|
||||
you generally already read the `First Steps with Celery
|
||||
<http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html>`_
|
||||
guide in the official Celery documentation.
|
||||
If your application has a long running task, such as processing some uploaded
|
||||
data or sending email, you don't want to wait for it to finish during a
|
||||
request. Instead, use a task queue to send the necessary data to another
|
||||
process that will run the task in the background while the request returns
|
||||
immediately.
|
||||
|
||||
Installing Celery
|
||||
-----------------
|
||||
Celery is a powerful task queue that can be used for simple background tasks
|
||||
as well as complex multi-stage programs and schedules. This guide will show you
|
||||
how to configure Celery using Flask, but assumes you've already read the
|
||||
`First Steps with Celery <http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html>`_
|
||||
guide in the Celery documentation.
|
||||
|
||||
Celery is on the Python Package Index (PyPI), so it can be installed with
|
||||
standard Python tools like :command:`pip` or :command:`easy_install`::
|
||||
Install
|
||||
-------
|
||||
|
||||
Celery is a separate Python package. Install it from PyPI using pip::
|
||||
|
||||
$ pip install celery
|
||||
|
||||
Configuring Celery
|
||||
------------------
|
||||
Configure
|
||||
---------
|
||||
|
||||
The first thing you need is a Celery instance, this is called the celery
|
||||
application. It serves the same purpose as the :class:`~flask.Flask`
|
||||
|
|
@ -36,15 +39,18 @@ This is all that is necessary to properly integrate Celery with Flask::
|
|||
from celery import Celery
|
||||
|
||||
def make_celery(app):
|
||||
celery = Celery(app.import_name, backend=app.config['CELERY_RESULT_BACKEND'],
|
||||
broker=app.config['CELERY_BROKER_URL'])
|
||||
celery = Celery(
|
||||
app.import_name,
|
||||
backend=app.config['CELERY_RESULT_BACKEND'],
|
||||
broker=app.config['CELERY_BROKER_URL']
|
||||
)
|
||||
celery.conf.update(app.config)
|
||||
TaskBase = celery.Task
|
||||
class ContextTask(TaskBase):
|
||||
abstract = True
|
||||
|
||||
class ContextTask(celery.Task):
|
||||
def __call__(self, *args, **kwargs):
|
||||
with app.app_context():
|
||||
return self.run(*args, **kwargs)
|
||||
|
||||
celery.Task = ContextTask
|
||||
return celery
|
||||
|
||||
|
|
@ -53,11 +59,12 @@ from the application config, updates the rest of the Celery config from
|
|||
the Flask config and then creates a subclass of the task that wraps the
|
||||
task execution in an application context.
|
||||
|
||||
Minimal Example
|
||||
An example task
|
||||
---------------
|
||||
|
||||
With what we have above this is the minimal example of using Celery with
|
||||
Flask::
|
||||
Let's write a task that adds two numbers together and returns the result. We
|
||||
configure Celery's broker and backend to use Redis, create a ``celery``
|
||||
application using the factor from above, and then use it to define the task. ::
|
||||
|
||||
from flask import Flask
|
||||
|
||||
|
|
@ -68,26 +75,27 @@ Flask::
|
|||
)
|
||||
celery = make_celery(flask_app)
|
||||
|
||||
|
||||
@celery.task()
|
||||
def add_together(a, b):
|
||||
return a + b
|
||||
|
||||
This task can now be called in the background:
|
||||
This task can now be called in the background::
|
||||
|
||||
>>> result = add_together.delay(23, 42)
|
||||
>>> result.wait()
|
||||
65
|
||||
result = add_together.delay(23, 42)
|
||||
result.wait() # 65
|
||||
|
||||
Running the Celery Worker
|
||||
-------------------------
|
||||
Run a worker
|
||||
------------
|
||||
|
||||
Now if you jumped in and already executed the above code you will be
|
||||
disappointed to learn that your ``.wait()`` will never actually return.
|
||||
That's because you also need to run celery. You can do that by running
|
||||
celery as a worker::
|
||||
If you jumped in and already executed the above code you will be
|
||||
disappointed to learn that ``.wait()`` will never actually return.
|
||||
That's because you also need to run a Celery worker to receive and execute the
|
||||
task. ::
|
||||
|
||||
$ celery -A your_application.celery worker
|
||||
|
||||
The ``your_application`` string has to point to your application's package
|
||||
or module that creates the `celery` object.
|
||||
or module that creates the ``celery`` object.
|
||||
|
||||
Now that the worker is running, ``wait`` will return the result once the task
|
||||
is finished.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue