forked from orbit-oss/flask
explain when to use a task queue
remove deprecated abstract attr from celery add explanation of example task [ci skip]
This commit is contained in:
parent
f13e3fc352
commit
d5a88bf0d3
1 changed files with 43 additions and 35 deletions
|
|
@ -1,24 +1,27 @@
|
||||||
Celery Based Background Tasks
|
Celery Background Tasks
|
||||||
=============================
|
=======================
|
||||||
|
|
||||||
Celery is a task queue for Python with batteries included. It used to
|
If your application has a long running task, such as processing some uploaded
|
||||||
have a Flask integration but it became unnecessary after some
|
data or sending email, you don't want to wait for it to finish during a
|
||||||
restructuring of the internals of Celery with Version 3. This guide fills
|
request. Instead, use a task queue to send the necessary data to another
|
||||||
in the blanks in how to properly use Celery with Flask but assumes that
|
process that will run the task in the background while the request returns
|
||||||
you generally already read the `First Steps with Celery
|
immediately.
|
||||||
<http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html>`_
|
|
||||||
guide in the official Celery documentation.
|
|
||||||
|
|
||||||
Installing Celery
|
Celery is a powerful task queue that can be used for simple background tasks
|
||||||
-----------------
|
as well as complex multi-stage programs and schedules. This guide will show you
|
||||||
|
how to configure Celery using Flask, but assumes you've already read the
|
||||||
|
`First Steps with Celery <http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html>`_
|
||||||
|
guide in the Celery documentation.
|
||||||
|
|
||||||
Celery is on the Python Package Index (PyPI), so it can be installed with
|
Install
|
||||||
standard Python tools like :command:`pip` or :command:`easy_install`::
|
-------
|
||||||
|
|
||||||
|
Celery is a separate Python package. Install it from PyPI using pip::
|
||||||
|
|
||||||
$ pip install celery
|
$ pip install celery
|
||||||
|
|
||||||
Configuring Celery
|
Configure
|
||||||
------------------
|
---------
|
||||||
|
|
||||||
The first thing you need is a Celery instance, this is called the celery
|
The first thing you need is a Celery instance, this is called the celery
|
||||||
application. It serves the same purpose as the :class:`~flask.Flask`
|
application. It serves the same purpose as the :class:`~flask.Flask`
|
||||||
|
|
@ -36,15 +39,18 @@ This is all that is necessary to properly integrate Celery with Flask::
|
||||||
from celery import Celery
|
from celery import Celery
|
||||||
|
|
||||||
def make_celery(app):
|
def make_celery(app):
|
||||||
celery = Celery(app.import_name, backend=app.config['CELERY_RESULT_BACKEND'],
|
celery = Celery(
|
||||||
broker=app.config['CELERY_BROKER_URL'])
|
app.import_name,
|
||||||
|
backend=app.config['CELERY_RESULT_BACKEND'],
|
||||||
|
broker=app.config['CELERY_BROKER_URL']
|
||||||
|
)
|
||||||
celery.conf.update(app.config)
|
celery.conf.update(app.config)
|
||||||
TaskBase = celery.Task
|
|
||||||
class ContextTask(TaskBase):
|
class ContextTask(celery.Task):
|
||||||
abstract = True
|
|
||||||
def __call__(self, *args, **kwargs):
|
def __call__(self, *args, **kwargs):
|
||||||
with app.app_context():
|
with app.app_context():
|
||||||
return self.run(*args, **kwargs)
|
return self.run(*args, **kwargs)
|
||||||
|
|
||||||
celery.Task = ContextTask
|
celery.Task = ContextTask
|
||||||
return celery
|
return celery
|
||||||
|
|
||||||
|
|
@ -53,11 +59,12 @@ from the application config, updates the rest of the Celery config from
|
||||||
the Flask config and then creates a subclass of the task that wraps the
|
the Flask config and then creates a subclass of the task that wraps the
|
||||||
task execution in an application context.
|
task execution in an application context.
|
||||||
|
|
||||||
Minimal Example
|
An example task
|
||||||
---------------
|
---------------
|
||||||
|
|
||||||
With what we have above this is the minimal example of using Celery with
|
Let's write a task that adds two numbers together and returns the result. We
|
||||||
Flask::
|
configure Celery's broker and backend to use Redis, create a ``celery``
|
||||||
|
application using the factor from above, and then use it to define the task. ::
|
||||||
|
|
||||||
from flask import Flask
|
from flask import Flask
|
||||||
|
|
||||||
|
|
@ -68,26 +75,27 @@ Flask::
|
||||||
)
|
)
|
||||||
celery = make_celery(flask_app)
|
celery = make_celery(flask_app)
|
||||||
|
|
||||||
|
|
||||||
@celery.task()
|
@celery.task()
|
||||||
def add_together(a, b):
|
def add_together(a, b):
|
||||||
return a + b
|
return a + b
|
||||||
|
|
||||||
This task can now be called in the background:
|
This task can now be called in the background::
|
||||||
|
|
||||||
>>> result = add_together.delay(23, 42)
|
result = add_together.delay(23, 42)
|
||||||
>>> result.wait()
|
result.wait() # 65
|
||||||
65
|
|
||||||
|
|
||||||
Running the Celery Worker
|
Run a worker
|
||||||
-------------------------
|
------------
|
||||||
|
|
||||||
Now if you jumped in and already executed the above code you will be
|
If you jumped in and already executed the above code you will be
|
||||||
disappointed to learn that your ``.wait()`` will never actually return.
|
disappointed to learn that ``.wait()`` will never actually return.
|
||||||
That's because you also need to run celery. You can do that by running
|
That's because you also need to run a Celery worker to receive and execute the
|
||||||
celery as a worker::
|
task. ::
|
||||||
|
|
||||||
$ celery -A your_application.celery worker
|
$ celery -A your_application.celery worker
|
||||||
|
|
||||||
The ``your_application`` string has to point to your application's package
|
The ``your_application`` string has to point to your application's package
|
||||||
or module that creates the `celery` object.
|
or module that creates the ``celery`` object.
|
||||||
|
|
||||||
|
Now that the worker is running, ``wait`` will return the result once the task
|
||||||
|
is finished.
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue