forked from orbit-oss/flask
Added celery pattern
This commit is contained in:
parent
3fdcefbcda
commit
05f162329d
2 changed files with 93 additions and 0 deletions
92
docs/patterns/celery.rst
Normal file
92
docs/patterns/celery.rst
Normal file
|
|
@ -0,0 +1,92 @@
|
||||||
|
Celery Based Background Tasks
|
||||||
|
=============================
|
||||||
|
|
||||||
|
Celery is a task queue for Python with batteries included. It used to
|
||||||
|
have a Flask integration but it became unnecessary after some
|
||||||
|
restructuring of the internals of Celery with Version 3. This guide fills
|
||||||
|
in the blanks in how to properly use Celery with Flask but assumes that
|
||||||
|
you generally already read the `First Steps with Celery
|
||||||
|
<http://docs.celeryproject.org/en/master/getting-started/first-steps-with-celery.html>`_
|
||||||
|
guide in the official Celery documentation.
|
||||||
|
|
||||||
|
Installing Celery
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
Celery is on the Python Package Index (PyPI), so it can be installed with
|
||||||
|
standard Python tools like ``pip`` or ``easy_install``::
|
||||||
|
|
||||||
|
$ pip install celery
|
||||||
|
|
||||||
|
Configuring Celery
|
||||||
|
------------------
|
||||||
|
|
||||||
|
The first thing you need is a Celery instance, this is called the celery
|
||||||
|
application. It serves the same purpose as the :class:`~flask.Flask`
|
||||||
|
object in Flask, just for Celery. Since this instance is used as the
|
||||||
|
entry-point for everything you want to do in Celery, like creating tasks
|
||||||
|
and managing workers, it must be possible for other modules to import it.
|
||||||
|
|
||||||
|
For instance you can place this in a ``tasks`` module. While you can use
|
||||||
|
Celery without any reconfiguration with Flask, it becomes a bit nicer by
|
||||||
|
subclassing tasks and adding support for Flask's application contexts and
|
||||||
|
hooking it up with the Flask configuration.
|
||||||
|
|
||||||
|
This is all that is necessary to properly integrate Celery with Flask::
|
||||||
|
|
||||||
|
from celery import Celery
|
||||||
|
|
||||||
|
def make_celery(app):
|
||||||
|
celery = Celery(app.import_name, broker=app.config['CELERY_BROKER_URL'])
|
||||||
|
celery.conf.update(app.config)
|
||||||
|
TaskBase = celery.Task
|
||||||
|
class ContextTask(TaskBase):
|
||||||
|
abstract = True
|
||||||
|
def __call__(self, *args, **kwargs):
|
||||||
|
with app.app_context():
|
||||||
|
return TaskBase.__call__(self, *args, **kwargs)
|
||||||
|
celery.Task = ContextTask
|
||||||
|
return celery
|
||||||
|
|
||||||
|
The function creates a new Celery object, configures it with the broker
|
||||||
|
from the application config, updates the rest of the Celery config from
|
||||||
|
the Flask config and then creates a subclass of the task that wraps the
|
||||||
|
task execution in an application context.
|
||||||
|
|
||||||
|
Minimal Example
|
||||||
|
---------------
|
||||||
|
|
||||||
|
With what we have above this is the minimal example of using Celery with
|
||||||
|
Flask::
|
||||||
|
|
||||||
|
from flask import Flask
|
||||||
|
|
||||||
|
app = Flask(__name__)
|
||||||
|
app.config.update(
|
||||||
|
CELERY_BROKER_URL='redis://localhost:6379',
|
||||||
|
CELERY_RESULT_BACKEND='redis://localhost:6379'
|
||||||
|
)
|
||||||
|
celery = make_celery(app)
|
||||||
|
|
||||||
|
|
||||||
|
@celery.task()
|
||||||
|
def add_together(a, b):
|
||||||
|
return a + b
|
||||||
|
|
||||||
|
This task can now be called in the background:
|
||||||
|
|
||||||
|
>>> result = add_together.delay(23, 42)
|
||||||
|
>>> result.wait()
|
||||||
|
65
|
||||||
|
|
||||||
|
Running the Celery Worker
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
Now if you jumped in and already executed the above code you will be
|
||||||
|
disappointed to learn that your ``.wait()`` will never actually return.
|
||||||
|
That's because you also need to run celery. You can do that by running
|
||||||
|
celery as a worker::
|
||||||
|
|
||||||
|
$ celery -A your_application worker
|
||||||
|
|
||||||
|
The ``your_application`` string has to point to your application's package
|
||||||
|
or module that creates the `celery` object.
|
||||||
|
|
@ -39,3 +39,4 @@ Snippet Archives <http://flask.pocoo.org/snippets/>`_.
|
||||||
deferredcallbacks
|
deferredcallbacks
|
||||||
methodoverrides
|
methodoverrides
|
||||||
requestchecksum
|
requestchecksum
|
||||||
|
celery
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue