Task with bind=True
I have a celery task that runs a computation lasting a few seconds.
from celery import states
@celery.task(name="crunch.task", bind=True)
def crunch(self, data):
try:
pass
# ... run computation with data here
except Exception as exc:
self.update_state(
state=states.FAILURE,
meta={"details": "error details here"}
)
raise exc
The important feature here is I'm using bind=True
which passes the task into the function as the self
parameter. This allows access to the task's update_state
method which is great for error handling.
Group job
I now want to run this task in a batch job using celery.group
.
@celery.task(name="batch.task", bind=True)
def batch(self, data_list):
try:
# HERE! IT SEEM WRONG TO PASS `SELF` INTO THE CHILD TASKS
job = group([crunch(self, data) for data in data_list]) # <-- here `self` should be created by celery!
job.async_apply()
except Exception as exc:
self.update_state(
state=states.FAILURE,
meta={"details": "error details here"}
)
raise exc
How can I make a celery.group
composed of tasks with bind=True
?
question from:
https://stackoverflow.com/questions/65891987/how-to-make-a-group-from-tasks-with-bind-true 与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…