Python Elasticsearch: Conditional batch upsert

Question

Using the python library, I am trying to do the following to an Elasticsearch index:

I am using the batch API to collect documents from a few different sources. Possibly even within one batch, some documents might be assigned the same id, and under some circumstances should replace an existing document with the same id.

For some reason, it seems not be possible to check for a lower version, or seq_no, but only for a concrete value. However in my case this is sufficient as fortunately I have only two cases.

So if I find that a document qualifies as higher priority I thought to assign it _seq_no:1 in the batch body, otherwise 0 and add _if_seq_no:0 in the batch body.

_body = { '_op_type':   'update',
          '_if_seq_no': 0,
          '_index':     INDEX,
          '_id':        None,
          '_seq_no':    None,
          '_source':    {'doc': { ... } }
        }

In the generator that feeds my batch updates, I replace _body['_id'] and _body['_seq_no'].

This was my idea how to solve the problem, in particularly after reading this recent unresolved discussion.

However, at this point I get the following error:

raise HTTP_EXCEPTIONS.get(status_code, TransportError)(elasticsearch.exceptions.RequestError: RequestError(400, 'action_request_validation_exception', 'Validation Failed: 1: ifSeqNo is set, but primary term is [0];2: ifSeqNo is set, but primary term is [0];3: ifSeqNo is set, but primary term is [0];4: ifSeqNo is set, but primar...

Am I following a sensible approach and do you have suggestions on how to complete it?

By the way, does update work if no document with the respective id exists? Before I was using create, but it said create operations do not support compare and set. use index instead.

question from:https://stackoverflow.com/questions/65915545/python-elasticsearch-conditional-batch-upsert

Answer 1

Waitting for answers