We are experimenting with a spring batch single thread to spring batch multithread.
The setup is rather easy :
- Reader : Read some data with a specific sorting.
- Processor : fetch some additional values.
- Writer : Write it all to an csv file in order of read.
So we have changed the reader to a JdbcPagingItemReader
and converted the sort from
order by firstname, lastname, id;
to
Map<String, Order> sortConfiguration = new HashMap<>();
sortConfiguration.put("firstname", Order.ASCENDING);
sortConfiguration.put("lastname", Order.ASCENDING);
sortConfiguration.put("id", Order.ASCENDING);
the commit-interval
is set to 200.
The batch runs fine, but our csv is completly out of order.
I assumed that spring would write in the file after each commit (and hoped that he write page per page in order), but the disorder is greater then chunks 200 lines.
I got for example line 1, 3 and 5 should be together in a thread and line 2 and 4 in another thread.
Is there any option to preserve the order or is the only way to abandon multithread?
question from:
https://stackoverflow.com/questions/65916086/spring-batch-multithread-same-sort-in-reader-as-writer 与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…