History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: BATCH-76
Type: New Feature New Feature
Status: Resolved Resolved
Resolution: Fixed
Priority: Major Major
Assignee: Dave Syer
Reporter: Dave Syer
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
Spring Batch

ItemWriter for SQL batch updates

Created: 03/Aug/07 05:12 AM   Updated: 06/Mar/08 06:58 AM
Component/s: None
Affects Version/s: 1.0-m2
Fix Version/s: 1.0.0.rc1

Time Tracking:
Original Estimate: 0.25d
Original Estimate - 0.25d
Remaining Estimate: 0.25d
Remaining Estimate - 0.25d
Time Spent: Not Specified
Remaining Estimate - 0.25d

Issue Links:
Related
 


 Description  « Hide
It is inefficient to emit updates one per item, but it is also a special case for JDBC that there is a batch update mechanism available. I think it might just be a specialised ItemProcessor.

There is a subtle issue with identifying failed items which is going to make things complicated. Eberhard had some experience with this already using a command pattern in the processing and then dividing the processing up so that the failed item can be identified by binary search (or other O(log(N)) algorithm).

[09:53:51] Eberhard Wolff says:
Say you have 1000 customers and you get a constraint violation at commit.
What you do is that you divide the 1000 in two packages of 500.
One will commit.
So the bad one is in the other.
And then you divide it again and again until you get the faulty one.

 All   Comments   Work Log   Change History   FishEye   Related Builds      Sort Order: Ascending order - Click to sort in descending order

Dave Syer - 23/Nov/07 02:46 AM - edited
See BATCH-194 for an approach that works with Hibernate - probably translatable into JDBC pretty easily. Not as fancy as the binary search proposed here, and is O(N), but that might not be a problem for infrequent failures / small chunks. Binary seach might be do-able as a separate item.

Dave Syer - 06/Mar/08 06:58 AM
Added BatchSqlUpdateItemWriter and a sample. Requires the use of a new interface ItemPreparedStatementSetter and the buffering of items until a flush in the writer.