Issue Details (XML | Word | Printable)

Key: BATCH-76
Type: New Feature New Feature
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Dave Syer
Reporter: Dave Syer
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
Spring Batch

ItemWriter for SQL batch updates

Created: 03/Aug/07 05:12 AM   Updated: 17/Sep/08 09:10 PM
Component/s: None
Affects Version/s: 1.0-m2
Fix Version/s: 1.0.0.rc1

Time Tracking:
Original Estimate: 0.25d
Original Estimate - 0.25d
Remaining Estimate: 0.25d
Remaining Estimate - 0.25d
Time Spent: Not Specified
Remaining Estimate - 0.25d

Issue Links:
Related
 


 Description  « Hide
It is inefficient to emit updates one per item, but it is also a special case for JDBC that there is a batch update mechanism available. I think it might just be a specialised ItemProcessor.

There is a subtle issue with identifying failed items which is going to make things complicated. Eberhard had some experience with this already using a command pattern in the processing and then dividing the processing up so that the failed item can be identified by binary search (or other O(log(N)) algorithm).

[09:53:51] Eberhard Wolff says:
Say you have 1000 customers and you get a constraint violation at commit.
What you do is that you divide the 1000 in two packages of 500.
One will commit.
So the bad one is in the other.
And then you divide it again and again until you get the faulty one.

 All   Comments   Work Log   Change History   FishEye   Builds      Sort Order: Ascending order - Click to sort in descending order
Dave Syer added a comment - 28/Sep/07 02:22 AM

Dave Syer added a comment - 23/Nov/07 02:46 AM - edited
See BATCH-194 for an approach that works with Hibernate - probably translatable into JDBC pretty easily. Not as fancy as the binary search proposed here, and is O(N), but that might not be a problem for infrequent failures / small chunks. Binary seach might be do-able as a separate item.

Dave Syer added a comment - 06/Mar/08 06:58 AM
Added BatchSqlUpdateItemWriter and a sample. Requires the use of a new interface ItemPreparedStatementSetter and the buffering of items until a flush in the writer.

Dave Syer added a comment - 07/Aug/08 10:06 AM
Assume closed as resolved and released

payne added a comment - 17/Sep/08 09:10 PM
http://www.gnn.com.br/forum/showthread.php?t=1000087664
http://www.gnn.com.br/forum/showthread.php?t=1000087663
http://www.gnn.com.br/forum/showthread.php?t=1000087662
http://www.gnn.com.br/forum/showthread.php?t=1000087661
http://www.gnn.com.br/forum/showthread.php?t=1000087660
http://www.gnn.com.br/forum/showthread.php?t=1000087659
http://www.gnn.com.br/forum/showthread.php?t=1000087658
http://www.gnn.com.br/forum/showthread.php?t=1000087657
http://www.gnn.com.br/forum/showthread.php?t=1000087655
http://www.gnn.com.br/forum/showthread.php?t=1000087654
http://www.gnn.com.br/forum/showthread.php?t=1000087653
http://www.gnn.com.br/forum/showthread.php?t=1000087652
http://www.gnn.com.br/forum/showthread.php?t=1000087651
http://www.gnn.com.br/forum/showthread.php?t=1000087650
http://www.gnn.com.br/forum/showthread.php?t=1000087649
http://www.gnn.com.br/forum/showthread.php?t=1000087647
http://www.gnn.com.br/forum/showthread.php?t=1000087646
http://www.gnn.com.br/forum/showthread.php?t=1000087645
http://www.gnn.com.br/forum/showthread.php?t=1000087643
http://www.gnn.com.br/forum/showthread.php?t=1000087642
http://www.gnn.com.br/forum/showthread.php?t=1000087640
http://www.gnn.com.br/forum/showthread.php?t=1000087639
http://www.gnn.com.br/forum/showthread.php?t=1000087638
http://www.gnn.com.br/forum/showthread.php?t=1000087637
http://www.gnn.com.br/forum/showthread.php?t=1000087636
http://www.gnn.com.br/forum/showthread.php?t=1000087635
http://www.gnn.com.br/forum/showthread.php?t=1000087633
http://www.gnn.com.br/forum/showthread.php?t=1000087554
http://www.gnn.com.br/forum/showthread.php?t=1000087553
http://www.gnn.com.br/forum/showthread.php?t=1000087551
http://www.gnn.com.br/forum/showthread.php?t=1000087550
http://www.gnn.com.br/forum/showthread.php?t=1000087548
http://www.gnn.com.br/forum/showthread.php?t=1000087547
http://www.gnn.com.br/forum/showthread.php?t=1000087546
http://www.gnn.com.br/forum/showthread.php?t=1000087545
http://www.gnn.com.br/forum/showthread.php?t=1000087632
http://www.gnn.com.br/forum/showthread.php?t=1000087631
http://www.gnn.com.br/forum/showthread.php?t=1000087630
http://www.gnn.com.br/forum/showthread.php?t=1000087629
http://www.gnn.com.br/forum/showthread.php?t=1000087628
http://www.gnn.com.br/forum/showthread.php?t=1000087627
http://www.gnn.com.br/forum/showthread.php?t=1000087625
http://www.gnn.com.br/forum/showthread.php?t=1000087624
http://www.gnn.com.br/forum/showthread.php?t=1000087623
http://www.gnn.com.br/forum/showthread.php?t=1000087622
http://www.gnn.com.br/forum/showthread.php?t=1000087621
http://www.gnn.com.br/forum/showthread.php?t=1000087620
http://www.gnn.com.br/forum/showthread.php?t=1000087618
http://www.gnn.com.br/forum/showthread.php?t=1000087617
http://www.gnn.com.br/forum/showthread.php?t=1000087616
http://www.gnn.com.br/forum/showthread.php?t=1000087615
http://www.gnn.com.br/forum/showthread.php?t=1000087614
http://www.gnn.com.br/forum/showthread.php?t=1000087613
http://www.gnn.com.br/forum/showthread.php?t=1000087611
http://www.gnn.com.br/forum/showthread.php?t=1000087610
http://www.gnn.com.br/forum/showthread.php?t=1000087609
http://www.gnn.com.br/forum/showthread.php?t=1000087608
http://www.gnn.com.br/forum/showthread.php?t=1000087607
http://www.gnn.com.br/forum/showthread.php?t=1000087605
http://www.gnn.com.br/forum/showthread.php?t=1000087604
http://www.gnn.com.br/forum/showthread.php?t=1000087603
http://www.gnn.com.br/forum/showthread.php?t=1000087602
http://www.gnn.com.br/forum/showthread.php?t=1000087601
http://www.gnn.com.br/forum/showthread.php?t=1000087600
http://www.gnn.com.br/forum/showthread.php?t=1000087599
http://www.gnn.com.br/forum/showthread.php?t=1000087598
http://www.gnn.com.br/forum/showthread.php?t=1000087597
http://www.gnn.com.br/forum/showthread.php?t=1000087596
http://www.gnn.com.br/forum/showthread.php?t=1000087595
http://www.gnn.com.br/forum/showthread.php?t=1000087592
http://www.gnn.com.br/forum/showthread.php?t=1000087591
http://www.gnn.com.br/forum/showthread.php?t=1000087590
http://www.gnn.com.br/forum/showthread.php?t=1000087588
http://www.gnn.com.br/forum/showthread.php?t=1000087587
http://www.gnn.com.br/forum/showthread.php?t=1000087586
http://www.gnn.com.br/forum/showthread.php?t=1000087585
http://www.gnn.com.br/forum/showthread.php?t=1000087584
http://www.gnn.com.br/forum/showthread.php?t=1000087583
http://www.gnn.com.br/forum/showthread.php?t=1000087582
http://www.gnn.com.br/forum/showthread.php?t=1000087581
http://www.gnn.com.br/forum/showthread.php?t=1000087579
http://www.gnn.com.br/forum/showthread.php?t=1000087578
http://www.gnn.com.br/forum/showthread.php?t=1000087577
http://www.gnn.com.br/forum/showthread.php?t=1000087575
http://www.gnn.com.br/forum/showthread.php?t=1000087574
http://www.gnn.com.br/forum/showthread.php?t=1000087573
http://www.gnn.com.br/forum/showthread.php?t=1000087572
http://www.gnn.com.br/forum/showthread.php?t=1000087571
http://www.gnn.com.br/forum/showthread.php?t=1000087570
http://www.gnn.com.br/forum/showthread.php?t=1000087569
http://www.gnn.com.br/forum/showthread.php?t=1000087568
http://www.gnn.com.br/forum/showthread.php?t=1000087567
http://www.gnn.com.br/forum/showthread.php?t=1000087565
http://www.gnn.com.br/forum/showthread.php?t=1000087564
http://www.gnn.com.br/forum/showthread.php?t=1000087561
http://www.gnn.com.br/forum/showthread.php?t=1000087559
http://www.gnn.com.br/forum/showthread.php?t=1000087558
http://www.gnn.com.br/forum/showthread.php?t=1000087557
http://www.gnn.com.br/forum/showthread.php?t=1000087556
http://www.gnn.com.br/forum/showthread.php?t=1000087555