Dave Syer added a comment - 18/Aug/08 11:52 AM

Dave Syer added a comment - 18/Aug/08 11:52 AM Added TransactionAwareBufferedWriter and used it in file writers. Flush and commit. Sunny day is always fine, but failure and rollback are less obvious. When is file pointer and other state updated?   1) Flush fails in TX synch. TX will roll back but state is already updated unless it is also managed in a TX synch.   2) Flush in TX synch and commit fails. TX may roll back depending on TX manager. State is already updated. 2) is more of a corner case, but without an XA file resource we are not going to make any progress. 1) is relatively mainstream. 1) only applies to flat file and xml writer cases (there is no persistent state in other writers). If the flush fails then the persistent state will be fine after the rollback, but the local volatile state will be inconsistent, so processing cannot continue (retry or skip). To fix that we would have to read the volatile state back from the database at the start of every chunk. On the other hand... if a flush has failed, the process in trouble for loads of other reasons. E.g. there is no way to know the contents of a file after such a failure. We typically deal with such pathologies by truncating on restart - means that the failure has to be fatal (e.g. use FlushFailedException). In which case there is no need to worry about the volatile state either - it won't be used if the exception is always treated as fatal.