Issue Details (XML | Word | Printable)

Key: BATCH-504
Type: New Feature New Feature
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Robert Kasanicky
Reporter: Dave Syer
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
Spring Batch

Provide text file based implementation of Daos

Created: 24/Mar/08 11:34 AM   Updated: 07/Aug/08 10:07 AM
Component/s: Core
Affects Version/s: 1.0.0.rc1
Fix Version/s: 1.0.1

Time Tracking:
Original Estimate: 1d
Original Estimate - 1d
Remaining Estimate: 1d
Remaining Estimate - 1d
Time Spent: Not Specified
Remaining Estimate - 1d


 Description  « Hide
Provide text file based implementation of Daos. It would lower the barrier to entry, and make it much easier for people to pick up Spring Batch and try it out. In many cases the benefit of using an RDBMS for the batch meta-data will not be worth the effort of installing and maintaining a database instance (e.g. for jobs involving only files).

 All   Comments   Work Log   Change History   FishEye   Builds      Sort Order: Ascending order - Click to sort in descending order
Robert Kasanicky added a comment - 24/Mar/08 02:03 PM
How about simply providing an example how to configure hsqldb to use disk-based tables? I don't see a benefit in having text file daos compared to bundled database (be it hsql, derby or h2).

Wayne Lund added a comment - 24/Mar/08 10:28 PM
I agree with Robert. It doesn't get any easier than hsqldb as far as relational database and I'm not sure the advantage of a file dao.That being said, I do know that the resources used with persistence in projects like EMF use XML storage in an XMI format. I personally don't think its much easier than a db but it supplies the necessary behavior of query support through uri's etc. Is that what you were thinking?

Dave Syer added a comment - 25/Mar/08 12:43 AM
I think HSQL files are a great idea. Robert: can you look into it a bit and post back a patch for the samples, perhaps? I think the key questions are:

1) are the files human readable?
2) can they be archived and uploaded into a "real" database as a migration option?

Robert Kasanicky added a comment - 25/Mar/08 03:11 AM
The HSQL files I've seen are basically a log of database operations (CREATE, INSERT), so they are human-readable. I'm not sure how easy the uploading to "real" database would be given the discrepancies in SQL syntax.

Anyway, I'll dig into this post 1.0.

Dave Syer added a comment - 25/Mar/08 03:31 AM
I guess you could always do an upload to another database using Spring Batch!

Robert Kasanicky added a comment - 11/Apr/08 04:55 AM
Configuring HSQL to use files instead of pure in-memory mode is trivial - just replace "mem" with "file" in the uri and HSQLDB will create the files in the current directory (catch - remove the initializing datasource wrapper after first run).

However things don't really work as expected. Both STEP_EXECUTION and JOB_EXECUTION entries have status 'UNKNOWN' although the job ran successfully. The entry for step execution is overwritten for each step i.e. eventually there is entry only for the last step. When I tried running second multi-step job, it suddenly had entry for each step (but still all statuses 'UNKNOWN'). Third job didn't create any entry although I ran it several times.

Any ideas what might cause the database to behave so unpredictably (it works fine when run as standalone server)? I'll take a look at Derby and H2, one of them will hopefully just work.

Dave Syer added a comment - 11/Apr/08 05:16 AM
I think maybe you just need to be careful to shutdown the database properly when the VM / ApplicationContext exits.

Robert Kasanicky added a comment - 11/Apr/08 07:24 AM
I tried switching to embedded Derby (we already provide everything necessary) and it works well (with the exception of BATCH-577).

@Dave: if proper shutdown was the issue shouldn't we have the same trouble using 'mem' protocol (e.g. restart sample wouldn't work)?

Dave Syer added a comment - 11/Apr/08 07:54 AM
No, that wouldn't be an issue because the VM is never stopped (the database is always active in all tests). It isn't a "real" restart, in the sense that the VM stops and then starts again. It's a "simulated" restart.

(N.B. you can switch to embedded Derby by commenting out a line in data-source-context.xml:

<!--import resource="alt-data-source-context.xml" /-->

.)

Dave Syer added a comment - 11/Apr/08 08:30 AM
No, that wouldn't be an issue because the VM is never stopped (the database is always active in all tests). It isn't a "real" restart, in the sense that the VM stops and then starts again. It's a "simulated" restart.

(N.B. you can switch to embedded Derby by commenting out a line in data-source-context.xml:

<!--import resource="alt-data-source-context.xml" /-->

.)

Robert Kasanicky added a comment - 14/Apr/08 08:26 AM
Dave,

you were right about the proper shutdown - I tried adding a DisposableBean that issues "SHUTDOWN" to the hsql database and everything works fine that way.

Robert Kasanicky added a comment - 28/Apr/08 05:37 AM
I think we can close this issue? And maybe start a new one to document how to setup embedded database.

Dave Syer added a comment - 28/Apr/08 06:11 AM
I think this is done (open a new one for the docos if you like - it would probably just be a code snippet for your DispoableBean and a reference to HSQLDB docs?).

Dave Syer added a comment - 07/Aug/08 10:07 AM
Assume closed as resolved and released