Issue Details (XML | Word | Printable)

Key: BATCH-116
Type: New Feature New Feature
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Lucas Ward
Reporter: Peter Zozom
Votes: 0
Watchers: 4
Operations

If you were logged in you would be able to see more operations.
Spring Batch

Create XML input/output source which will work directly with StAX parser

Created: 28/Aug/07 09:06 AM   Updated: 25/Sep/07 08:59 PM
Component/s: Infrastructure
Affects Version/s: 1.0-m2
Fix Version/s: 1.0-m2
Security Level: Public (Public Issues)

Time Tracking:
Not Specified

File Attachments: 1. Text File infrastructure-more-tests.patch (6 kB)
2. Text File new-xml-io-sources-070918.patch (667 kB)
3. Zip Archive new-xml-io-sources-with-tests.zip (110 kB)
4. Text File new-xml-io-sources.patch (111 kB)
5. Text File newStaxInputSource.patch (45 kB)
6. Text File stax-sample.patch (6 kB)
7. Text File stax.patch (26 kB)

Issue Links:
Related
 


 Description  « Hide
Create XML input/output source which will work directly with StAX parser.
Requirements for the I/O sources are:
1. they will work directly with StAX - this means i/o sources will be responsible for managing StAX reader/writer (open,close and read)
2. they won't use any OXM directly, but should allow to plug in any of major OXM frameworks
3. they will support skip, restart, transactions and statistics
4. Input source will support validation against XSD schema

 All   Comments   Work Log   Change History   FishEye   Builds      Sort Order: Ascending order - Click to sort in descending order
Peter Zozom added a comment - 28/Aug/07 09:18 AM
I have created new Xml I/O source (see attached patch) which use StAX directly and allow to plug-in any of OXM framework supported by Spring-WS.

I have tested this code with Jaxb2 and it works correctly, but I was not able to get I/O sources working correctly with other OXM frameworks (XMLBeans, Jibx, Jaxb1, Castor and Xstream).
Problem is with turning off the xml declaration. For example XmlBeans is always writing start document and end document event for each record written (although I have configured marshaller to suppress xml declaration with setSaveNoXmlDecl()).

Also I have created sample job (staxJob) which demonstrates how to use new sources. Currently it uses XmlBeans for unmarshalling and marshalling.

Dave Syer added a comment - 29/Aug/07 03:18 AM
Peter,

Looks nice. But the patch only contains one unit test - and that is a sample job. Can you attach some test cases that test the features, and show the problems with Xstream etc.?

Also what is happening with BATCH-113 - do you have a recommendation for what should be kept and what discarded in the xml/io packages. This is a blocker for releasing 1.0-m2 so I would really like to get someone's input, who was involved in the technical discussions.

Dave.

Peter Zozom added a comment - 30/Aug/07 05:30 AM
Dave,

I will create unit tests for input and output source and also I will create samples for each Marshaller/Unmarshaller implementation showing all problems and I will attach them here as patch. This week I'm out of office so I will do it next week (on Monday or Tuesday).

Regarding BATCH-113, once all issues with new implementations will be resolved and performance for these i/o sources will be satisfactory, we can remove XmlInputSource, XmlInputSource2 and XmlOutputSource and their suport classes from the Infrastructure.

Peter

Arjen Poutsma added a comment - 30/Aug/07 09:46 AM
There seems to be something wrong with the path: all changes are included twice. Peter, could you create a new patch, which only has the changes once? Makes it easier for me to review.

Thanks,

Arjen

Robert Kasanicky added a comment - 05/Sep/07 06:15 AM
With Peter we have added tests for all supported OXMs, for both input and output. Output is problematic, only JiBX currently creates well-formed xml.

The tests are attached as patch.

Arjen Poutsma added a comment - 15/Sep/07 06:19 AM - edited
The code in attached patch logs and rethrows exception that occurs. This is a bad practice: every exception will be logged multiple times this way.

My suggestion is to remove the logging statements from the catch{...} blocks, and simply wrap and rethrow exceptions.

Robert Kasanicky added a comment - 17/Sep/07 09:39 AM
Attaching the refactored stax input source

Robert Kasanicky added a comment - 18/Sep/07 09:37 AM
Attached patch (new-xml-io-sources-070918.patch) contains:
- refactored IO sources
- Unit tests
- Integration tests

Notes:
- integration tests for output source are excluded due to failing XMLUnit asserts (fail although the created xml files are correct)
- Jibx and XmlBeans tests run only from command line (Eclipse does not see the generated sources/classes in target folder)
- Trade class is still in integration/src/main because Jibx maven plugin can't see it otherwise

Robert Kasanicky added a comment - 24/Sep/07 10:17 AM
Attached patch contains minor refactorings of the stax package internals and increased test coverage (infrastructure subproject).

The patch file format is different from the previously submitted patches (this one was made by mercurial), but I applied it smoothly with subclipse (with the help of "ignore leading path segments").

Robert Kasanicky added a comment - 25/Sep/07 06:37 AM
Attached patches contain:
 - improved test coverage for stax package
 - sample job output validation

Lucas Ward added a comment - 25/Sep/07 08:59 PM
Stax input and output sources have been added with sufficient unit test coverage (86% in the stax package) along with a sample job showing usage. Integration tests for both Caster and XStream have been added as well. One of the attached patches includes tests for other OXM frameworks, but they were not included because of issues running them. A new issue will be raised against m3 to add additional OXM integration tests.