Issue Details (XML | Word | Printable)

Key: BATCH-640
Type: Improvement Improvement
Status: Closed Closed
Resolution: Won't Fix
Priority: Major Major
Assignee: Robert Kasanicky
Reporter: Gaetan Pitteloud
Votes: 1
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
Spring Batch

FieldSetMapper.mapLine() should contain the line number

Created: 26/May/08 06:55 AM   Updated: 03/Oct/08 03:42 AM
Component/s: Infrastructure
Affects Version/s: 1.0.1
Fix Version/s: 2.0.0.M2

Time Tracking:
Original Estimate: 0.25d
Original Estimate - 0.25d
Remaining Estimate: 0.25d
Remaining Estimate - 0.25d
Time Spent: Not Specified
Remaining Estimate - 0.25d

Issue Links:
Depends
 


 Description  « Hide
As mentioned in the javadoc for FieldSet, this interface is the file-equivalent of the jdbc ResultSet. spring-batch provides a mapper interface -FieldSetMapper- for mapping the FieldSet to an object, in the same way as spring-jdbc provides a RowMapper interface used to map a ResultSet to an Object.

This equivalence is very useful and makes sense, but I would appreciate to receive the line number as a parameter to the method, in the same way spring-jdbc does for its RowMapper interface:

public interface FieldSetMapper {

/**
* Method used to map data obtained from a file into an object.
*/
public Object mapLine(FieldSet fs, int lineNum);
}


In FlatFileItemReader :
public Object read() throws Exception {
String line = readLine();

if (line != null) {
int lineCount = getReader().getPosition();
try {
FieldSet tokenizedLine = tokenizer.tokenize(line);
return fieldSetMapper.mapLine(tokenizedLine, lineCount);
}
catch (RuntimeException ex) {
throw new FlatFileParseException("Parsing error at line: " + lineCount + " in resource="
+ resource.getDescription() + ", input=[" + line + "]", ex, line, lineCount);
}
}
return null;
}

I need the line number in order to create the object that is mapped to the FieldSet. There is no way to do this except by creating my own FieldSetMapper interface and use it in my own FlatFileItemReader implementation, that overrides your FlatFileItemReader's read() method.

 All   Comments   Work Log   Change History   FishEye   Builds      Sort Order: Ascending order - Click to sort in descending order
Dave Syer added a comment - 26/May/08 07:11 AM
It's pretty unusual I would say to need the line number to create a domain object (what would it mean in business terms?). Can't you add a column to the file that is being read with the extra information you need?

Gaetan Pitteloud added a comment - 26/May/08 07:42 AM
I am not really constructing a pure domain object, but rather a holder for a domain object that contains additional technical information attached to it; the goal is to be aware of the line number we are treating in the writer.

Dave Syer added a comment - 26/May/08 10:45 AM
So, can't you add a column to the input file?

Gaetan Pitteloud added a comment - 28/May/08 04:09 AM
This is not possible to add a column to the input file, as I do not control it (I receive it from a third party). Anyway, I cannot imagine a file that contains the line number as the first field of each line; this information must be held by the parser of the file itself (this is actually what LineReader.getPosition() does).

Further, holding line number in the file would prevent any operation on it : split file in smaller portions, re-order, remove or add lines.

Dave Syer added a comment - 05/Jun/08 09:59 AM
Added rownum parameter to field set mapper (with default value -1 defined as constant in FieldSetMapper).

Lucas Ward added a comment - 05/Jun/08 10:28 AM
Dave, can we really make a change like that in a point release? Won't this change break everyone's FieldSetMappers?

Dave Syer added a comment - 05/Jun/08 10:46 AM
It was marked for 1.1. Yes it will break everyone's FieldSetMappers. You think we should revert and push this out to 2.0?

Dave Syer added a comment - 05/Jun/08 12:24 PM
reverted changes and updated to 2.0 fix version

Dave Syer added a comment - 17/Jul/08 04:35 AM
Added lineNum parameter back to FieldSetMapper

Dave Syer added a comment - 05/Aug/08 04:57 PM
We decided to pull the linenum back out of the FieldSetMapper (the analogy with RowMapper is not helpful), and make the line and line number (range) available through FieldSetMetaData (which is only available on read).

Dave Syer added a comment - 09/Aug/08 09:13 AM
Assigned to Lucas because it is linked to BATCH-719

Robert Kasanicky added a comment - 02/Oct/08 03:40 AM
This issue is superceded by BATCH-719

Robert Kasanicky added a comment - 02/Oct/08 04:07 AM
closed inappropriately - BATCH-719 doesn't deal with the line number

Robert Kasanicky added a comment - 02/Oct/08 04:49 AM
> I am not really constructing a pure domain object, but rather a holder for a domain object that contains additional technical information attached to it; the goal is to be aware of the line number we are treating in the writer.

Gaetan, would implementing ItemReadListener and counting reads do the job? Passing the line number from the input file to writer through the item inevitably leads to spaghetti code I think. The question only is whether the spaghetti is in user's or framework's code - line number needs to cut through all abstractions between the raw line and domain object. There must be a better way (at least I hope so).

Robert Kasanicky added a comment - 02/Oct/08 06:03 AM - edited
On seconds thoughts ItemReadListener won't help due to rollbacks and restart.

How about this: FlatFileItemReader and ResourceLineReader can expose a getter for current line number. ItemProcessor that wants to bundle the line number with the item would then be coupled to the reader (have a reference to the reader bean in Spring config).

Dave Syer added a comment - 02/Oct/08 07:12 AM
I thought we were going to have a FieldSetMetaData with our "best" guess at the line number (i.e. the last line read from the file). Cf BATCH-719.

Robert Kasanicky added a comment - 03/Oct/08 03:42 AM
This issue is now superceded by BATCH-863 that introduces a single interface for String-to-item mapping that includes the line number argument.