[#BATCH-341] Parameter generation strategy

Spring Batch

Parameter generation strategy

Created: 05/Feb/08 11:08 PM Updated: 29/Aug/08 10:21 AM

Component/s:

Core

Affects Version/s:

None

Fix Version/s:

2.0.0.M3

Time Tracking:

Original Estimate:

Remaining Estimate:

Time Spent:

Not Specified

Environment:

N/A

Issue Links:

Related

This issue is related to:

~~BATCH-597~~ Retrieving Job Parameters for a given job name

Description

« Hide

Would it be possible to come up with a parent table abstraction for each of the following Spring Batch tables: BATCH_JOB_INSTANCE, BATCH_JOB_INSTANCE_PARAMS, and BATCH_STEP_INSTANCE?

The idea here is to essentially maintain a master list of jobs, steps and job parameters that have no association to a particular run / instance. These tables together would for an enterprise provide visibility into the jobs that run under the Spring Batch umbrella, while providing insight into the steps(complexity of steps) involved and the nature of arguments needed to run a particular job. Example: Numerous steps associated with a job could signal a complex and perhaps long running job.

Please see below for definitions of the parent table abstractions:

BATCH_JOB:
A master table that has a list of jobs and job related attributes. This is the parent table for BATCH_JOB_INSTANCE table.

BATCH_STEP:
A master table that has various steps associated with a particular job. This references the job_id of BATCH_JOB table. This would be a child table of BATCH_JOB but a parent table for BATCH_STEP_INSTANCE table.

BATCH_JOB_PARAMS:
A master table that has a list of argument names and values.

All

Comments

Work Log

Change History

FishEye

Related Builds

Sort Order:

[ Permlink | « Hide ]

Gregory Kick added a comment - 06/Feb/08 09:45 AM

can you provide some examples of what sort of data might be in each of these tables?

[ Permlink | « Hide ]

Dave Syer added a comment - 07/Feb/08 12:32 PM

Couldn't they implemented as simple SQL views? In any case I would not be keen to add new DDL that wasn't needed by any of our Java repositories.

[ Permlink | « Hide ]

Wayne Lund added a comment - 08/Feb/08 12:27 AM

This issue of where the meta data for jobs should be defined has come up before. We've seen some projects that use a visio template for visualizing the job streams and the relationships with each other. Batch meta data can also include volumetrics for a job (e.g. planned frequency of runs, expected volumes, dependencies, protocol to be used for transfers like ftp, error management policies, etc.) One project raised this issue and said they kept their meta information in a very similar schema discussed by Jayanth in the database schema. We've seen others that largely kept this information in a combination of spreadsheets and an enterprise scheduler.

I think there's room for discussing the meta-information on batch jobs but I disagree with the term parent tables. I think a more accurate way of viewing these tables is that they are more like meta-class tables where the "Job Class" behavior can be stored for a given Job. This would also be a nice structure to have for organizing reports on time intervals for SLA reporting. I'd be interested in seeing how many projects see this as an important feature because we've purposely shied away from it because of the various ways projects track meta-data in their solutions.

[ Permlink | « Hide ]

Gregory Kick added a comment - 08/Feb/08 07:06 AM

+1 Wayne's comment.

[ Permlink | « Hide ]

Ben Hale added a comment - 20/Mar/08 04:40 AM

This close to a 1.0.0 I don't think we can make a change this significantly. For 2.0.0 we can though.