Unexpected rollbackCount and calls of shouldSkip() during item write

Spring documentation (Pg. 46, Section: 5.1.7) says:

By default, regardless of retry or skip, any exceptions thrown from the ItemWriter will cause the transaction controlled by the Step to rollback. If skip is configured as described above, exceptions thrown from the ItemReader will not cause a rollback.

My commit interval is set to 10. So my understanding of above paragraph is, if their is error in reading 7th record out of the chunk of 10, the item will be skipped and the correct 9 records will be sent ahead by itemReader.

However, if the 7th record is in error during writing - none of the 10 records will be w开发者_开发知识库ritten and a rollback will happen.

However, when I am including the error thrown in my skipPolicy, itemWriter IS writing the remaining 9 records to the database skipping the errored one. This is contradictory to what is mentioned above.

Can any one please explain the concept of "skip during item writing".

Also even though single error is thrown I am getting the following:

SkipCount as -1 twice, then as 0 once, and again -1 once in my shouldSkip(Object, Throwable) method. -- I am not getting this behavior.

Also rollback count is 2 -- what does it mean ? why it is 2 ?

@michael Would it be possible for you to explain the behavior using some scenario!!

like "i am reading 20 records from a file and writing to a database after some processing. I have a skip policy set for some exception. what will happen if the exception occurrs during - read, process, write -- how the chunks will be committed, how default retry works, how the counts will be updated, etc. etc...."

It will really be a big help for me, as I am still confused with the behavior..

From your usecase description it seems, that you mix different concepts.

You describe a skip scenario but you seem to expect skip should work like a no-rollback scenario.

from the spring batch documentation

skip:

errors encountered while processing should not result in Step failure, but should be skipped instead

vs no-rollback:

If skip is configured as described above, exceptions thrown from the ItemReader will not cause a rollback.

in my own words skip means:

If the step encounters an error during read/process/write, the current chunk will be rollbacked and each item of the chunk is read/processed/written individually - without the bad item. Basically Spring Batch falls back to commit-rate 1 for the bad chunk and goes back to the specified commit-rate after the bad chunk.

Also rollback count is 2 -- what does it mean ? why it is 2 ?

from B.5. BATCH_STEP_EXECUTION

ROLLBACK_COUNT: The number of rollbacks during this execution. Note that this count includes each time rollback occurs, including rollbacks for retry and those in the skip recovery procedure.

(emphasize mine)

Also even though single error is thrown I am getting the following:

SkipCount as -1 twice, then as 0 once, and again -1 once in my shouldSkip(Object, Throwable) method. -- I am not getting this behavior.

i tried a simple skip job with both configuration styles, skip-policy and skip-limit with skippable-exception, both worked identically in relation to rollback and skip counts

(step metadata is allright but shouldSkip(...) seems to be called a lot more than expected)

I'd like to explain one issue you mentioned:

SkipCount as -1 twice, then as 0 once, and again -1 once in my shouldSkip(Object, Throwable) method. -- I am not getting this behavior.

I don't know to which signature of the shouldSkip() method you refer to, but in my SkipPolicy interface there is only one method with the following signature:

boolean shouldSkip(Throwable t, int skipCount) throws SkipLimitExceededException;

This method decides whether the Exception e given the skipCount should be skipped or not.

Unfortunately, the programmers of Spring Batch misuse this method to test, whether an exception is skippable in general regardless of the current skip count. That's why there are several calls to this method with the skipCount parameter set to -1.

So just don't wonder about the behaviour you saw.