1. Overview
Spring Batch is a powerful framework for developing robust batch applications. In our previous tutorial, we introduced Spring Batch.
In this tutorial, we'll build on the previous one and learn how to set up and create a basic batch-driven application using Spring Boot.
2. Maven Dependencies
In one item writer class as list then i push in to DB, for incorrect data 5 records i need another item writer to push in to another csv file.this is my requirement.please share ur ideas how to doif my idea is wrong please suggest correct way of implementing my requirement. Spring Batch provides readers and writers to read and write data form various file systems/databases such as MongoDB, Neo4j, MySQL, XML, flatfile, CSV, etc. To include a reader in your application, you need to define a bean for that reader, provide values to all the required properties within the bean, and pass the id of such bean as a value to. This is a basic idea of batch processing, And in this post we are going to take a look at the spring batch processing. Spring Batch Processing. S p ring batch is an opensource batch processing.
First, let’s add the spring-boot-starter-batch to our pom.xml:
We'll also add the org.hsqldb dependency, which is available from Maven Central as well:
3. Defining a Simple Spring Batch Job
We're going to build a job that imports a coffee list from a CSV file, transforms it using a custom processor, and stores the final results in an in-memory database.
3.1. Getting Started
Let's start by defining our application entry point:
As we can see, this is a standard Spring Boot application. As we want to use default configuration values where possible, we're going to use a very light set of application configuration properties.
We'll define these properties in our src/main/resources/application.properties file:
This property contains the location of our input coffee list. Each line contains the brand, origin, and some characteristics of our coffee:
As we're going to see, this is a flat CSV file, which means Spring can handle it without any special customization.
Next, we'll add a SQL script schema-all.sql to create our coffee table to store the data:
Conveniently Spring Boot will run this script automatically during startup.
3.2. Coffee Domain Class
Subsequently, we'll need a simple domain class to hold our coffee items:
As previously mentioned, our Coffee object contains three properties:
- A brand
- An origin
- Some additional characteristics
4. Job Configuration
Now, on to the key component, our job configuration. We'll go step by step, building up our configuration and explaining each part along the way:
Firstly, we start with a standard Spring @Configuration class. Next, we add a @EnableBatchProcessing annotation to our class. Notably, this gives us access to many useful beans that support jobs and will save us a lot of leg work.
Furthermore, using this annotation also provides us with access to two useful factories that we'll use later when building our job configuration and jobs steps.
For the last part of our initial configuration, we include a reference to the file.input property we declared previously.
4.1. A Reader and Writer for Our Job
Now, we can go ahead and define a reader bean in our configuration:
In short, our reader bean defined above looks for a file called coffee-list.csv and parses each line item into a Coffee object.
Likewise, we define a writer bean:
This time around, we include the SQL statement needed to insert a single coffee item into our database, driven by the Java bean properties of our Coffee object. Handily the dataSource is automatically created by @EnableBatchProcessing annotation.
4.2. Putting Our Job Together
Lastly, we need to add the actual job steps and configuration:
As we can see, our job is relatively simple and consists of one step defined in the step1 method.
Let's take a look at what this step is doing:
- First, we configure our step so that it will write up to ten records at a time using the chunk(10) declaration
- Then, we read in the coffee data using our reader bean, which we set using the reader method
- Next, we pass each of our coffee items to a custom processor where we apply some custom business logic
- Finally, we write each coffee item to the database using the writer we saw previously
On the other hand, our importUserJob contains our job definition, which contains an id using the build-in RunIdIncrementer class. We also set a JobCompletionNotificationListener, which we use to get notified when the job completes.
To complete our job configuration, we list each step (though this job has only one step). We now have a perfectly configured job!
5. A Custom Coffee Processor
Let's take a look in detail at the custom processor we defined previously in our job configuration:
Of particular interest, the ItemProcessor interface provides us with a mechanism to apply some specific business logic during our job execution.
To keep things simple, we define our CoffeeItemProcessor, which takes an input Coffee object and transforms each of the properties to uppercase.
6. Job Completion
Additionally, we're also going to write a JobCompletionNotificationListener to provide some feedback when our job finishes:
In the above example, we override the afterJob method and check the job completed successfully. Moreover, we run a trivial query to check that each coffee item was stored in the database successfully.
7. Running Our Job
Now that we have everything in place to run our job, here comes the fun part. Let's go ahead and run our job:
Spring Batch Flat File Item Writer Example Free
As we can see, our job ran successfully, and each coffee item was stored in the database as expected.
8. Conclusion
In this article, we've learned how to create a simple Spring Batch job using Spring Boot. First, we started by defining some basic configuration.
Spring Batch Flat File Item Writer Example Pdf
Then, we saw how to add a file reader and database writer. Finally, we took a look at how to apply some custom processing and check our job was executed successfully.
As always, the full source code of the article is available over on GitHub.