Spring Boot + Spring Batch Job Parameters Example
Overview
In enterprise application batch job usually consists of multiple steps. During each step we may perform the following operations -- read input data
- process input data
- write the processed data
One of the features of Spring Batch is that is provides resusable components. These components can be used to process large valumes of data and perform operations like JobRestart, Skip, Job processing metadata, Logging and tracing and Transaction Management. What are Job Parameters Job parameters are key-value-pairs of data you provide when you start a job. A job has different job instances and these job instances differ on the basis of job parameters. JobParameters can be used for identification during the job run. They have reserved names, so to access them we can use Spring Expression Language
For example we can provide the following as job parameters-
- File path of the source location containing the files to be processed by String Batch
- The path of the target location where the processed files should be written by Spring Batch
Lets Begin
The maven project structure we will be developing is as follows -
The pom.xml will be as follows-
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.javainuse</groupId> <artifactId>spring-batch-jobparameter</artifactId> <version>0.0.1-SNAPSHOT</version> <packaging>jar</packaging> <name>jobParameters</name> <description>Demo project for Spring Boot</description> <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>2.1.1.RELEASE</version> <relativePath /> <!-- lookup parent from repository --> </parent> <properties> <java.version>1.8</java.version> </properties> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-batch</artifactId> </dependency> <dependency> <groupId>com.h2database</groupId> <artifactId>h2</artifactId> <scope>runtime</scope> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> </plugins> </build> </project>
Job Configuration
This is the job configuration class where we are creating the necessary beans to perform the batch job.- JobBuilderFactory - Convenient factory for a JobBuilder which sets the JobRepository automatically. We will be autowiring this bean provided by Spring Batch.
- StepBuilderFactory - Convenient factory for a StepBuilder which sets the JobRepository and PlatformTransactionManager automatically. We will be autowiring this bean provided by Spring Batch.
- Job - Batch domain object representing a job. A job execution may consist of multiple steps which need to be excuted. We are executing a single step named moveFilesStep. Also we start the job using the autowired JobBuilderFactory.
- Step - Batch domain interface representing the configuration of a step. In this step we call the moveFilesTasklet to execute the task of moving the files from source to destination directory.
- Tasklet - The Tasklet is a simple interface with just one method execute. Using this we can perform single tasks like executing queries, deleting files. In our case we will be moving the files from source to destination directory.
package com.javainuse.configuration;
import java.io.IOException;
import java.nio.file.DirectoryStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepScope;
import org.springframework.batch.core.step.tasklet.Tasklet;
import org.springframework.batch.repeat.RepeatStatus;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import static java.nio.file.StandardCopyOption.REPLACE_EXISTING;
@Configuration
public class JobConfiguration {
@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
@Bean
@StepScope
public Tasklet moveFilesTasklet(@Value("#{jobParameters['sourceDir']}") String sourceDirLocation,
@Value("#{jobParameters['destinationDir']}") String destinationDirLocation) {
return (stepContribution, chukContext) -> {
Path sourceDir = Paths.get(sourceDirLocation);
Path destinationDir = Paths.get(destinationDirLocation);
try (DirectoryStream<Path> directoryStream = Files.newDirectoryStream(sourceDir)) {
for (Path path : directoryStream) {
System.out.println("copying " + path.toString());
Path d2 = destinationDir.resolve(path.getFileName());
System.out.println("destination File=" + d2);
Files.move(path, d2, REPLACE_EXISTING);
}
} catch (IOException ex) {
ex.printStackTrace();
}
return RepeatStatus.FINISHED;
};
}
@Bean
public Step moveFilesStep() {
return stepBuilderFactory.get("moveFilesStep").tasklet(moveFilesTasklet(null, null)).build();
}
@Bean
public Job jobParametersJob() {
return jobBuilderFactory.get("jobParametersJob").start(moveFilesStep()).build();
}
}
MainApp
This is Spring Boot main project. Just run as a spring boot project- JobLauncher - Simple interface for controlling jobs, including possible ad-hoc executions, based on different runtime identifiers. It is extremely important to note that this interface makes absolutely no guarantees about whether or not calls to it are executed synchronously or asynchronously. The javadocs for specific implementations should be checked to ensure callers fully understand how the job will be run.
- Job - In a Spring Batch application, a job is the batch process that is to be executed. It runs from start to finish without interruption.
package com.javainuse;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@EnableBatchProcessing
@SpringBootApplication
public class JobParametersApplication implements CommandLineRunner {
@Autowired
private JobLauncher jobLauncher;
@Autowired
private Job job;
public static void main(String[] args) {
SpringApplication.run(JobParametersApplication.class, args);
}
@Override
public void run(String... args) throws Exception {
// Pass the required Job Parameters from here to read it anywhere within
// Spring Batch infrastructure
JobParameters jobParameters = new JobParametersBuilder().addString("sourceDir", "C://inputLocation")
.addString("destinationDir", "C://outputLocation").toJobParameters();
JobExecution execution = jobLauncher.run(job, jobParameters);
System.out.println("STATUS :: " + execution.getStatus());
}
}
Also by default, Spring Boot executes any job in out application context at startup.
So our tasklet will be executed twice: once at application startup and once when we invoke the API using the JobLauncher.
On application startup when the tasklet is involved the job parameters are null and so our application will throw a null
pointer exception.We will be disabling running jobs at startup using following property -
spring.batch.job.enabled=false
Console output
After running the program, you'll see that "message" value has been read in the Tasklet here.
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.1.1.RELEASE)
(2024)-01-05 00:12:18.377 INFO 7764 --- [ main] com.javainuse.JobParametersApplication : Starting JobParametersApplication on VAIO with PID 7764 (C:\codeusingjava-show\spring-batch-db\target\classes started by SONY in C:\codeusingjava-show\spring-batch-db)
(2024)-01-05 00:12:18.382 INFO 7764 --- [ main] com.javainuse.JobParametersApplication : No active profile set, falling back to default profiles: default
(2024)-01-05 00:12:19.960 INFO 7764 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
(2024)-01-05 00:12:20.397 INFO 7764 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
(2024)-01-05 00:12:20.416 INFO 7764 --- [ main] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: H2
(2024)-01-05 00:12:20.649 INFO 7764 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : No TaskExecutor has been set, defaulting to synchronous executor.
(2024)-01-05 00:12:21.005 INFO 7764 --- [ main] com.javainuse.JobParametersApplication : Started JobParametersApplication in 3.134 seconds (JVM running for 3.653)
(2024)-01-05 00:12:21.116 INFO 7764 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=jobParametersJob]] launched with the following parameters: [{sourceDir=C://inputLocation, destinationDir=C://outputLocation}]
(2024)-01-05 00:12:21.152 INFO 7764 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step1]
copying C:\inputLocation\hello.txt
destination File=C:\outputLocation\hello.txt
copying C:\inputLocation\hello2.txt
destination File=C:\outputLocation\hello2.txt
copying C:\inputLocation\hello3.txt
destination File=C:\outputLocation\hello3.txt
copying C:\inputLocation\hello4.txt
destination File=C:\outputLocation\hello4.txt
copying C:\inputLocation\hello5.txt
destination File=C:\outputLocation\hello5.txt
copying C:\inputLocation\hello6.txt
destination File=C:\outputLocation\hello6.txt
(2024)-01-05 00:12:21.260 INFO 7764 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=jobParametersJob]] completed with the following parameters: [{sourceDir=C://inputLocation, destinationDir=C://outputLocation}] and the following status: [COMPLETED]
STATUS :: COMPLETED
(2024)-01-05 00:12:21.285 INFO 7764 --- [ Thread-2] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
(2024)-01-05 00:12:21.288 INFO 7764 --- [ Thread-2] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.