spring-batch.adoc

사례

• Spring Batch 애플리케이션 성능 향상을 위한 주요 팁 / 제2회 Kakao Tech Meet

  • Writer에서 다건을 병렬처리, UPDATE 몰아서 하기, batchUpdate 활용

  • https://cheese10yun.github.io/spring-batch-update-performance/

• Batch Performance 극한으로 끌어올리기: 1억 건 데이터 처리를 위한 노력 / if(kakao)2022

  • Read

    • MySQL에서 limit, offset 지정방식의 페이징쿼리는 건수가 늘어갈 때 뒤로 갈수록 느려진다.

    • Cursor 조회로도 대량 데이터 조회 가능. 단 JpaCurSorItemReader는 사용하지 말것

  • Aggregation

    • MySQL에서 group by query 대신 Redis로 집계.command pipeline 을 활용하면 원격 호출 횟수를 최적화 할 수 있음.

  • Write

    • JDBC batchUpdate를 유도할 것

    • JPA는 성능최적화 관점에서 배치와 잘 맞지 않음

  • 배치 구동 환경

    • Spring Cloud Data Flow * K82를 용해서 batch 스케쥴링, 오케스트레이션, 모니터링을 하고 있음.

예제

• https://github.com/mminella/Pro-Spring-Batch-source-code

• https://github.com/mminella/Spring-Batch-Talk

• https://github.com/mminella/Spring-Batch-Talk-2.0

JavaConfig

http://blog.codecentric.de/en/2013/06/spring-batch-2-2-javaconfig-part-1-a-comparison-to-xml/

• Part 1 : A comparison to XML

• Part 2 : JobParameters, ExecutionContext and StepScope

• Part 3 : Profiles and environments</font>

• Part 4 : Job inheritance

• Part 5 : Modular configurations

• Part 6 : Partitioning and Multi-threaded Step

Transaction

• https://blog.codecentric.de/en/2012/03/transactions-in-spring-batch-part-1-the-basics/

• https://blog.codecentric.de/en/2012/03/transactions-in-spring-batch-part-2-restart-cursor-based-reading-and-listeners/

• https://blog.codecentric.de/en/2012/03/transactions-in-spring-batch-part-3-skip-and-retry/

Architecture

• https://blog.codecentric.de/en/2014/11/enterprise-java-batch-best-practice-architecture/

Monitoring

• https://blog.codecentric.de/en/2013/04/monitoring-spring-batch-with-appdynamics/

• https://trifork.nl/blog/spring-boot-observability-spring-batch-jobs/

JSR 352

• JSR-352 소개 : http://www.planetjones.co.uk/blog/25-05-2013/introducing-jsr-352-java-batch-ee-7.html

• Spring batch 3.0의 JSR-352 관련 스펙들 : https://jira.springsource.org/issues/?jql=labels%20%3D%20JSR-352

• 포럼에 관련 질문 : http://forum.springsource.org/showthread.php?138022-Spring-Batch-and-JSR352

• JSR-352와 Spring batch의 차이점 : http://blog.codecentric.de/en/2013/07/spring-batch-and-jsr-352-batch-applications-for-the-java-platform-differences/

• http://blog.springsource.org/2013/08/23/spring-batch-3-0-milestone-1-released/

• https://github.com/mminella/jsr352-springbatch-and-you

• https://blog.codecentric.de/en/2013/11/batch-processing-java-enterprise-edition-jsr-352-jee7-spring-batch/

핵심 commit

• retry 별도 프로젝트 분리 : https://github.com/SpringSource/spring-batch/commit/e827990f08c04122538d4ec17b0e90b8aa7ed577

• non-identifying job parameters https://github.com/spring-projects/spring-batch/commit/557515df45c0f596588418d53c3f2bae3781c1c3