Skip to content

Latest commit

 

History

History
431 lines (293 loc) · 28.3 KB

CHANGELOG.md

File metadata and controls

431 lines (293 loc) · 28.3 KB
Raw

Please refer to GitHub issues for changelog starting from v2.3: https://github.com/kalisio/krawler/issues.

Changelog

v2.2.0 (2023-01-09)

Full Changelog

Implemented enhancements:

  • Allow pipeline to be templated in createMongoAggregation hook #235
  • Create a hook to run a task #234
  • Support post method for http tasks #232
  • Improve basic auth hook to support storing a cookie #231
  • Improve basic auth hook to support sending form data #230
  • Improve basic auth hook to support token request #229

Fixed bugs:

  • Tests relying on Meteo France API do not work anymore #233

Merged pull requests:

v2.1.0 (2022-10-24)

Full Changelog

Implemented enhancements:

  • Add a hook to specify outputs of a task #218
  • Feathers hooks should be able to use distributed services #217
  • Hook to call a Feathers API should allow to transform data before sending #213
  • Add params option to Feathers service method hook #212

Fixed bugs:

  • Krawler API mode does not work anymore #215
  • Job without tasks exit as failing #214

Merged pull requests:

v2.0.0 (2022-08-30)

Full Changelog

Implemented enhancements:

  • Allow to read TXT files #209
  • Add hooks to use a Feathers API #207
  • Allow to read KML files #204
  • Tasks order in job result array #203
  • Ensure the store path exists when creating a store of type of FS #202
  • Provide a envsubst hook #201
  • Provide a globFTP hook #200
  • Allow to specify Xml2js parsing options #199
  • Refactor FTP hooks using lftp #198
  • Reflect healthcheck status in exit code #196
  • Add a hook to check for existing key in store #179
  • Upgrade NodeJS to v16 #177
  • runCommand hook should allow to pipe stdout/stderr #154
  • Async job should manage tasks in a queue #151
  • Make callOnHookItems more configurable #150
  • Add a progress bar during job execution #98
  • Upgrade Feathers to v5 #82
  • Integrate a logger like winston #37
  • Make krawler able to react to external systems #34

Fixed bugs:

  • writeCSV hook produces empty files #169

Merged pull requests:

v1.2.0 (2020-12-22)

Full Changelog

Implemented enhancements:

  • Add an option to avoid transforming Json objects in place #144
  • Allow NWP hook to generate tasks for past runs #140
  • Healthcheck should not fail when given unknown options #139
  • Improve healthcheck messages posted on Slack #137
  • Enhance mergeJson hook to deep merge objects #132
  • Match predicate not applied when items are an array #130

Fixed bugs:

  • GeoJson features not transformed on write/updateMongoCollection hooks #145
  • Upper limit exceeded in NWP tasks with a oldest run interval #138
  • Boolean values in match filter are lost after templating #136
  • When a job timeout, krawler hangs with the following error: Cannot read property 'type' of undefined #135
  • Permission denied to run krawler command in container #134

Merged pull requests:

v1.1.1 (2020-05-13)

Full Changelog

Implemented enhancements:

  • Upgrade to latest Node.js LTS #91

Fixed bugs:

  • Build not working anymore #115
  • Tile size is wrongly computed in grid #114

v1.1.0 (2020-04-28)

Full Changelog

Implemented enhancements:

  • User-defined functions could be async #113
  • Add an option to keep past forecast data in NWP hook #106
  • Alllow to apply a tranform in the CSV hooks #105

Fixed bugs:

  • transformJson hook mapping on undefined properties #112
  • A job with an error hook will make krawler exit with 'tasks.forEach is not a function' when a task throws an error #110
  • Healthcheck fails when behind a proxy #109
  • FTP connection fails when behind a proxy #108
  • Healthcheck fail if the healthcheck endpoint does not respond #107
  • templateQueryObject cast a property of type of number to a date #104

v1.0.1 (2020-01-02)

Full Changelog

Fixed bugs:

  • Dates contained in MongoDB queries are lost #103

v1.0.0 (2019-12-11)

Full Changelog

Implemented enhancements:

  • Convert number type on templated queries #102
  • Create an update Mongo DB collection hook #101
  • Allow to provide bulkWrite options in writeMongoCollection hook #100
  • Simplify CI scripts #97
  • Improve runCommand hook to manage multiple commands #96
  • Enhance travis pipeline #94
  • Allow to create a MongoDB Aggregation #93
  • Enable Greenkeeper to ease dependency upgrades #83

Fixed bugs:

  • faultTolerant hook raises an exception and stops the job #99
  • Node hangs when a job with a timeout is finished #92
  • getFTP hook may incorrectly register the output file #87

v0.9.2 (2019-09-22)

Full Changelog

Implemented enhancements:

  • Update dockerfile base image #90
  • Update dependencies #81

Fixed bugs:

  • User/Password not defined in authentication hook when stored as environment variable in jobfile #86
  • Docker tests do not work anymore on Travis #85
  • Generated grid is not a square depending on latitude #84
  • WCS query with resample options is invalid with MapServer #80

v0.9.1 (2019-06-19)

Full Changelog

Implemented enhancements:

  • Improve healthcheck with job run time #76
  • Healthcheck script should manage alert opening/closing #73

Fixed bugs:

  • NWP tests fail on GFS #79
  • Fault-tolerant hooks do not catch errors #78
  • Unexpected error "Cannot read property 'type' of undefined" #77
  • Error messages should be redirected to stderr #75
  • Healthcheck script fails if an error is raised by a hook #74
  • Healthcheck script slack notification does not have hyperlink #72
  • Healthcheck script failure in unknown circumstances #71

v0.9.0 (2019-05-15)

Full Changelog

Implemented enhancements:

  • Add healthcheck script #70
  • Improve healthcheck and error management for fault-tolerant jobs #69
  • Update MongoDB driver version to enable connection to MongoDB Atlas #66
  • The krawler container should ship GDAL #63
  • Allow to list an FTP remoteDir #61
  • Enhance MongoDB vocabulary #31

Fixed bugs:

  • Incorrect value of MongoDb database name #65
  • disconnectFTP hook should execute the quit command #60

v0.8.1 (2019-02-19)

Full Changelog

Implemented enhancements:

  • Allow to convert numbers to string on JSON transformation #59
  • Add a hook that could transform OSM data to GeoJSON #56
  • Add a task which can handle an Overpass API query #55

Fixed bugs:

  • The default url to perform overpass task seems to be deprecated #58
  • runCommand hook might prevent job to finish #38

v0.8.0 (2019-01-15)

Full Changelog

Implemented enhancements:

  • Add ability to unzip files #50
  • Add a healthcheck entry point #49
  • Add some capabilities to handle FTP #48
  • Add a WFS task type #46
  • Stop job after a given number of failed tasks #44
  • Allow the template function to tackle properties of type of array of objects #42
  • Allow to create swarm service #41
  • Creating multiple single indices does not work #40
  • Create a apply hook to run a custom function on items #36
  • Add a retry capability for tasks in async job #35
  • CRON jobs are launched immediately #32
  • Migrate to Feathers V3 #28
  • Enhance docker hooks #27
  • Make krawler able to communicate with external systems #24
  • Refactor using only hooks #4

Fixed bugs:

  • Unzip hook returns before file has been written #52
  • Unauthorize API requests in non-API mode #51
  • Job hangs after mongo connection failure #47
  • readJson hook fails silently on perse error #45
  • Concurrent tasks skipped when a task fails #33
  • Possible memory leak #30
  • Mongo client not correctly cleared #29

Merged pull requests:

  • [Snyk] Fix for 1 vulnerable dependencies #17 (snyk-bot)

v0.7.0 (2018-07-17)

Full Changelog

Fixed bugs:

  • MongoDB job never completes #26

v0.6.0 (2018-07-10)

Full Changelog

Implemented enhancements:

  • Added numerical weather prediction models management #22

v0.5.2 (2018-06-28)

Full Changelog

Fixed bugs:

  • krawler command not available #21

v0.5.1 (2018-06-27)

Full Changelog

Fixed bugs:

  • CLI mode broken since v0.5 #20

v0.5.0 (2018-06-27)

Full Changelog

Implemented enhancements:

  • Tag which hooks to be run in case of errors #19
  • Generate web app from job description #18
  • Support MongoDB like PG #16

v0.4.0 (2018-06-04)

Full Changelog

Implemented enhancements:

  • Fault-tolerant jobs/tasks #15

v0.3.1 (2018-05-31)

Full Changelog

Implemented enhancements:

  • Date/time format conversion #14

v0.3.0 (2018-05-23)

Full Changelog

Implemented enhancements:

  • Support CRON jobs #12

v0.2.3 (2018-05-07)

Full Changelog

Fixed bugs:

  • Recurring jobs stop on error #11

Merged pull requests:

v0.2.2 (2018-05-02)

Full Changelog

v0.2.1 (2018-05-02)

Full Changelog

v0.2.0 (2018-05-02)

Full Changelog

Implemented enhancements:

  • Improve transformJson #9
  • Add a filter to select which tasks are processed by a given hook #8
  • Add a in-memory store #6

Fixed bugs:

  • Subsequent tasks overwrite first produced file when writing to S3 #7
  • HTTP error codes not correctly handled #2

* This Changelog was automatically generated by github_changelog_generator