4454 - Added deletion to bulk import

[Rob9786]

Make sure you have checked all steps below.

Issue

• My PR addresses the following issues and references them in the PR title. For example, "Issue 1234 - My Sleeper
  PR"
  • Resolves Bulk import jobs are never deleted from S3 #4454

Tests

• My PR adds the following tests OR does not need testing for this extremely good reason:
  • shouldDeleteJsonFileAfterImport

Documentation

• In case of new functionality, my PR adds documentation that describes how to use it, or I have linked to a
  separate issue for that below.
• If I have added or removed any dependencies from the project, I have updated the NOTICES file.

[Rob9786]

Hi Tristan,

When the IT test runs, it triggers three import drivers, BulkImportJobDataframeDriver, BulkImportJobRDDDriver and BulkImportDataframeLocalSortDriver.

Each of these have a main method that call BulkImportDriver.start passing in args.

BulkImportDriver.start then extracts the configBucket from the args and loads the InstanceProperties from that bucket. The InstanceProperties contain the location of the BULK_IMPORT_BUCKET, which is used in loadJob() to create the json file that needs to then be deleted.

In the Delete test I need to check the contents of BULK_IMPORT_BUCKET to make sure the json file has been deleted. I'm trying to find out what the value of BULK_IMPORT_BUCKET needs to be.