Closed
Description
I'm trying to use header transformations when importing data from a CSV file. When I look at the values using byebug
I get a value like this
(byebug) array_of_hashes[0]
{:countycode=>37, :firstname=>"FIRSTNAME", :lastname=>"LASTNAME", :addressline1=>"739 TWIN OAKS AVE", :city=>"CHULA VISTA", :zip=>91910, :email=>"REDACTED", :dob=>"REDACTED", :precinct=>"CHULA VISTA", :precinctnumber=>528500.0, :precinctid=>19099}
but I am expecting something like
{:county_code=>37, :firstname=>"FIRSTNAME", :last_name=>"LASTNAME", :residential_address1=>"739 TWIN OAKS AVE", :residential_city=>"CHULA VISTA", :residential_zip=>91910, :email=>"REDACTED", :date_of_birth=>"REDACTED", :precinct=>"CHULA VISTA", :precinct_number=>528500.0, :precinct_id=>19099}
import task
task :seed, [:batch_id] => :environment do |_t, _args|
options = {
chunk_size: 1000,
header_transformations: [
key_mapping: key_mapping
]
}
Dir.entries(dir_path = Rails.root.join('data_files/records')).each do |file_name|
next unless file_name.end_with?('.csv')
chunks = SmarterCSV.process("#{dir_path}/#{file_name}", options)
progress = ProgressBar.create(title: "#{dir_path}/#{file_name}", total: chunks.size)
Parallel.map(chunks, in_threads: 10) do |chunk|
worker(chunk)
progress.increment
end
File.rename("#{dir_path}/#{file_name}", "#{dir_path}/#{file_name}.done")
end
end
def key_mapping
{
firstname: :first_name,
lastname: :last_name,
middlename: :middle_name,
precint: :precinct_name,
precinctnumber: :precinct_number,
precintid: :precinct_id,
dob: :date_of_birth,
email: :email,
addressline1: :residential_address1,
addressline2: :residential_secondary_addr,
city: :residential_city,
zip: :residential_zip
}
end
def worker(array_of_hashes)
byebug
RecordImportWorker.perform_async array_of_hashes, @model[:id]
AddressImportWorker.perform_async array_of_hashes, @model[:id]
end
Headers in file
CountyCode,FirstName,MiddleName,LastName,AddressLine1,AddressLine2,City,Zip,Email,DOB,Precinct,PrecinctNumber,PrecinctId
Metadata
Metadata
Assignees
Labels
No labels