We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
I'm trying to use header transformations when importing data from a CSV file. When I look at the values using byebug I get a value like this
byebug
(byebug) array_of_hashes[0] {:countycode=>37, :firstname=>"FIRSTNAME", :lastname=>"LASTNAME", :addressline1=>"739 TWIN OAKS AVE", :city=>"CHULA VISTA", :zip=>91910, :email=>"REDACTED", :dob=>"REDACTED", :precinct=>"CHULA VISTA", :precinctnumber=>528500.0, :precinctid=>19099}
but I am expecting something like
{:county_code=>37, :firstname=>"FIRSTNAME", :last_name=>"LASTNAME", :residential_address1=>"739 TWIN OAKS AVE", :residential_city=>"CHULA VISTA", :residential_zip=>91910, :email=>"REDACTED", :date_of_birth=>"REDACTED", :precinct=>"CHULA VISTA", :precinct_number=>528500.0, :precinct_id=>19099}
task :seed, [:batch_id] => :environment do |_t, _args| options = { chunk_size: 1000, header_transformations: [ key_mapping: key_mapping ] } Dir.entries(dir_path = Rails.root.join('data_files/records')).each do |file_name| next unless file_name.end_with?('.csv') chunks = SmarterCSV.process("#{dir_path}/#{file_name}", options) progress = ProgressBar.create(title: "#{dir_path}/#{file_name}", total: chunks.size) Parallel.map(chunks, in_threads: 10) do |chunk| worker(chunk) progress.increment end File.rename("#{dir_path}/#{file_name}", "#{dir_path}/#{file_name}.done") end end def key_mapping { firstname: :first_name, lastname: :last_name, middlename: :middle_name, precint: :precinct_name, precinctnumber: :precinct_number, precintid: :precinct_id, dob: :date_of_birth, email: :email, addressline1: :residential_address1, addressline2: :residential_secondary_addr, city: :residential_city, zip: :residential_zip } end def worker(array_of_hashes) byebug RecordImportWorker.perform_async array_of_hashes, @model[:id] AddressImportWorker.perform_async array_of_hashes, @model[:id] end
CountyCode,FirstName,MiddleName,LastName,AddressLine1,AddressLine2,City,Zip,Email,DOB,Precinct,PrecinctNumber,PrecinctId
The text was updated successfully, but these errors were encountered:
Had a similar issue, turns out header_transformations was added in 2.x which has not been released yet, so you are probably using the 1.2.
header_transformations
You can find 1.2 docs here (which have a key_mapping option, just without header_transformation): https://github.com/tilo/smarter_csv/tree/1.2-stable#deprecated-1x-options-to-be-replaced-in-20
key_mapping
header_transformation
Or if you want to use 2.0 (unstable): #153 (comment)
Sorry, something went wrong.
No branches or pull requests
I'm trying to use header transformations when importing data from a CSV file. When I look at the values using
byebug
I get a value like thisbut I am expecting something like
import task
Headers in file
The text was updated successfully, but these errors were encountered: