Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimization - add flexible column inputs to SQL raw data loading scrips. Ideally read column names from file, sql copy the file then drop any columns not used in processing #48

Open
drotheram opened this issue Jan 28, 2021 · 0 comments
Assignees
Labels

Comments

@drotheram
Copy link
Contributor

Sometimes small changes in raw files such as the addition of an index column break the workflow and create maintenance overhead. The stack would be more flexible and robust if it dynamically reads file headers rather than assuming the inputs conform to a rigid standard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants