How to add name to rows when doing an append join #1335
Replies: 5 comments
-
I couldn't think of a clean visidata version that didn't involve writing a custom command in Python, but I got nerd-sniped, and had to script a Zsh version: #!/usr/bin/env zsh
output_file=combined
echo 'student,test,score' > "$output_file"
for file in *.csv; do
# zsh magic to remove the file extension
filename=($file(:r))
# sed magic to add the filename and a comma to the start of all lines except the first line.
sed -n -e "2,$ s/^/"$filename",/gp" "$file" >> "$output_file"
done
# if we use the csv extension from the start, it gets picked up by the for loop...
mv "$output_file" "$output_file".csv
|
Beta Was this translation helpful? Give feedback.
-
Thanks David! I should have mentioned I do something similar as a workaround in the shell, but wanted a simple way to do it within Visidata. I like your use of sed and zsh filename modifications. I'm using a favorite tool of aborruso: https://github.com/johnkerl/miller I use the miller put and cat verbs. |
Beta Was this translation helpful? Give feedback.
-
@frosencrantz you are a visidata master. But why do not use a visidata macro in which for each sheet, a column with the result of expression |
Beta Was this translation helpful? Give feedback.
-
Thanks, @aborruso I was trying to use the native Visidata without diving into the Python. You are right it would need something that can add a column to all the selected sheets in an IndexSheet of I don't know enough vd how to write a simple command or function that would add a column to selected sheets in an IndexSheet/SheetsSheet that would work within the context of each sheet, so it would be possible to add that P.S. David I hope you are good. I see your account is now listed as deleted. I hope we'll see you again. Thanks again for your suggestion. |
Beta Was this translation helpful? Give feedback.
-
Hi @frosencrantz the way I'm able to do it is this: Starting from a folder with my csv files and this vd
I run mkdir -p ./out
for i in *.csv;do name=$(basename "$i" .csv); vd -y -b -p fro.vd "$i" -o ./out/"$name".csv;done And I have in output all the csv files with the new sheet name column. And then I run
In output I have
|
Beta Was this translation helpful? Give feedback.
-
I encountered a new question with some data I'm working with. This is similar to my question in #1331 .
Now I have my data split across multiple CSV files, with one file per student, all the same column names. I want to load all the data files into Visidata, and then do an append join. But I want to make sure each lines reports the student name, in this case the file name is a good proxy for the student's name.
A.csv:
B.csv:
C.csv:
So I end up with the data labeled and combined, like below. What is a good way to have Visidata label the data with the filename so I see the source of each row.
Beta Was this translation helpful? Give feedback.
All reactions