How to reference columns with spaces in '=.....' expressions

[halloleo]

I have many CSVs with column names with spaces. In these sheets how can I use '=....' expressions to generate new columns?

Let's say, I have a column name Final Aperture which I want to use in a new column. I tried:

• =Final Aperture
• =Final_Aperture
• ='Final Aperture'
• =f'{Final Aperture}'

In neither case I got a column with copy of Final Aperture's content.

I read #142 and tried options.force_valid_colnames = True in my .visidatarc, but =Fin still completes to =Final Aperture which gives me a syntax error.

What can I do? I'm running out of ideas.

Many thanks for pointers!

[saulpw]

From the CHANGELOG (for 2.0):

options.force_valid_colnames renamed to options.clean_names

We should be able to add option aliases in deprecated.py, but it hasn't been implemented yet. We don't usually change option names but did this for the big 2.0 so that it could reasonably apply to sheet names as well.

Also remember in a pinch that you can just rename the column. That's all this option does anyway.

[halloleo]

Thanks heaps for the clarification with the option. :-)

So, columns need to be in Python identifier syntax to be used in =..., right?

Sorry to ask, but what's the easiest way then to rename all columns to identifiers? And how do I revert them back at the end when I write the file back to disk?

[saulpw]

Well, it's not very straightforward, but I'm thinking that you could do this:

1. Shift+C to open the Columns Sheet
2. gs to select all columns
3. g*[^A-z0-9_]+/_ on the name (first) column to replace any number of non-identifier chars with a single _
4. switch over to the previous sheet without quitting (use Ctrl+^ or from the Sheets Sheet)
5. ...do whatever you want with these column names...
6. then when done, go back to the Columns Sheet and press Shift+U to undo the original name changes.
7. now you can q back to the original sheet and save.

Note that if you used the clean names in any expression columns you want to save, you'll have to make a snapshot column ("freeze") with ' (single-quote) before the undo.

[halloleo]

Wow, I love the trick with switching to the base sheet and then later going back to the column sheet. Thanks, that's super cool! (But indeed not very straightforward...)

[halloleo]

One other thing about the clean_names = True option: Is it possible to set this option on the command line? I don't want to set it in my rc file for all CSVs...

[frosencrantz]

There is also @jsvine plugin: https://github.com/jsvine/visidata-plugins#normcol

[saulpw]

@halloleo Yes, you can --clean-names=True (and variants should also work).

[halloleo]

@frosencrantz Thanks for the tip. The plugin looks great. - Can I use it with Saul's trick to revert columns back at the end when I need to write the file back to disk?

[frosencrantz]

@halloleo , normcol supports undo, only on the sheet: https://github.com/jsvine/visidata-plugins/blob/stable/plugins/normcol.py#L119

So it doesn't look like you can use Saul's trick.

If it also had a binding on the ColumnSheet, I suppose it could support that trick. It sounds like a useful feature.