Merge pull request #1551 from saulpw/develop

v2.10.2

CHANGELOG.md

@@ -1,5 +1,43 @@
 # VisiData version history
 
+# v2.10.2 (2022-10-XX)
+
+- add .vdx, a simplified new cmdlog format
+- add `-N`/`--nothing` command to disable loading .visidatarc and plugin addons
+- add `addcol-aggr` to add an aggregator column to the **FreqTable** without needing to
+  regenerate it (requested by @geekscrapy #1541)
+
+## Improvements
+
+- [cli] load commandline file arguments from the start (requested by @reagle #1471)
+- [cli] `--config=''` now does not try to load any config
+- [open] rename `zo` `open-cell` command to `open-cell-file`
+- [loaders whl] load python .whl (reported by @frosencrantz #1539)
+
+## Bugfixes
+
+- [cli] fix for empty arg
+- [DirSheet] fix bug where `Enter` no longer opened a file from the **DirSheet** (reported by @frosencrantz #1527)
+- [input paste] fix pasting via a Path via `Ctrl+Y`  into input (reported by @frosencrantz #1546)
+- [menu] allow VisiData to run without menu
+- [mouse] catch any curses.getmouse() errors (reported by @geekscrapy #1553)
+- [performance] allow vd to be truly idle (reported by WizzardUU #1532)
+- [plugins_autoload] catch error for environment having invalid package metadata (reported by @jsdealy #1529)
+- [plugins_autoload] catch exception if plugin fails to load
+- [plugins-autoload] fix check for if plugins_autoload is set in args
+- [plugins-autoload] update for importlib-metadata 5.0 API (reported by @jkerhin #1550)
+- [pyobj] undo rename of `open-row`/`open-cell` (were renamed to `open-X-pyobj`) (revert of eff9833e6A)
+- [sheets] ensure IndexSheets are precious, and that **SheetsSheet** is not (reported by @frosencrantz #1547)
+- [unzip-http] extracting a file now checks for overwrite (reported by @frosencrantz #1452)
+- [windows clipboard] fix piping to clip command through stdin (thanks @daviewales for the fix; reported by @pshangov #1431)
+
+## API
+
+- expose `CommandLogBase` (was `_CommandLog`)
+- [options] allow FooSheet.options instead of .class_options
+- add seperate non-async `select_row`, `toggle_row`, and `unselect_row` for selection of single rows
+- the before/after decorators now do not fail if api functions they are decorating do not already exist
+
 # v2.10.1 (2022-09-14)
 
 ## Improvements

README.md

@@ -1,5 +1,5 @@
 
-# VisiData v2.10.1 [![twitter @VisiData][1.1]][1] [![CircleCI](https://circleci.com/gh/saulpw/visidata/tree/stable.svg?style=svg)](https://circleci.com/gh/saulpw/visidata/tree/stable) [![Gitpod ready-to-code](https://img.shields.io/badge/Gitpod-ready--to--code-blue?logo=gitpod)](https://gitpod.io/#https://github.com/saulpw/visidata)
+# VisiData v2.10.2 [![twitter @VisiData][1.1]][1] [![CircleCI](https://circleci.com/gh/saulpw/visidata/tree/stable.svg?style=svg)](https://circleci.com/gh/saulpw/visidata/tree/stable) [![Gitpod ready-to-code](https://img.shields.io/badge/Gitpod-ready--to--code-blue?logo=gitpod)](https://gitpod.io/#https://github.com/saulpw/visidata)
 
 A terminal interface for exploring and arranging tabular data.
 

dev/checklists/release.md

@@ -30,14 +30,25 @@
 
    b. update the date in the manpage;
 
-   c. update version number on README and front page of website (update dates here too);
+   c. update version number on README
 
    d. bump version in `__version__` in source code (visidata/main.py, visidata/__init__.py) and setup.py;
 
 6. Run dev/mkman.sh to build the manpage and updated website
     - Run ./mkmanhtml.sh, and move that to visidata.org:site/docs/man, and to visidata:docs/man.md
 
-7. Push `develop` to testpypi
+7. Merge `develop` to stable
+
+8. Merge `stable` back into other branches
+
+    a. if the branch works with minimal conflicts, keep the branch
+
+    b. otherwise, clean out the branch
+
+
+9. Push code to stable
+
+10. Push `develop` to pypi
 
     a. set up a ~/.pypirc
 
@@ -57,69 +68,38 @@
     password:
     ```
 
-    b. push to testpypi
 
+  Push to pypi
     ```
     python3 setup.py sdist bdist_wheel --universal
-    twine upload dist/* -r testpypi
+    twine upload dist/*
     ```
 
-8. Test install from testpypi
-
-   a. on virgin instance
-
-   b. on mac and linux
-
-   c. See if windows works
-
-   d. from git clone
-
-   ```
-   pip3 install -i https://test.pypi.org/simple/ visidata
-   ```
-
-9. Merge `develop` to stable
-
-10. Merge `stable` back into other branches
-
-    a. if the branch works with minimal conflicts, keep the branch
-
-    b. otherwise, clean out the branch
-
-
-11. Push code to stable
-
-12. Push stable to pypi
-
-```
-twine upload dist/*
-```
-
-13. Test install/upgrade from pypi
+11. Test install/upgrade from pypi
 
   a. Build and deploy the website
 
    b. Ask someone else to test install
 
-14. Create a tag `v#.#.#` for that commit
+12. Create a tag `v#.#.#` for that commit
 
 ```
 git tag v#.#.#
 git push --tags
 ```
 
-15. Write up the release notes and add it to `www/releases.md`. Add it to index.md.
+13. Write up the release notes and add it to `www/releases.md`. Add it to index.md.
 
-16. Upload new motd
+14. Upload new motd
 
-17. Update the website by pushing to master. Update with new manpage. Update redirect to point to new manpage.
+15. Update the website by pushing to master. Update with new manpage. Update redirect to point to new manpage.
     - release notes
 
-18. Comb through issues and close the ones that have been solved, referencing the version number
+16. Comb through issues and close the ones that have been solved, referencing the version number
 
-19. Post github release notes on patreon.
+17. Post github release notes on patreon.
 
-20. Update the other distributions.
+18. Update the other distributions.
 
 # conda
 

docs/api/options.rst

@@ -35,16 +35,24 @@ Options can be overridden globally, or for all sheets of a specific type, or onl
 The options context should be referenced directly when setting:
 
     - ``sheet.options`` to *set* an option on a specific sheet instance (**sheet override**).
-    - ``<SheetType>.class_options`` to *set* a option default for a particular type of Sheet (**class override**).
+    - ``<SheetType>.options`` to *set* a option default for a particular type of Sheet (**class override**).
     - ``vd.options`` (or plain ``options``) to *set* an option globally for all sheets, except for sheets that have a sheet override (**global override**).
 
+.. note::
+
+    Class overrides prior to v2.10
+    ------------------------------
+
+    In VisiData v2.10 and earlier, class overrides had to be specified using ``<SheetType>.class_options``.
+    In subsequent versions, ``.options`` can be used for getting and setting options at all levels.
+
 Use ``sheet.options`` to *get* an option within the context of a specific sheet.
 This is strongly preferred, so the user can override the option setting on a sheet-specific basis.
 However, some options and situations are truly sheet-agnostic, and so ``vd.options`` (or plain ``options``) will *get* an option using the context of the **top sheet**.
 
 When getting an option value, VisiData will look for a sheet override first, then class overrides next (from most specific subclass all the way up to BaseSheet), then a global override, before returning the default value from the option definition itself.
 
-In general, plugins should use ``FooSheet.class_options`` to override values for the plugin-specific sheet type.
+In general, plugins should use ``FooSheet.options`` to override values for the plugin-specific sheet type.
 
 .. note::
 
@@ -118,4 +126,4 @@ Examples
     sheet.options.color_current_row = 'bold blue'
 
     # option set for all DirSheets
-    DirSheet.class_options.color_current_row = 'reverse green'
+    DirSheet.options.color_current_row = 'reverse green'

setup.py

@@ -3,7 +3,7 @@
 from setuptools import setup
 # tox can't actually run python3 setup.py: https://github.com/tox-dev/tox/issues/96
 #from visidata import __version__
-__version__ = '2.10.1'
+__version__ = '2.10.2'
 
 setup(name='visidata',
       version=__version__,

tests/diff-join.vd

@@ -3,7 +3,7 @@ sheet	col	row	longname	input	keystrokes	comment
 			open-file	tests/data2.tsv	o	
 data1	Key		key-col		!	
 data1			sheets-stack		S	
-sheets		キdata2	open-row-pyobj		^J	
+sheets		キdata2	open-row		^J	
 data2	Key		key-col		!	
 data2			sheets-stack		S	
 sheets		キdata2	select-row		s	

tests/edit-joinkey-2.vd

@@ -16,4 +16,4 @@ data2+data1	A		key-col		!
 data2+data1	Key	キ2	edit-cell	4	e	
 data2+data1			reload-sheet		^R	
 data2			sheets-stack		S	
-sheets		キdata1	open-row-pyobj		^J	
+sheets		キdata1	open-row		^J	

tests/edit-joinregular-2.vd

@@ -15,4 +15,4 @@ data2+data1			sort-keys-asc		g[
 data2+data1	A		key-col		!	
 data2+data1	C	キ2	edit-cell	a3	e	
 data2+data1			sheets-stack		S	
-sheets		キdata2	open-row-pyobj		^J	
+sheets		キdata2	open-row		^J	

tests/invalid_unicode_sqlite.vd

@@ -2,4 +2,4 @@ sheet	col	row	longname	input	keystrokes	comment
 global		encoding	set-option	latin-1		
 invalid_unicode		encoding	set-option	latin-1		
 			open-file	tests/invalid_unicode.sqlite	o	
-invalid_unicode		キTest	open-row-pyobj		Enter	open sheet with copies of rows referenced in current row
+invalid_unicode		キTest	open-row		Enter	open sheet with copies of rows referenced in current row

tests/join-cols-single-sheet.vd

@@ -6,4 +6,4 @@ sample_columns		キRegion	select-row		s	select current row
 sample_columns		キRep	select-row		s	select current row
 sample_columns			join-cols		&	add column from concatenating selected source columns
 sample_columns			sheets-stack		S	open Sheets Stack: join or jump between the active sheets on the current stack
-sheets		キsample	open-row-pyobj		^J	open sheet with copies of rows referenced in current row
+sheets		キsample	open-row		^J	open sheet with copies of rows referenced in current row

tests/join-different-types.vd

@@ -1,16 +1,16 @@
 sheet	col	row	longname	input	keystrokes	comment
 			open-file	tests/joining_error.xlsx	o	
 			open-file	tests/joining_error_interesting_records.csv	o	
-joining_error		キrecords	open-row-pyobj		^J	open sheet with copies of rows referenced in current row
+joining_error		キrecords	open-row		^J	open sheet with copies of rows referenced in current row
 joining_error_records			sheets-stack		S	open Sheets Stack: join or jump between the active sheets on the current stack
 sheets		キjoining_error_interesting_records	slide-up		K	slide current row up
 sheets		キjoining_error_interesting_records	slide-up		K	slide current row up
 sheets		キjoining_error_interesting_records	select-row		s	select current row
 sheets		キjoining_error_records	select-row		s	select current row
-sheets		キjoining_error_interesting_records	open-row-pyobj		^J	open sheet referenced in current row
+sheets		キjoining_error_interesting_records	open-row		^J	open sheet referenced in current row
 joining_error_interesting_records	Row		key-col		!	toggle current column as a key column
 joining_error_interesting_records			sheets-stack		S	open Sheets Stack: join or jump between the active sheets on the current stack
-sheets		キjoining_error_records	open-row-pyobj		^J	open sheet referenced in current row
+sheets		キjoining_error_records	open-row		^J	open sheet referenced in current row
 joining_error_records	Row		key-col		!	toggle current column as a key column
 joining_error_records			sheets-stack		S	open Sheets Stack: join or jump between the active sheets on the current stack
 sheets		キjoining_error_interesting_records	slide-up		K	slide current row up

tests/join-non-unique-cols.vd

@@ -14,4 +14,4 @@ sheets			join-sheets	outer	&
 benchmark+sample			columns-sheet		C	
 benchmark+sample_columns	name		sort-desc		]	
 benchmark+sample_columns			sheets-stack		S	
-sheets		キbenchmark+sample	open-row-pyobj		^J	
+sheets		キbenchmark+sample	open-row		^J	

tests/listofdictobj.vd

@@ -1,4 +1,4 @@
 sheet	col	row	longname	input	keystrokes	comment
 			open-file	sample_data/y77d-th95.json.gz	o	
 y77d-th95	geolocation		expand-col-depth	1	z(	expand current column of containers to given depth (0=fully)
-	geolocation.coordinates	0	open-cell-pyobj		z^J	open sheet with copies of rows referenced in current cell
+	geolocation.coordinates	0	open-cell		z^J	open sheet with copies of rows referenced in current cell

tests/load-ods.vd

@@ -1,3 +1,3 @@
 sheet	col	row	longname	input	keystrokes	comment
 			open-file	sample_data/benchmark.ods	o	
-benchmark		キbenchmark	open-row-pyobj		Enter	open sheet with copies of rows referenced in current row
+benchmark		キbenchmark	open-row		Enter	open sheet with copies of rows referenced in current row

tests/load-sqlite-view.vd

@@ -1,3 +1,3 @@
 sheet	col	row	longname	input	keystrokes	comment
 			open-file	sample_data/employees.sqlite	o	
-employees		キemp_view	open-row-pyobj		Enter	open sheet with copies of rows referenced in current row
+employees		キemp_view	open-row		Enter	open sheet with copies of rows referenced in current row

tests/load-zip.vd

@@ -1,3 +1,3 @@
 sheet	col	row	longname	input	keystrokes	comment
 			open-file	sample_data/benchmark.zip	o	
-benchmark		キsample_data/,benchmark.csv	open-row-pyobj		Enter	open sheet with copies of rows referenced in current row
+benchmark		キsample_data/,benchmark.csv	open-row		Enter	open sheet with copies of rows referenced in current row

tests/sqlite_withoutrowid.vd

@@ -1,3 +1,3 @@
 sheet	col	row	longname	input	keystrokes	comment
 			open-file	tests/without_rowid.db	o	
-without_rowid		キwithoutrowid	open-row-pyobj		Enter	open sheet with copies of rows referenced in current row
+without_rowid		キwithoutrowid	open-row		Enter	open sheet with copies of rows referenced in current row

tests/xlsx-color-cells.vd

@@ -1,4 +1,4 @@
 sheet	col	row	longname	input	keystrokes	comment
 global		xlsx_meta_columns	set-option	True		
 			open-file	sample_data/color-merged-cells.xlsx	o	
-color-merged-cells		キSheet1	open-row-pyobj		Enter	open sheet with copies of rows referenced in current row
+color-merged-cells		キSheet1	open-row		Enter	open sheet with copies of rows referenced in current row

tests/xlsx-empty-cell.vd

@@ -1,3 +1,3 @@
 sheet	col	row	longname	input	keystrokes	comment
 			open-file	sample_data/empty-cell.xlsx	o	
-empty-cell		キSheet1	open-row-pyobj		Enter	open sheet with copies of rows referenced in current row
+empty-cell		キSheet1	open-row		Enter	open sheet with copies of rows referenced in current row

tests/xlsx-merged-cells.vd

@@ -1,3 +1,3 @@
 sheet	col	row	longname	input	keystrokes	comment
 			open-file	sample_data/color-merged-cells.xlsx	o	
-color-merged-cells		キSheet1	open-row-pyobj		Enter	open sheet with copies of rows referenced in current row
+color-merged-cells		キSheet1	open-row		Enter	open sheet with copies of rows referenced in current row

visidata/__init__.py

@@ -1,6 +1,6 @@
 'VisiData: a curses interface for exploring and arranging tabular data'
 
-__version__ = '2.10.1'
+__version__ = '2.10.2'
 __version_info__ = 'VisiData v' + __version__
 __author__ = 'Saul Pwanson <[email protected]>'
 __status__ = 'Production/Stable'
@@ -67,6 +67,8 @@ def getGlobals():
 import visidata.pyobj
 import visidata.loaders.json
 import visidata._open
+import visidata.metasheets
+import visidata.cmdlog
 import visidata.save
 import visidata.clipboard
 import visidata.slide
@@ -75,14 +77,12 @@ def getGlobals():
 
 import visidata.menu
 import visidata.choose
-import visidata.metasheets
 import visidata.join
 import visidata.aggregators
 import visidata.describe
 import visidata.pivot
 import visidata.freqtbl
 import visidata.melt
-import visidata.cmdlog
 import visidata.freeze
 import visidata.regex
 import visidata.canvas
@@ -105,6 +105,7 @@ def getGlobals():
 import visidata.memory
 import visidata.macros
 import visidata.macos
+import visidata.repeat
 
 import visidata.loaders.csv
 import visidata.loaders.archive
@@ -148,6 +149,8 @@ def getGlobals():
 import visidata.loaders.arrow
 import visidata.loaders.parquet
 
+import visidata.loaders.vdx
+
 import visidata.form
 
 import visidata.ddwplay

visidata/_input.py

@@ -60,7 +60,7 @@ def until_get_wch(scr):
     return ret
 
 
-def splice(v, i, s):
+def splice(v:str, i:int, s:str):
     'Insert `s` into string `v` at `i` (such that v[i] == s[0]).'
     return v if i < 0 else v[:i] + s + v[i:]
 
@@ -227,7 +227,7 @@ def find_nonword(s, a, b, incr):
         elif ch == '^U':                           v = v[i:]; i = 0  # clear to beginning
         elif ch == '^V':                           v = splice(v, i, until_get_wch(scr)); i += 1  # literal character
         elif ch == '^W':                           j = find_nonword(v, 0, i-1, -1); v = v[:j+1] + v[i:]; i = j+1  # erase word
-        elif ch == '^Y':                           v = splice(v, i, vd.memory.clipval)
+        elif ch == '^Y':                           v = splice(v, i, str(vd.memory.clipval))
         elif ch == '^Z':                           suspend()
         # CTRL+arrow
         elif ch == 'kLFT5':                        i = find_nonword(v, 0, i-1, -1)+1; # word left

visidata/_open.py

@@ -132,4 +132,4 @@ def loadInternalSheet(vd, cls, p, **kwargs):
 
 
 BaseSheet.addCommand('o', 'open-file', 'vd.push(openSource(inputFilename("open: "), create=True))', 'Open file or URL')
-TableSheet.addCommand('zo', 'open-cell', 'vd.push(openSource(cursorDisplay))', 'Open file or URL from path in current cell')
+TableSheet.addCommand('zo', 'open-cell-file', 'vd.push(openSource(cursorDisplay))', 'Open file or URL from path in current cell')