cmdutils and flatapi

[Erotemic]

This is a bigger PR, even though it only contains 2 functions: cmd and enable_flatapi. However the first of these functions is one of my most useful tools as measured by the latest ubelt documentation: (in clocks in at 82 times used in a non-ubelt project, placing at 14 / 127 of my most frequently used functions). The second is a developer convenience that incurs minimal overhead during runtime.

---

New Function: cmd

Lets start with cmd first. The boltons/cmdutils.py and tests\test_cmdutils.py are the key contributions of this PR. There are a few other minor changes, but I can remove those if need-be.

At its most basic it just executes some shell code. The reason this is better than subprocess.Popen (other than the name is much easier to type) is because it allows you to easily "tee" the output of your shell code. You can get the output in a dictionary and have it print to stdout at the same time and in real time (mostly, up to buffering issues that are frankly over my head). This is really useful when developing because you want to see the output of your long running process, but you might also want to be able to interact with it. In addition to these features you can also "detach" the process and run it asynchronously in the background.

I think cmd would be a really useful addition to boltons. I could talk about cmd a lot, but I'm just going to let its signature speak for itself:

def cmd(command, shell=False, detach=False, cwd=None,
        env=None, tee=None, tee_backend='auto', verbose=0):
    """
    Executes a command in a subprocess.

    The advantage of this wrapper around subprocess is that
    (1) you control if the subprocess prints to stdout,
    (2) the text written to stdout and stderr is returned for parsing,
    (3) cross platform behavior that lets you specify the command as a string
    or tuple regardless of whether or not shell=True.
    (4) ability to detach, return the process object and allow the process to
    run in the background (eventually we may return a Future object instead).

    Implementation is based on the collection of code samples on stackoverflow
    [1]_ [2]_ [3]_.


    Args:
        command (str or Sequence): bash-like command string or tuple of
            executable and args

        shell (bool, default=False): if True, process is run in shell.

        detach (bool, default=False):
            if True, process is detached and run in background.

        cwd (PathLike, optional): path to run command

        env (str, optional): environment passed to Popen

        tee (bool, optional): if True, simultaneously writes to stdout while
            capturing output from the command. If not specified, defaults to
            True if verbose > 0.  If detach is True, then this argument is
            ignored.

        tee_backend (str, optional): backend for tee output.
            Valid choices are: "auto", "select" (POSIX only), and "thread".

        verbose (int, default=0): verbosity mode. Can be 0, 1, 2, or 3.
            0 is quiet, 3 is loud.

    Returns:
        dict: info - information about command status.
            if detach is False ``info`` contains captured standard out,
            standard error, and the return code
            if detach is False ``info`` contains a reference to the process.

    Notes:

        * While this function strives to be cross-platform, there are certain
          insurmountable issues that arise when handling multiple shell
          languages.

        * Inputs can either be text or tuple based. On UNIX we ensure
            conversion to text if shell=True, and to tuple if shell=False. On
            windows, the input is always text based.  See [3]_ for a potential
            cross-platform shlex solution for windows.

    References:
        .. [1] https://stackoverflow.com/questions/11495783/redirect-subprocess-stderr-to-stdout
        .. [2] https://stackoverflow.com/questions/7729336/display-subprocess-stdout-without-distorti
        .. [3] https://stackoverflow.com/questions/33560364/python-windows-parsing-command-lines-with-shlex

    Example:
        >>> info = cmd(('echo', 'simple cmdline interface'), verbose=1)
        >>> print(info)  # xdoctest: +IGNORE_WANT
        simple cmdline interface
        {'out': 'simple cmdline interface\n',
         'err': '',
         'ret': 0,
         'proc': <subprocess.Popen at 0x7f0a4723a2e8>,
         'cwd': None,
         'command': "echo 'simple cmdline interface'"}

        >>> # The following commands demonstrate multiple ways co call cmd
        >>> info = cmd('echo str noshell', verbose=0)
        >>> assert info['out'].strip() == 'str noshell'

        >>> # windows echo will output extra single quotes
        >>> info = cmd(('echo', 'tuple noshell'), verbose=0)
        >>> assert info['out'].strip().strip("'") == 'tuple noshell'

        >>> # Note this command is formatted to work on win32 and unix
        >>> info = cmd('echo str&&echo shell', verbose=0, shell=True)
        >>> assert info['out'].strip() == 'str' + chr(10) + 'shell'

        >>> info = cmd(('echo', 'tuple shell'), verbose=0, shell=True)
        >>> assert info['out'].strip().strip("'") == 'tuple shell'
    """

---

New Function: enable_flatapi

Next is "enable_flatapi", which you'll have to tell me if you like or not. It lazily populates the global boltons API so everything is top-level (which helps reduce coder keystrokes). I used my mkinit tool to autogenerate the import statements needed to declare all boltons functions in a local scope, then I simply write to the globals scope. By default it does nothing but define one function in __init__, however if you call that function, then all of a sudden you can do

import boltons
boltons.enable_flatapi()
print(boltons.get_python_info())

You can also set BOLTONS_ENABLE_FLATAPI=TRUE in your environment to enable the behavior by default. I personally like this feature a lot; I think it adds minimal complexity to achieve a useful behavior.

---

Notes

Also note a pretty cool refactoring technique I did. I left the command line in.
When I needed to port my tests for cmd from ubelt to boltons, I needed to remove the dependencies on ubelt. I did this with a refactoring tool that currently exists in my netharn package on gitlab. I basically used this shell command:

xdoctest -m netharn.export.closer _closefile --fpath=$HOME/code/boltons/tests/test_cmdutils.py --modnames=ubelt,

to execute a "Closer" that recursively goes finds all dependencies on a particular package and recursively goes through that package to copy and paste all the relevant bits of code statically to the top of the file (note: my package does not execute the code to do this). Because its auto-generated its not perfect, so I did a bit of manual cleanup at the end to tie up the lose ends. It definitely made the task of porting code much less time consuming than it usually is. I mention it because I think it has the potential to become a really powerful refactoring tool, and I'd be interested in your opinion.

[Erotemic]

Note I believe some tests are failing due to the lack of xdoctest, if you decide to merge that PR I will rebase and the tests should pass. (I guess that would mean dropping 2.6 support from boltons or adding 2.6 support to xdoctest, so maybe I disable those doctests for now).