v1.6.1: SQLAlchemy 2.0, drop Python 3.7, and more!

[bmeares]

v1.6.0 – v1.6.1

Breaking Changes

• Dropped Python 3.7 support.
  The latest pandas requires 3.8+, so to use Pandas 1.5.x, we have to finally drop Python 3.7.

• Upgrade SQLAlchemy to 2.0.5+.
  This includes better transaction handling with connections. Other packages which use SQLAlchemy may not yet support 2.0+.

• Removed MQTTConnector.
  This was one of the original connectors but was never tested or used in production. It may be reintroduced via a future mqtt plugin.

Bugfixes and Improvements

• Stop execution when improper command-line arguments are passed in.
  Incorrect command-line arguments will now return an error. The previous behavior was to strip the flags and execute the action anyway, which was undesirable.

  $ mrsm show pipes -c
  
   💢 Invalid arguments:
    show pipes -c
  
     🛑 argument -c/-C/--connector-keys: expected at least one argument

• Allow bootstrap connector to create custom connectors.
  The bootstrap connector wizard can now handle registering custom connectors. It uses the REQUIRED_ATTRIBUTES list set in the custom connector class when determining what to ask for.

• Allow custom connectors to omit __init__()
  If a connector is created via @make_connector and doesn't have an __init__() function, the base one is used to create the connector with the correct type (derived from the class name) and verify the REQUIRED_ATTRIBUTES values if present.

• Infer a connector's type from its class name.
  The type of a connector is now determined from its class name (e.g. FooConnector would have a type foo). When inheriting from Connector, it is no longer required to explictly pass the type before the label. For backwards compatability, the legacy method still behaves as expected.

  from meerschaum.connectors import (
      Connector,
      make_connector,
      get_connector,
  )
  
  @make_connector
  class FooConnector:
      REQUIRED_ATTRIBUTES = ['username', 'password']
  
  conn = get_connector(
      'foo',
      username = 'abc',
      password = 'def',
  )

• Allow connectors to omit a label.
  The default label main will be used if label is omitted.

• Add meta keys to connectors.
  Like pipes, the meta property of a connector returns a dictionary with the kwargs needed to reconstruct the connector.

  conn = mrsm.get_connector('sql:temp', flavor='sqlite', database=':memory:')
  print(conn.meta)
  # {'type': 'sql', 'label': 'temp', 'database': ':memory:', 'flavor': 'sqlite'}

• Remove NUL bytes when inserting into PostgreSQL.
  PostgreSQL doesn't support NUL bytes in text ('\0'), so these characters are removed from strings when copying into a table.

• Cache pipe.exists() for 5 seconds.
  Repeated calls to pipe.exists() will be sped up due to short-term caching. This cache is invalidated when syncing or dropping a pipe.

• Fix an edge case with subprocesses in headless environments.
  Checks were added to subprocesses to prevent using interactive features when no such features may be available (i.e. termios).

• Added pprint(), get_config(), and attempt_import() to the top-level namespace.
  Frequently used functions pprint(), get_config(), and attempt_import() have been promoted to the root level of the meerschaum namespace, i.e.:

  import meerschaum as mrsm
  mrsm.pprint(mrsm.get_config('meerschaum'))
  
  sqlalchemy = mrsm.attempt_import('sqlalchemy')

• Fix CLI for MSSQL.
  The interactive CLI has been fixed for Microsoft SQL Server.

---

This discussion was created from the release v1.6.1: SQLAlchemy 2.0, drop Python 3.7, and more!.