Thoughts on adding support for datasets

[tobiasbp]

Looking at datasets, there are two types: FILESYSTEM & DATASET

They do not have the same fields. My first though on how to handle this, would be to have one abstract class Dataset with the
properties shared by both types of dataset. Then we would need two other classes ( DatasetFilesystem and DatasetDataset) implementng Dataset end expanding with relevant properties. What Class of object should be in the list datasets in Machine? That approach seems like a mess.

I guess you could just have the two abstract classes Filesystem & Dataset. Then the Machine would a list of datasets and another list of filesystems?

What do you think @sdwilsh ? What would be a good way to handle this?

Type filesystem

{
                'atime': {'parsed': False,
                          'rawvalue': 'off',
                          'source': 'INHERITED',
                          'value': 'OFF'},
                'casesensitivity': {'parsed': 'sensitive',
                                    'rawvalue': 'sensitive',
                                    'source': 'NONE',
                                    'value': 'SENSITIVE'},
                'children': [],
                'compression': {'parsed': 'lz4',
                                'rawvalue': 'lz4',
                                'source': 'INHERITED',
                                'value': 'LZ4'},
                'compressratio': {'parsed': '1.63x',
                                  'rawvalue': '1.63x',
                                  'source': 'NONE',
                                  'value': '1.63x'},
                'copies': {'parsed': 1,
                           'rawvalue': '1',
                           'source': 'DEFAULT',
                           'value': '1'},
                'deduplication': {'parsed': 'off',
                                  'rawvalue': 'off',
                                  'source': 'DEFAULT',
                                  'value': 'OFF'},
                'exec': {'parsed': True,
                         'rawvalue': 'on',
                         'source': 'DEFAULT',
                         'value': 'ON'},
                'id': 'freenas-boot/grub',
                'mountpoint': None,
                'name': 'freenas-boot/grub',
                'origin': {'parsed': '',
                           'rawvalue': '',
                           'source': 'NONE',
                           'value': ''},
                'pool': 'freenas-boot',
                'quota': {'parsed': None,
                          'rawvalue': '0',
                          'source': 'DEFAULT',
                          'value': None},
                'readonly': {'parsed': False,
                             'rawvalue': 'off',
                             'source': 'DEFAULT',
                             'value': 'OFF'},
                'recordsize': {'parsed': 131072,
                               'rawvalue': '131072',
                               'source': 'DEFAULT',
                               'value': '128K'},
                'refquota': {'parsed': None,
                             'rawvalue': '0',
                             'source': 'DEFAULT',
                             'value': None},
                'refreservation': {'parsed': None,
                                   'rawvalue': '0',
                                   'source': 'DEFAULT',
                                   'value': None},
                'reservation': {'parsed': None,
                                'rawvalue': '0',
                                'source': 'DEFAULT',
                                'value': None},
                'share_type': 'UNIX',
                'snapdir': {'parsed': None,
                            'rawvalue': 'hidden',
                            'source': 'DEFAULT',
                            'value': 'HIDDEN'},
                'sync': {'parsed': 'standard',
                         'rawvalue': 'standard',
                         'source': 'DEFAULT',
                         'value': 'STANDARD'},
                'type': 'FILESYSTEM'
}

Type volume

,
 {
'children': [],
  'comments': {'parsed': 'Virtual Machines go here',
               'rawvalue': 'Virtual Machines go here',
               'source': 'INHERITED',
               'value': 'Virtual Machines go here'},
  'compression': {'parsed': 'lz4',
                  'rawvalue': 'lz4',
                  'source': 'INHERITED',
                  'value': 'LZ4'},
  'compressratio': {'parsed': '1.00x',
                    'rawvalue': '1.00x',
                    'source': 'NONE',
                    'value': '1.00x'},
  'copies': {'parsed': 1, 'rawvalue': '1', 'source': 'DEFAULT', 'value': '1'},
  'deduplication': {'parsed': 'off',
                    'rawvalue': 'off',
                    'source': 'DEFAULT',
                    'value': 'OFF'},
  'id': 'ultraman/vm/pfSense_pfSense_clone2',
  'mountpoint': None,
  'name': 'ultraman/vm/pfSense_pfSense_clone2',
  'origin': {'parsed': 'ultraman/vm/pfSense@pfSense_clone2',
             'rawvalue': 'ultraman/vm/pfSense@pfSense_clone2',
             'source': 'NONE',
             'value': 'ultraman/vm/pfSense@pfSense_clone2'},
  'pool': 'ultraman',
  'readonly': {'parsed': False,
               'rawvalue': 'off',
               'source': 'DEFAULT',
               'value': 'OFF'},
  'refreservation': {'parsed': None,
                     'rawvalue': '0',
                     'source': 'DEFAULT',
                     'value': None},
  'reservation': {'parsed': None,
                  'rawvalue': '0',
                  'source': 'DEFAULT',
                  'value': None},
  'share_type': None,
  'sync': {'parsed': 'standard',
           'rawvalue': 'standard',
           'source': 'DEFAULT',
           'value': 'STANDARD'},
  'type': 'VOLUME',
  'volblocksize': {'parsed': 16384,
                   'rawvalue': '16384',
                   'source': 'NONE',
                   'value': '16K'},
  'volsize': {'parsed': 8589934592,
              'rawvalue': '8589934592',
              'source': 'LOCAL',
              'value': '8G'}
}

[sdwilsh]

Is it the case that VOLUME is a zfs volume (block device that is created with zfs create -V), and FILESYSTEM is a dataset (filesystem that is user-visibile and created with zfs create)?

Regardless, I think it might make sense to split this up, and use the query function to filter to only get one type and expose each on the object.

[tobiasbp]

Your understanding of the types is correct.

So you think we should create two new completely separate abstract classes DatasetVolume & DatasetFilesystem? What do you think about having a base class Dataset which DatasetVolume and DatasetFilesystem would inherit from?

What about creating a data type for the four-value dicts returned for most of the values? How would you define that?

  'volblocksize': {'parsed': 16384,
                   'rawvalue': '16384',
                   'source': 'NONE',
                   'value': '16K'},

What do you think @sdwilsh ?

[sdwilsh]

So you think we should create two new completely separate abstract classes DatasetVolume & DatasetFilesystem? What do you think about having a base class Dataset which DatasetVolume and DatasetFilesystem would inherit from?

All of this sounds reasonable.

What about creating a data type for the four-value dicts returned for most of the values? How would you define that?

Maybe something like this:

class DatasetPropertyType(enum.Enum):
  ...

class DatasetProperty(object):
  def __init__(self, parsed: Any, source: DatasetPropertyType) -> None:
    self._value = parsed
    self._source = source

  @property
  def value(self) -> Any:
    return self._value

  @property
  def source(self) -> DatasetPropertyType:
    return self._source

It looks like value in the object is what they'd want to show in the UI, whereas parsed looks like the value most useful for an API like ours

[sdwilsh]

#92 does the basics that are common between both of these. I might land that as-is, and we can then add an the additional specializations to it as needed (I do not need the specializations today for the Home Assistant integration).

I didn't add the DatasetProperty stuff either, but I'll probably toss that in before merging.