Skip to content

GSF TSL Adapter List

Jump to bottom
J. Ritchie Carroll edited this page Aug 11, 2022 · 7 revisions

GSF Time-Series Library (TSL) Adapter List


Adapter Overview

The time-series data processing library (TSL) in the Grid Solutions Framework is a core collection of classes used to manage, process and respond to dynamic changes in fast moving streaming time-series data in real-time. TSL components allow applications to be architected as measurement routing systems. Any application can host the library which will allow the application to become a "measurement router" that can host any of the TSL's pluggable adapters. More information about the time-series adapters that are available as part of the Grid Solutions Framework can be found in the online adapter documentation.

The openPDC, SIEGate, and opehHistorian are examples of projects based on the TSL. To build a time-series application that can use any of these adapters, or other custom adapters built on the TSL, see the Project Alpha GitHub site and some simple steps on building a time-series library project.

The time-series adapter library can be molded to route measurements in any fashion, but the system defines three fundamental measurement processing categorizations to kick start most any time-series processing architecture; these are the Input, Action and Output interface adapters - plus, a recent addition called Filter. These base level adapters are called Iaon Adapters (pronounced "yo-wan" if you want to speak it). Each adapter type performs a basic time-series data operation:

Input Adapters: Used to "map" measurements from a data source, i.e., assign a timestamp and an ID to measured values parsed from a stream of data. New measurements are presented to the system by calling void OnNewMeasurements(ICollection<IMeasurement> measurements) method.

Interface: IInputAdapter, base class: InputAdapterBase

Action Adapters: Commonly used to filter and sort measurements by time allowing adapter to take an action, e.g., a calculation, on a synchronized set of data provided in the abstract void PublishFrame(IFrame frame, int index) method. The frame parameter contains a collection of measurements with correlated timestamps. If the action causes the creation of new measurements, new measurements are presented to the system by calling void OnNewMeasurements(ICollection<IMeasurement> measurements) method.

Interface: IActionAdapter, base class: ActionAdapterBase

The common use case for an action adapter is to sort incoming measurements by time. However, for other use cases where the burden of sorting is unneeded, another type of action adapter exists called a Facile Action Adapter which simply provides measurements as they arrive for any kind of unsorted type processing with a base class called the FacileActionAdapterBase.

Output Adapters: Used to queue measurements for processing, without sorting. Queued measurements are presented to the adapter for processing via the void ProcessMeasurements(IMeasurement[] measurements) method. If measurements continue to build up in memory and are not processed in a timely manner they will be removed from the queue as protective measure to prevent catastrophic out-of-memory failures. Since output adapters are used to archive data, this is often the slowest part in the system as disks tend to be a bottleneck. Outputs can optionally be set to filter based on a measurement's defined Source property which allows multiple outputs to be targeted to several different distributed outputs, helping large systems to stay ahead of the incoming data stream.

Interface: IOutputAdapter, base class: OutputAdapterBase

Filter Adapters: Used to apply global pre-processing and filtering to all incoming measurements. All measurements arriving from Input Adapters will be provided to each defined filter adapter, in the configured filter order. All measurement values, including timestamps, flags and data can be adjusted as appropriate - or the measurement can be excluded entirely from further processing. The modified measurements that should proceed with Iaon processing are presented to the adapter via the void ProcessMeasurements(IEnumerable<IMeasurement> measurements) method. For exlusion of measurements from further processing, filter adapters should override the void HandleNewMeasurements(ICollection<IMeasurement> measurements) method.

Interface: IFilterAdapter, base class: FilterAdapterBase

As this can be an invasive type of processing, its use should be very judicious, however, valid use cases include pre-processing of data value quality to assign a richer set of state flags for each measurement before proceeding with further, normal Iaon processing.

Collection Adapters: In addition to individual input, action, and output adapters, there are also independent adapter collections acting as a containers for multiple adapters yet are exposed to the Iaon system as a single adapter from an input / output measurement perspective. These adapters request as input the set of desired input measurements for all the adapters in the collection and/or expose as output the set of possible cumulative output measurements for all the adapters in the collection. Normally these collection-based adapters will internally mange routing input measurements to each of the adapters in the collection, as appropriate. There are base classes for these types of collection-based adapters as follows:

  • Input Collection Adapter: Interface: IIndependentAdapterManager, base class: IndependentInputAdapterManagerBase
  • Action Collection Adapter: Interface: IIndependentAdapterManager, base class: IndependentActionAdapterManagerBase
  • Output Collection Adapter: Interface: IIndependentAdapterManager, base class: IndependentOutputAdapterManagerBase

An example of this kind of adapter is the Bulk Sequence Calculator. This collection-based adapter creates multiple instances of the single Sequence Calculator adapter, one for each available set of configured inputs.

Helpful Adapter Links

Adapter Categories

Computational Adapters

The following computational adapters have been developed over the years by GPA for processing real-time streaming synchrophasor data. All adapters are included with all GPA time-series products, e.g., openPDC and openHistorian.

Some recently added adapters, e.g., the Bulk Sequence Calculator, will auto-configure itself, creating output measurements as needed based on existing configuration.

  • Angle Difference - (action) calculates an angle difference from time-series inputs taking into account wrapping
  • Average Frequency (area) - (action) calculates an average frequency from validated time-series inputs
  • Average Frequency (one second) - (action) calculates average of the input frequencies over each full second
  • Frequency Excursion - (action) detects a frequency excursion from a configured set of time-series inputs
  • Impedance - (action) calculates impedance outputs from a configured set of time-series inputs
  • Loss-of-Field - (action) detects loss-of-field from a configured set of time-series inputs
  • Power - (action) calculates a single set of P/Q/S values from configured time-series inputs
  • Power (bulk) - (action-collection) calculates bulk P/Q/S values based on configured phasor mappings from time-series inputs
  • Power Stability - (action) calculates a standard deviation of power useful simple oscillation detection from time-series inputs
  • Reference Angle - (action) creates a virtual angle from several time-series input angles that can be used as a reference angle
  • Reference Magnitude - (action) creates a virtual reference angle magnitude from several time-series input magnitudes
  • Sequence Values - (action) calculates a single set of sequence values (+/-/0) from configured time-series inputs
  • Sequence Values (bulk) - (action-collection) uses to calculate bulk sequence values (+/-/0) based on system configuration

Dynamic Calculator Adapters

The dynamic calculation adapters are used to process expressions using the Fast Lightweight Expression Evaluator (FLEE).

In the case of the database and e-mail notification adapters, the format will be to create a boolean expression based on incoming time-series measurements such that when the expression evaluates to True, the database operation will execute or the e-mail notification will be sent.

See the following documentation on FLEE expressions: Flee wiki examples

  • Email Notification - (action) sends an e-mail when time-series values trigger a Boolean expression as True
  • Dynamic Calculator - (action) produces a new time-series value based on a user equation (expression) from time-series inputs
  • Database Notification - (action) executes a database operation when time-series values trigger a Boolean expression as True

Phasor Protocol Adapters

The following phasor protocol adapters form the foundation of allowing a time-series application to consume, produce and operate on streaming synchorphasor data from a measurement perspective. Mapping of frames of synchrophasor data to and from measurements happens in the Phasor Measurement Mapper and the Base Concentration Adapter, both of which operate on most any synchorphasor protocol, e.g., IEEE C37.118.

Testing Adapters

The testing adapters are available to developers and users experimenting with time-series framework to produce various kinds of data feeds for the purposes of scaling a system for stress testing or validating adapter development under various conditions. The adapters also include a time-synchronization adapter that will take incoming data, e.g., from a GPS-time synchronized PMU, and use the incoming data to set the local clock -- this is useful for computers in lab environments where hardware clock synchronization options many not be available and NTP based-synchronization options are not accurate enough.

Database / Historian Adapters

These adapters provide production quality connectivity options to commericial historians, e.g., OSIsoft PI and eDNA, as well as providing options to push data into RDBMS and time-series databases. Do keep in mind that the time-series framework combined with high-volume, streaming synchrophasor data can quickly overwhelm these types of data archive systems. If you are experimenting with time-series archival systems, we suggest you take a look at GPA's openHistorian 2.0 which was explicitly designed for this kind of high-speed data.

Data Quality Monitoring Adapters

Products like the openPDC use the data quality adapters to monitor and alarm on data quality issues in streaming time-series data, covering a variety of common quality issues with streaming data, e.g., data flat-line scenarios (i.e., streaming but value is fixed) and out-of-range issues.

  • Alarm Adapter - (facile-action) used generate alarms based on time-series data
  • Alarm State Flagger - (filter) used to generate new measurement quality flags based on alarm states
  • Best Value Selector - (action) used to produces a new signal by selecting the best values from a time-series values
  • Flatline Test - (action) used test time-series values to determine if values have flat-lined
  • Range Test - (action) used test time-series values to determine if values satisfy a range condition
  • State Flags Transfer Adapter - (filter) used to transfers flags to / from time-series values
  • Device Statistic Adapters - (facile-action) used to monitor and report device alarm states based on time-series input values

File Operation Adapters

Time-series operations as it relates to files include options for reading and writing values that can be mapped to measurements in the GSF Time-Series Library. Existing options include an ICCP style adapter which will drop a file periodically with a configured set of measurements in a format consumable by many ICCP systems in use with SCADA deployments, providing a simple way to get calculations or other external time-series data into SCADA.

  • COMTRADE File Exporter - (action) used to export time-series values to COMTRADE file
  • ICCP File Exporter - (action) used to save time-series data a file suitable for ICCP import / SCADA
  • CSV Export Adapter - (output) exports time-series values to CSV files on a schedule with a fixed format of Timestamp,ID,Value
  • CSV Import Adapter - (input) imports time-series values from CSV files on a schedule and fixed format of Timestamp,ID,Value
  • CSV Input Adapter - (input) imports time-series values from CSV files in highly-customizable fashion with column mappings
  • CSV Output Adapter - (output) exports time-series values to a CSV formated: Signal ID,Measurement Key,Timestamp,Value
  • File Block Reader - (facile-action) used to read and transport file data via time-series framework (e.g., over STTP)
  • File Block Writer - (output) used to receive and write file data via time-series framework (e.g., over STTP)
  • EPRI File Exporter - (action) used to export calculated time-series values to formats used by EPRI analysis applications
  • EPRI File Importer - (input) used to import calculated time-series values from formats used by EPRI analysis applications
  • EPRI Metric Importer - (input) used to import time-series statistic values from formats used by EPRI analysis applications
  • FTP Reader - (facile-action) used to read and download files using FTP
  • Metadata Export - (facile-action) exports system metadata to a file for transport as time-series data chunks (e.g., over STTP)
  • Metadata Import - (facile-action) imports system metadata from a file-based transport as time-series data chunks (e.g., over STTP)

Publish / Subscribe Adapters

The following adapters are used to transffer streaming data using publish-subscribe data transfer protocols that have been optimized for exchanging streaming time series style data, such as synchrophasor data that is used in the electric power industry, over Internet Protocol (IP). The protocols, e.g., STTP/IEEE 2664, support transferring both real-time and historical time series data at full or down-sampled resolutions. These types of protocols have benefits that are realized at scale when multiplexing very large numbers of time series data points at high speed, such as, hundreds of times per second per data point.

Ideal use cases for protocols like STTP include those that need to securely exchange large numbers of continuously measured data. See the GPA overview of STTP for more detail.

  • STTP (IEEE 2664) Publisher - (action-collection) used to publish time-series data using the Streaming Telemetry Transport Protocol
  • STTP (IEEE 2664) Subscriber - (input) used to subscribe to time-series data using the Streaming Telemetry Transport Protocol
  • GEP Publisher - (action-collection) used to publish time-series data using the Gateway Exchange Protocol
  • GEP Subscriber - (input) used to subscribe to time-series data using the Gateway Exchange Protocol

Additional Adapters

The remaining adapters include various options for processing time-series data, including receiving time-series measurements from DNP3 and Modbus.

  • Azure Event Hub - (output) exports time-series values to an Azure event hub for cloud based analysis
  • DNP3 - (input) uses to import time-series values from a DNP3 data source
  • Modbus Poller - (input) import time-series values from a Modbus data source
  • WAV Audio Input Adapter - (input) used to read audio data from a WAV file for export as time-series data
  • Audio Input Adapters - (input) used to sample audio data from a microphone for export as time-series data
Clone this wiki locally