Survey
: A solicitation for input from aRespondent
.Response
: A completed or in progress response to a survey. A survey has exactly one response.Library
: The collection of allresponses
.Registry
: The collection of availablesurvey objects
for use inresponses
.Analysis
: The process of converting many survey responses to a Sankey diagram by anAnalyst
interacting with theLibrary
. Surveys are searched by title,tags
, and/or other survey object fields.
📝 We think it would be really cool for surveys to be "composed", e.g. a respondent fills out a survey for a single application and its upstreams. Then, another respondent fills out a survey focused on an SBA and its upstream applications. The data for the upstream applications in the second survey can come from the response to the first survey, enabling one large graph to be composed from multiple surveys.
A list of available response objects
. Drives consistency in naming across surveys
to
support analysis
and also to ease data entry for Respondents
, over time.
Respondents
may add new applications
, data products
, and observing systems
to the registry
.
There is a generic Registered Survey Object
structure that applies to Applications,
Data Product, and Observing Systems. There is a separate structure to register
Societal Benefit Area objects
.
Ongoing efforts are being made to align this data structure with best practices and partner organization structures (e.g. Polar Observing Assets working group, Arctic Data Center, Federated Search crew) with the goal to be able to import or sync items in the registry from an external source.
Object Type
: Type of Object -Observing System
,Data Product
, orApplication
. (Multiplicity: 1..1; Format: Pick from Observing System, Data Product, or Application)Short Name
: Short name/description of the object, which would be displayed in the analysis to save space. Preferably in the format [Organization acronym] [name], e.g. USGS Streamgage Network. (Multiplicity: 1..1; Format: Text)Full Name
: Full name of the object. In the case where the full name is brief, this may be the same as theshort name
. Preferably in the format [Organization acronym] [name], e.g. USGS Groundwater and Streamflow Information Program. (Multiplicity: 1..1; Format: Text)Organization
: The entity/entities responsible for the operation of the observing system, data product, or application. Preferably in the format [Full name] [acronym], e.g. National Snow and Ice Data Center (NSIDC). (Multiplicity: 1..n; Format: Text/List)- NOTE: future enhancement would be to use Type-ahead standard names
Funder
: The entity responsible for funding the observing system, data product, or application. Preferably in the format [Full name] [acronym], e.g. National Snow and Ice Data Center (NSIDC). (Multiplicity: 1..n; Format: Text/List)- NOTE: future enhancement would be to use Type-ahead standard names
Country/Countries
: The countries contributing to running or funding an object. (Multiplicity: 1..n; Format: Pick from ISO 3166 or similar)Website
: The URL to access the referenced object directly (Multiplicity: 0..1; Format: URL)Description
: Short summary of the object, including geographic or thematic scope. (Multiplicity: 1..1; Format: Text)Contact Name
: The name of a point of contact for the object, e.g. a data manager or program coordinator. (Multiplicity: 0..1; Format: Text)Contact Title
: The title of a point of contact for the object. (Multiplicity: 0..1; Format: Text)Contact Email
: The email address for a point of contact for the object. (Multiplicity: 0..1; Format: Email)Tags
: Please add three or more tags to indicate thematic- or geographic- information about this object to aid in analysis and discovery. (Multiplicity: 0..n; Format: Text)- This is a user-defined field that admins can and will edit. It creates additional flexibility in
the analyis, for example allowing for a regionally- or thematically-focused
analysis
. - For the early phase of this tool, the description above encourages users to include multiple tags, but the software does not require it.
- This is a user-defined field that admins can and will edit. It creates additional flexibility in
the analyis, for example allowing for a regionally- or thematically-focused
Version
: Match the version identification system used by the observing system, data product, or application managers. This field allows theanalyses
to reflect change over time. (Multiplicity: 0..n; Format: Text)Persistent Identifier
: A standard way to refer back to the object's source, usually a DOI (Multiplicity: 0..n; Format: Text)Real
: A boolean indicating that an object is real (not hypothetical). Maybe could be calledhypothetical
instead? (Multiplicity: 1..1; Format: Boolean)
Application
objects also include:
-
Application Performance Criteria
: Text description of what the ideal performance of this data product looks like. (Multiplicity: 1..1; Format: Text) -
Application Performance Rating
: 0-100 rating of the performance of the application compared to theApplication Performance Criteria
(0=No performance, 100 = perfect) (Multiplicity: 1..1; Format: Number 0-100) -
Rationale
: Text description answering the question: What accounts for this performance rating? -
Gaps
: If the rating is less than "ideal" what improvements are needed.NOTE We intend to do
current state
anddesired state
analyses
. In adesired state
analysis
, thereal
field allows us todesired state
(hypothetical) objects. For instance, the USGS stream gage network if funding increased and there were X number of additional gages. This allows us to trace the impact of changes on the entire network. Related to the surveytype
.
The rating answers two questions - how important is a particular Observing System
,
Data Product
, or Application
and how well does it perform. It is reflected in the
analysis
by the thickness and color of the link connecting two survey objects
. For
instance: Imagine a satelite (observing system
) that is very important to a sea ice
data product
but performs poorly because of persistent Arctic cloud cover and high
latency (gaps
. Those would be linked by a thick (high criticality
) red (low
performance
) line.
flowchart LR
A[Observing Systems]--> |supports| B[Data Products]
A --> |self| A
B --> |supports| C(Applications)
B --> B
C --> C
C --> |supports| D[Societal Benefit Areas]
Response objects exist both as a definition in the registry
and an instantiation with
rating(s) and other fields associated with a response
. The following specifications
pertain to rated instances, not registry
definitions.
Link
: Defines whichsurvey objects
are connected. The rating is applied to thatlink
.Criticality rating
: 0-10 rating of the criticality of an input to an output, e.g. criticality of anobserving system
to adata product
. This answers the question: On a scale of 1-10, how much would the loss of this input impact the performance of yourdata product
orapplication
(1 - very little impact; 10 - complete loss of performance).Criticality rationale
: Text description answering the question: What accounts for this criticality rating? If there is a close equivalent product, why do you prefer this one?Performance rating
: 0-100 rating of the performance of the subject. Answers the question: What is your satisfaction with this input? (0=No performance, 100 = perfect)Performance rationale
: Text description answering the question: What accounts for this performance rating? Include any journal articles, statements or contextual observations that might help us to understand your rating.Gaps
: If the rating is less than "ideal" what improvements are needed.Variable or Attribute
: If anobserving system
ordata product
contains many observable properties or variables, this allows arespondent
to specify which field they used.Rated by
: Link to therespondent
who provided this rating.
The node color
of an observing system
or a data product
is defined in this
document: https://docs.google.com/presentation/d/1RmEGcPkC3_9o3qeAndv0QvAcdZwFIHC-/edit#slide=id.g1e651286dde_0_54
The node color
of an application
is rated separately based on the application performance
compared to the application performance criteria
.
While only a small number of respondents will provide responses on Observing Systems
through
Applications
, a larger number of respondents (up to ~20) will evaluate the relationships between
Applications
and Societal Benefit Areas
. For now, we expect users to follow instructions, but
eventually, we may want to apply restrictions so that the right people can only fill out the right
parts.
ID
: Computer generated identifierTitle
: Unique name of surveyDescription
: Narrative description of the survey topicTags
: This is a user-defined field that admins can and will edit. It creates additional flexibility in the analyis, for example allowing for a regionally- or thematically-focusedanalysis
.Respondents
: List ofrespondents
with edit-accessDescription
: Narrative description of the survey topic; may be displayed as a prompt to the user.Status
: WIP, Published, Closed, Archived? Should we have dedicated date fields, e.g.opened_date
,published_date
, etc.
---
title: Survey state
---
flowchart LR
new["New"]
wip["In progress"]
review["In review"]
published["Published"]
archived["Archived"]
any["Any state"]
new --> |response updated| wip
wip --> |response completed| review
review --> |admin rejects| wip
review --> |admin approves| published
published --> |admin unapproves| review
any <--> |admin sets| archived
Private
: Can this survey be viewed by non-registered members? (Or should it be restricted to individuals?)
Year
: Allows for repeated analyses to show change over time - (We currently havecreated_timestamp
field which can show when it was originally created. Year seems too wide?)Type
: Surveys could be requesting information about the current state or desired state ofresponse objects
. For "desired state" surveys, we wouldn't be interested ingaps
. (TODO: This concept needs to be refined in the future.)Parent
: Another survey that this one is based on. (TODO: Do we want a version number?)
While most surveys
will link observing systems, data products, applications, and societal benefit
areas, some may simply allow for a data manager to link the data product
to observing systems
without any related application
. Additionally, a group of up to ~20 respondents
known as a
societal benefit cohort
may provide ratings on how 1-3 specific SBAs
relate to the
application
.
Respondents
:Experts
who can contribute toresponse
.Tags
: Arbitrary strings for grouping surveys. For example:river-watch
.Version
: Surveys can be re-issued, with changes, as a new version. E.g. a new survey is created as version "1" (version may be represented to user as creation date). Later, the survey is copied to version "2", altered, and re-issued, possibly to a differentrespondent
. We also copy the old response to the new version so therespondent(s)
do not need to start from scratch.Validated date
: Date when an admin marked this survey valid for inclusion in thelibrary
. (NOTE: If populated represents the status is at least validated. Probably want to do all statuses this way.)Status
: Draft, ready to validate, validated, ??? (TODO: Better to use date fields instead? e.g. a submitted response hassubmitted_date
populated, a draft does not)
A visualization that is generated from the library
of responses by an Analyst
selecting filters based on fields of response objects
, such as tags
. We envision
being able to blend responses
around common Objects or Fields within those Objects, for
example: "show all responses
tied to a given data product
". Or "show all where tags
include 'rivers'", etc.
Name
: UniqueDescription
Analyst
- Filter criteria or list of
responses
associated with the analysis. (TODO: If we don't want an analysis to change over time, we should probably link it toresponses
by ID. But if we do want the analysis to be automatically updated as moreresponses
are submitted with e.g. a specifictag
, we should use filter criteria)
A user who creates a survey
and invites experts
to respond, in addition to
validating responses
and marking them valid/complete.
When creating a survey
, the admin either creates or selects an application
in the
registry to associate with the survey.
Can create surveys
by creating new versions
of old surveys.
Fields/relationships:
Name
Email
ORCID iD
Surveys created
Affiliation
Biography
A user who is invited to register to provide a response
to a survey
. Admins will
assign Surveys to Experts. Experts can contribute to multiple Surveys; Surveys can have
many Experts.
Experts may add new applications
, data products
, and observing systems
to the
registry
, or may select existing objects
.
Fields/relationships:
Name
Email
ORCiD
(optional? | validate format, e.g.:0000-0003-3260-5445
)Biography
Affiliation
(should this be saved on a response-by-response basis? Affiliation may change between surveys)Tags
of interest (areas of expertise, e.g.river-watch
,hydrology
)Responses
submitted
These are people who are viewing the data. Not contributing to it in any way.
A user who registers to generate one or more analysis
. Eventually, we’d like Analysts
to be able to save their Analyses, recreate them and update them. I’m not sure we need
or should track things like Orcid ID, Bio, Affiliation for Analysts, but we could give
them option. It would help with our analytics.
We might also need to support a Guest Analyst so someone could do visualizations without registering. If we don’t want to create user accounts, then basically everyone would be a Guest Analyst and they could just export .jpgs and JSON to recreate their analyses.
Fields/relationships:
Name
ORCiD
(optional? | validate format, e.g.:0000-0003-3260-5445
)Biography
Affiliation
Tags
of interest (areas of expertise, e.g.river-watch
)Analyses
submitted
A group of respondents
who provide responses
to surveys
focused on SBA ratings. A cohort
will generally be solicited to complete many Societal Benefit surveys
, focusing on a
small number of SBAs
, to provide a view of the societal benefits of many
applications
.
Fields/relationships:
Name
: UniqueDescription
Expert(s)
: Members of cohort
Enable analysts to filter surveys by "views". We imagine a "River watch" view, a "Rivers" view, a "Risk mitigation" view, "A sea ice index" view.
Admins would manage the views. E.g. create a new view, give it a name, and populate tags
for surveys that should display in that view.