Skip to content

Latest commit

 

History

History
110 lines (71 loc) · 6.94 KB

open-source-software-prize.md

File metadata and controls

110 lines (71 loc) · 6.94 KB

Open Source Software Prize

Pattern Theme / Category

  • Community Building
  • DEI
  • Promoting good practices
  • OSS Discovery
  • Reward & Recognition

OSPO Problems / Challenges

  • A lack of up to date information on how much open source software is being produced in a university.
  • A gap in awareness and/or motivation amongst development teams on best practices in open source and open science.
  • A need to uphold diversity and inclusion commitments.

Context

A research university creates large volumes of research outputs across every discipline.

University policy allows research teams to release their software as open source without requiring institutional permission. This approach fosters innovation but results in limited visibility into the scope and impact of open source contributions across the university.

A project registry has been set up by the university to track and increase the visibility of open source projects of note in the university.

Forces

Many labs and projects are expert practitioners in open source and open science. However, they may not be known or receive due recognition outside of siloed networks within their university.

Researchers and development teams on open science/open source projects are already overburdened.
There is no clear incentive for open source projects to add their work to the university’s open source project registry.

Without an ‘institutional gatekeeper’ or established framework in relation to open source and open science projects within the university, there is no mechanism for surfacing good practices, or for labs and projects to learn from each other.

Solution

Develop a university-wide ‘Open Source Software Prize’ with the objectives of:

  • Identifying labs and research teams working in open source that are outside of known networks.
  • Tying prize eligibility to being listed in the Registry (to incentivize participation and discovery).
  • Raising awareness of and elevating best practices in open source and open science.
  • Rewarding best practices and those projects making an impact in relation to diversity and inclusion.
  • Contributing to the ORCID ecosystem to recognize researchers and build trust.

The prize can be used as a vehicle for encouraging the adoption of open science tools to underrepresented fields.

ORCID records are permanent and belong to the researcher. Leveraging ORCID for official prize assertions makes the distinction more visible, durable, and relevant to the research community.

A sample list of typical awards for entrants may include:

  • Cash prizes.
  • Public recognition in academic fora.
  • Official distinctions that incorporate a ‘permanent academic recognition’ of achievement.

Award criteria should be devised to reward projects demonstrating practices and impact that the university wishes to foster in open source projects.

Awards criteria should account for biases and limitations in scholarly metrics. For example, some impactful software projects in computer science may not have associated articles or preprints. In the absence of DOIs, other measures of industry impact are needed.

A sample evaluation rubric may include:

  1. Scholarly Impact: Measure using field-normalized metrics like the Field Citation Ratio (FCR) to account for disciplinary differences.
  2. Industry Impact: Consider alternative measures such as GitHub activity, CHAOSS metrics, and downstream adoption in platforms like Hugging Face (for ML/AI projects).
  3. Open Source Best Practices: Evaluated adherence to principles like transparent development, documentation, and community engagement.
  4. Diversity and Inclusion: May be used as a tie-breaker, prioritizing projects with diverse and inclusive leadership or contributor communities.

Additional evaluation rubrics used by the GW OSPO for their student awards

Open Source Project Awards

Criteria - judging based on 4 categories evaluated with equal weight

  • 25% - Reproducibility - care taken to ensure the project is easily and accurately reproducible
  • 25% - Impact - Meaningful and interesting project
  • 25% - Adherence to open-source and software development best practices
  • 25% - Collaboration - inclusivity, cross discipline, demonstrative community building

Individual Contributor Awards

Complete applications will be evaluated and must meet a minimum threshold to be entered in the lottery:

  • Must include at least one meaningful contribution of code or documentation addressing existing issues and accepted by the maintainer of an existing active Project
  • The project you contribute to may be a GW or external project, but our evaluation committee will verify that the contributions are beneficial and not trivial

Please take care to understand that project maintainers can be extremely busy, so to be a good open source contributor, follow the codes of conduct and contribution guidelines and focus on tackling existing issues that align with your abilities.

Resulting Context

Entrants can be added to the existing project registry of open source projects within the university.

Entrants also adopt open science tools that provide permanent recognition of open science/open source contributions (e.g. ORCID).

The open source best practices criterion of the scoring rubric can be used as a didactic tool for helping teams improve how they share their code.

Known Instances

  • OpenSource@Stanford, Stanford Data Science Center for Open and Reproducible Science (CORES), Leland Stanford Junior University
  • GW OSPO Student Awards, George Washington University OSPO Student Award Program - Modeled after Stanford's program, with some modifications.

References

Contributor(s) & Acknowledgment