PkgSpec_LATEST
Packages are the common artifact that can be passed between SDK, V&V, and service platform. They can contain single VNFs, complex network services, or test results only.
We aim to design the most generic, reusable, feature complete, and compatible package format in the NFV landscape!
This page describes the package format used in 5GTANGO in a more technical way. The general goal is to keep 5GTANGO packages compatible to the ETSI VNF package format which is based on the TOSCA CSAR standard. In addition to that we will try to contribute innovations of our package format back to ETSI.
The following figure shows the general package concept as described in D2.2.
5GTANGO packages can contain single VNFs, complex network services, or test results only. This is a major difference to SONATA packages, but aligns them more with the view of ETSI, OSM, ONAP, or OpenBaton.
5GTANGO packages can contain different flavors of descriptors/artifacts for the same service, e.g., one NSD for 5GTANGO SP and one for OSM. This allows us to target different platforms with the same package and bypasses problems that occur if you try to automatically translate descriptors. We call this concept layering. Each layer is identified by one or more tags.
To support slim packages, it is possible to reference artifacts, e.g., disk images or any other file, from within the 5GTANGO package. The unpackager will try to resolve these references when the package is unpacked, e.g., download the referenced disk image.
To reference artifacts, we follow an approach that is inspired by Unix filesystem symlinks: For each reference, a file with the ending *.ref is created as a placeholder for the referenced artifact inside the package. This file contains all information required to resolve the reference, e.g., URIs, protocol information, checksums, or credentials of an FTP server. The benefit of this approach is that it fully compatible with the CSAR and ETSI standards, which would not be the case if we try to put reference information inside manifest files or package descriptors. The format of reference files is described below.
5GTANGO packages can reference other 5GTANGO packages using the vendor.name.version triple known from SONATA. With this, test result packages can reference the services to which they belong. Checksums allow to verify the integrity of references packages.
This section specifies the structure and file formats of a 5GTANGO package.
The CSAR standard supports two types of package formats: TOSCA CSAR v1.0 (CSAR in Chapter 16) and TOSCA simple profile YAML (CSAR 1.1 in Chapter 6). The first one provides a more specific package structure and does not require to use TOSCA descriptors to define the packaged service. ETSI SOL004 V2.3.1 mentions that is supports both formats, but its implementation of the second one is not specified very well and quite inconsistent. Further does 5GTANGO not use TOSCA simple profile YAML descriptors for service specification.
Because of this, we first focus on aligning our specification to the TOSCA CSAR v1.0 (CSAR in Chapter 16) format using the specification of ETSI SOL004 V2.3.1. We will support the second CSAR format later on in the project, once ETSI refines their package specification.
Package structure that is compatible to TOSCA CSAR v1.0 (CSAR in Chapter 16) and ETSI SOL004 V2.3.1.
Quote from the original specification:
A CSAR is a zip file containing at least two directories, the TOSCA-Metadata directory and the Definitions directory. Beyond that, other directories MAY be contained in a CSAR, i.e. the creator of a CSAR has all freedom to define the content of a CSAR and the structuring of this content as appropriate for the cloud application. The TOSCA-Metadata directory contains metadata describing the other content of the CSAR. This metadata is referred to as TOSCA meta file. This file is named TOSCA and has the file extension .meta.
In addition to this, ETSI SOL004 V2.3.1 defines that there MUST be an additional manifest file that contains ETSI standard-specific information and has the extension *.mf.
Minimal example:
/
├── TOSCA-Metadata
│ ├── TOSCA.meta
│ └── NAPD.yaml
├── Definitions
│ └── mynsd.yaml
└── mynsd.mf
General example:
/
├── TOSCA-Metadata
│ ├── TOSCA.meta
│ └── NAPD.yaml
├── Definitions
│ └── mynsd.yaml
.
.
.
├── Images
│ └── cloudimage.ref
├── Scripts
├── Tests
.
.
.
├── mynsd.mf
└── ChangeLog.txt
As mentioned above, this format is not yet supported by 5GTANGO. We will work on this in the future.
Entry point file as defined in TOSCA CSAR v1.0 (CSAR in Chapter 16). This file describes the content of the entire package. It contains the references to the included files, file types and checksums.
Quote from the original specification:
The TOSCA meta file includes metadata that allows interpreting the various artifacts within the CSAR properly. The TOSCA.meta file is contained in the TOSCA-Metadata directory of the CSAR. A TOSCA meta file consists of name/value pairs. The name-part of a name/value pair is followed by a colon, followed by a blank, followed by the value-part of the name/value pair. The name MUST NOT contain a colon. Values that represent binary data MUST be base64 encoded. Values that extend beyond one line can be spread over multiple lines if each subsequent line starts with at least one space. Such spaces are then collapsed when the value string is read. Each name/value pair is in a separate line. A list of related name/value pairs, i.e. a list of consecutive name/value pairs describing a particular file in a CSAR, is called a block. Blocks are separated by an empty line. The first block, called block_0, is metadata about the CSAR itself. All other blocks represent metadata of files in the CSAR.
The structure of block_0 in the TOSCA meta file is as follows:
Specification:
TOSCA-Meta-File-Version: digit.digit
CSAR-Version: digit.digit
Created-By: string
Entry-Definitions: string ?Example:
TOSCA-Meta-Version: 1.0 #(block_0 mandatory)
CSAR-Version: 1.0 #(block_0 mandatory)
Created-By: Manuel Peuster (Paderborn University) #(block_0 mandatory)
Entry-Definitions: Definitions/mynsd.yaml #(block_0 mandatory)
Entry-Manifest: mynsd.mf #(block_0 optional, defined by ETSI SOL004 spec.)
Entry-Change-Log: ChangeLog.txt #(block_0 optional, default ChangeLog.txt)
Entry-Tests: Tests/entrypoint.sh #(block_0 optional)
Entry-Licenses: Licenses/ #(block_0 optional, default Licenses/)
Name: TOSCA-Metadata/NAPD.yaml #(block_1 mandatory for 5GTANGO pkg.)
Content-Type: application/vnd.5gtango.napd #(block_1 mandatory for 5GTANGO pkg.)
.
.
.- Additional
Entry-*fields: Based on ETSI SOL004 V2.3.1 Section 4.3.2, 4.3.3, 4.3.4, 4.3.5, and 4.3.6. - Name: The relative path (from package root) to the file.
- Content type: The type of the file described. This type is a MIME type complying with the type/subtype structure. Vendor defined subtypes SHOULD start as usual with the string
vnd..
A valid 5GTANGO package MUST have a block_1 that points to the location of the NFV advanced package descriptor (NAPD), as shown in the example. The NAPD is used to implement advanced and novel features for 5GTANGO packages and will be described in the following sections in more detail. As a fallback, the packager will always check for TOSCA-Metadata/NAPD.yaml even if block_1 is not present.
This manifest file is required by ETSI SOL004 V2.3.1. And contains all the additional information required to distinguish a NFV/VNF package from a normal CSAR package.
Note by Paderborn University:
The design of this file is not very nice, e.g., it should be named like the entry descriptor, whereas a fixed name in the Definitions folder would make more sense. It would be also much nicer to have this as a YAML file instead of using block/line based field separations. However, we stick to the ETSI standard here and build our package design arround the existing shortcomings, with the hope that the ETSI standard improves over time.
Quote from ETSI SOL004 V2.3.1 Section 4.3.2:
A CSAR VNF package shall have a manifest file. The manifest file shall have an extension .mf and the same name as the main TOSCA definitions YAML file and be located at the root of the archive (archive without TOSCA-Metadata directory) or in the location specified by the TOSCA.meta file (archive with a TOSCA-Metadata directory). In the latter case, the corresponding entry shall be named "Entry-Manifest". The manifest file shall start with the VNF package metadata in the form of a name-value pairs. Each pair shall appear on a different line. The "name" and the "value" shall be separated by a colon.
The manifest files MAY contain digests of individual files contained in the package. If the manifest file does not include digests, the complete CSAR file shall be digitally signed by the VNF provider. A consumer of the VNF package verifies the digests in the manifest file by computing the actual digests and comparing them with the digests listed in the manifest file.
Quote from ETSI SOL004 V2.3.1 Section 5.2 and Section 5.3:
When the manifest file provides the VNF package integrity assurance (option 1 in clause 5.1) it contains the digests (hashes) for each individual file locally stored within the VNF package or referenced from it. Each file related entry of the manifest file includes the path or URI of the individual file, the hash algorithm and the generated digest. A consumer of the VNF package shall verify the digests in the manifest file by computing the actual digests and comparing them with the digests listed in the manifest file.
When the Manifest file provides the integrity assurance of the VNF package (option 1 in clause 5.1) it shall contain a list of blocks of name-value pairs, where each block is related to one file in the VNF package, where name and value are separated by a colon. Each block shall contain the following three name-value pair attributes: Source, Algorithm, and Hash.
Example:
vnf_product_name: vIDS-1-0-0 #(= 5GTANGO name, UTF-8 string)
vnf_provider_id: UPB #(= 5GTANGO vendor, UTF-8 string)
vnf_package_version: 1.0 #(= 5GTANGO version, digits separated by dots)
vnf_release_date_time: 2017.01.01T10:00+03:00 #(IETF RFC 3339)
Source: Definitions/mynsd.yaml
Algorithm: SHA-256
Hash: d0e7828293355a07c2dccaaa765c80b507e60e6167067c950dc2e6b0da0dbd8b
Source: scripts/cloud_init/cloud_init.cfg
Algorithm: SHA-256
Hash: f0e7828293355a07c2dccaaa765c80b507e60e6167067c950dc2e6b0da0dbd8bETSI SOL004 V2.3.1 does not yet support NS and test packages, which are required by 5GTANGO. Because of this, we define the following fields for manifest files of other package types:
Network service package:
ns_product_name: myNS #(= 5GTANGO name, UTF-8 string)
ns_provider_id: UPB #(= 5GTANGO vendor, UTF-8 string)
ns_package_version: 1.0 #(= 5GTANGO version, digits separated by dots)
ns_release_date_time: 2017.01.01T10:00+03:00 #(IETF RFC 3339)Test package:
test_product_name: firewall-test #(= 5GTANGO name, UTF-8 string)
test_provider_id: UPB #(= 5GTANGO vendor, UTF-8 string)
test_package_version: 1.0 #(= 5GTANGO version, digits separated by dots)
test_release_date_time: 2017.01.01T10:00+03:00 #(IETF RFC 3339)Even though we designed this package format as compatible as possible to the existing ETSI work, the 5GTANGO project still aims to introduce new features and innovations in the package format and NFV packaging concept. To do so, we add an additional descriptor file (NAPD.yaml) to 5GTANGO packages which contains all additional information that is required to implement novel package features that go beyond the CSAR or ETSI standard, e.g., tagging of artifacts. The NAPD.yaml can be viewed as the successor of the SONATA-NFV package descriptor file. This concept, of extending the standard compliant manifest files by an additional package descriptor file is also used by other projects, like OpenBaton and ONAP to overcome the shortcomings of the current standards.
We should propose concepts implemented in this file to ETSI and try to merge them into ETSI's manifest file specification, described in the last section. This includes that we propose to use YAML for the ETSI manifest instead of the currently used line- and block-based approach.
This file contains ALL information of the ETSI manifest file plus additional 5GTANGO specific information. With this it can simply replace the ETSI manifest file later on.
The introduced information redundancy is not nice, but required to be ETSI compatible. Anyhow, the required files are automatically generated by the packaging tool and do not introduce additional work for the NS or VNF developer.
The NAPD.yaml file uses YAML and its schema is defined in the tng-schema GitHub repository, just like any other descriptor of the 5GTANGO project.
Minimal example:
---
descriptor_schema: "https://raw.githubusercontent.com/sonata-nfv/tng-schema/master/package-specification/napd-schema.yml"
vendor: "eu.5gtango"
name: "example-package"
version: "0.1.1"
package_type: "application/vnd.5gtango.package.nsp" # MIME type of package, e.g., nsp, vnsp, tdp, trp
maintainer: "Manuel Peuster, Paderborn University"
release_date_time: "2017.01.01T10:00+03:00" # IETF RFC3339Complex example:
---
descriptor_schema: "https://raw.githubusercontent.com/sonata-nfv/tng-schema/master/package-specification/napd-schema.yml"
vendor: "eu.5gtango"
name: "example-package"
version: "0.1.1"
package_type: "application/vnd.5gtango.package.nsp" # MIME type of package, e.g., nsp, vnfp, tdp, trp
maintainer: "Manuel Peuster, Paderborn University"
release_date_time: "2017.01.01T10:00+03:00" # IETF RFC3339
description: "This is an example 5GTANGO package." # (optional)
logo: "icons/upb_logo.png" # (optional) path to logo file (PNG or JPEG)
package_content:
- source: "Definitions/mynsd.yaml"
algorithm: "SHA-256"
hash: "d0e7828293355a07c2dccaaa765c80b507e60e6167067c950dc2e6b0da0dbd8b"
content-type: "application/vnd.5gtango.nsd" # MIME type of file
- source: "scripts/scale/scale.sh"
algorithm: "SHA-256"
hash: "e0e7828293355a07c2dccaaa765c80b507e60e6167067c950dc2e6b0da0dbd8b"
content-type: "text/x-shellscript" # MIME type of file
tags: # (optional) list
- "eu.5gtango"
- "etsi.osm"
- source: "scripts/cloud_init/cloud_init.cfg"
algorithm: "SHA-256"
hash: "f0e7828293355a07c2dccaaa765c80b507e60e6167067c950dc2e6b0da0dbd8b"
content-type: "application/yaml" # MIME type of file
tags: # (optional) list
- "etsi.osm"
- source: "images/ubuntu_xenial.ref" # refernce to external file (next section)
algorithm: "SHA-256"
hash: "x0e7828293355a07c2dccaaa765c80b507e60e6167067c950dc2e6b0da0dbd8b"
content-type: "application/vnd.5gtango.ref" # MIME type of file
.
.
.Tags are entirely optional and will help to filter files for different target platforms (layering feature that is described at the beginning).
5GTANGO packages allow to reference arbitrary external files, e.g., disk images, to support slim packages. Instead of putting these references into the manifest files or the NAPD.yaml, we use symbolic link files in the package. Those files MUST have the name <filename>.ref and contain YAML formatted information. The advantage of this approach is that these files allow us to specify arbitrary references, e.g., to a webserver, a GIT repository or SFTP server without breaking compatibility to the ETSI/CSAR standards. In addition to this, this approach allows us to define additional metadata for a referenced file.
Note: The packaging tool will act as an resolver when packages are unpacked. It will have a plugin interface to implement a set of resolver plugins for different kinds of references (filesystem and HTTP downloads will be supported out-of-the-box).
Example: HTTP reference to a disk image (ubuntu_xenial.ref)
---
descriptor_schema: "https://raw.githubusercontent.com/sonata-nfv/tng-schema/master/package-specification/ref-schema.yml"
name: "ubuntu"
vendor: "Cononical"
version: "16.04"
proto: "http"
source: "http://ftp.halifax.rwth-aachen.de/ubuntu-releases/16.04.3/ubuntu-16.04.3-server-amd64.iso"
algorithm: "SHA-256"
hash: "x0e7828293355a07c2dccaaa765c80b507e60e6167067c950dc2e6b0da0dbd8b"
username: "username" # (optional)
password: "password" # (optional)Example: Reference to another 5GTANGO package that is stored in a 5GTANGO catalog (de.upb.my-vnf-package.2.3.tng)
---
descriptor_schema: "https://raw.githubusercontent.com/sonata-nfv/tng-schema/master/package-specification/ref-schema.yml"
name: "my-vnf-package"
vendor: "de.upb"
version: "2.3"
proto: "5gtango.catalog"
source: "http://catalog.5gtango-url.eu"
algorithm: "SHA-256"
hash: "z0e7828293355a07c2dccaaa765c80b507e60e6167067c950dc2e6b0da0dbd8b"
access_token: "234kjh2348dsaf9" # (optional)Further details about the referencing concept are to be developed during the implementation of the packaging tool.
The following list defines the 5GTANGO-specific MIME types.
| Name | MIME |
|---|---|
| 5GTANGO NFV advanced package descriptor (NAPD) | application/vnd.5gtango.napd |
| 5GTANGO network service package | application/vnd.5gtango.package.nsp |
| 5GTANGO network function package | application/vnd.5gtango.package.vnfp |
| 5GTANGO test definition package | application/vnd.5gtango.package.tdp |
| 5GTANGO test result package | application/vnd.5gtango.package.trp |
| 5GTANGO reference file | application/vnd.5gtango.ref |
| 5GTANGO network service descriptor | application/vnd.5gtango.nsd |
| 5GTANGO virtual network function descriptor | application/vnd.5gtango.vnfd |
| 5GTANGO test descriptor | application/vnd.5gtango.tstd |
| ... to be continued ... | application/vnd.5gtango.XXX |
They follow the convention to prefix vendor defined subtypes with vnd. (see here).
The following list defines possible tags to create package layers. It is just a convention. Users can define arbitrary tags.
| Name | Tag |
|---|---|
| 5GTANGO specific files | eu.5gtango |
| SONATA-NFV specific files | eu.sonata |
| OSM specific files | etsi.osm |
| ONAP specific files | lf.onap |
| OpenBaton specific files | fokus.openbaton |
TODO Manuel: The packaging tool is work in Progress and will be developed in the next months.
- TOSCA v1.0 (CSAR in Chapter 16)
- TOSCA YAML (CSAR in Chapter 6)
- ETSI SOL004 specification: VNF Packages
- ETSI SOL004 presentation
- OSM package format discussion for rel. 4
- OpenBaton TOSCA on-boarding
- ONAP package format
- Missing a feature: Write a GitHub issue
- Want to propose a change: Write a GitHub issue
- Want do discuss anything: Write a GitHub issue
The 5GTANGO package specification is developed under 5GTANGO WP4 within task T4.1.
Task leader: Paderborn University Contact persons:
- Manuel Peuster [email protected]
- Stefan Schneider [email protected]