Skip to content

xhuljamel/tutorials.Getting-Started

BranchesTags
 
 

Repository files navigation

FIWARE Banner NGSI v2

FIWARE Core Context Management License: MIT Support badge
Documentation

This is an Introductory Tutorial to the FIWARE Platform. We will start with the data from a supermarket chain’s store finder and create a very simple “Powered by FIWARE” application by passing in the address and location of each store as context data to the FIWARE context broker.

The tutorial uses cUrl commands throughout, but is also available as Postman documentation

Run in Postman

🇯🇵 このチュートリアルは日本語でもご覧い ただけます。
🇵🇹 Este tutorial também está disponível em português
🇪🇸 Este tutorial también está disponible en español

Contents

Details

Architecture

Our demo application will only make use of one FIWARE component - the Orion Context Broker. Usage of the Orion Context Broker is sufficient for an application to qualify as “Powered by FIWARE”.

Currently, the Orion Context Broker relies on open source MongoDB technology to keep persistence of the context data it holds. Therefore, the architecture will consist of two elements:

  • The Orion Context Broker which will receive requests using NGSI-v2
  • The underlying MongoDB database :
    • Used by the Orion Context Broker to hold context data information such as data entities, subscriptions and registrations

Since all interactions between the two elements are initiated by HTTP requests, the entities can be containerized and run from exposed ports.

Prerequisites

Docker

To keep things simple both components will be run using Docker. Docker is a container technology which allows to different components isolated into their respective environments.

  • To install Docker on Windows follow the instructions here
  • To install Docker on Mac follow the instructions here
  • To install Docker on Linux follow the instructions here

Docker Compose (Optional)

Docker Compose is a tool for defining and running multi-container Docker applications. A YAML file is used configure the required services for the application. This means all container services can be brought up in a single command. Docker Compose is installed by default as part of Docker for Windows and Docker for Mac, however Linux users will need to follow the instructions found here

You can check your current Docker and Docker Compose versions using the following commands:

docker-compose -v
docker version

Please ensure that you are using Docker version 20.10 or higher and Docker Compose 1.29 or higher and upgrade if necessary.

Starting the containers

Option 1) Using Docker commands directly

First pull the necessary Docker images from Docker Hub and create a network for our containers to connect to:

docker pull mongo:4.4
docker pull fiware/orion
docker network create fiware_default

A Docker container running a MongoDB database can be started and connected to the network with the following command:

docker run -d --name=mongo-db --network=fiware_default \
  --expose=27017 mongo:4.4 --bind_ip_all

The Orion Context Broker can be started and connected to the network with the following command:

docker run -d --name fiware-orion -h orion --network=fiware_default \
  -p 1026:1026  fiware/orion -dbhost mongo-db

Note: If you want to clean up and start again you can do so with the following commands

docker stop fiware-orion
docker rm fiware-orion
docker stop mongo-db
docker rm mongo-db
docker network rm fiware_default

Option 2) Using Docker Compose

All services can be initialised from the command-line using the docker-compose command or the newer docker compose command (without the hyphen - )found in Compose V2. Please clone the repository and create the necessary images by running the commands as shown:

git clone https://github.com/FIWARE/tutorials.Getting-Started.git
cd tutorials.Getting-Started
git checkout NGSI-v2

export $(cat .env | grep "#" -v)
docker compose -p fiware up -d

Note: If you want to clean up and start again you can do so with the following command:

docker compose -p fiware down

Creating your first "Powered by FIWARE" app

Checking the service health

You can check if the Orion Context Broker is running by making an HTTP request to the exposed port:

1️⃣ Request:

curl -X GET \
  'http://localhost:1026/version'

Response:

The response will look similar to the following:

{
    "orion": {
        "version": "3.0.0",
        "uptime": "0 d, 0 h, 17 m, 19 s",
        "git_hash": "d6f8f4c6c766a9093527027f0a4b3f906e7f04c4",
        "compile_time": "Mon Apr 12 14:48:44 UTC 2021",
        "compiled_by": "root",
        "compiled_in": "f307ca0746f5",
        "release_date": "Mon Apr 12 14:48:44 UTC 2021",
        "machine": "x86_64",
        "doc": "https://fiware-orion.rtfd.io/en/3.0.0/",
        "libversions": {
            "boost": "1_66",
            "libcurl": "libcurl/7.61.1 OpenSSL/1.1.1g zlib/1.2.11 nghttp2/1.33.0",
            "libmicrohttpd": "0.9.70",
            "openssl": "1.1",
            "rapidjson": "1.1.0",
            "mongoc": "1.17.4",
            "bson": "1.17.4"
        }
    }
}

What if I get a Failed to connect to localhost port 1026: Connection refused Response?

If you get a Connection refused response, the Orion Content Broker cannot be found where expected for this tutorial - you will need to substitute the URL and port in each cUrl command with the corrected IP address. All the cUrl commands tutorial assume that orion is available on localhost:1026.

Try the following remedies:

  • To check that the docker containers are running try the following:
docker ps

You should see two containers running. If orion is not running, you can restart the containers as necessary. This command will also display open port information.

  • If you have installed docker-machine and Virtual Box, the orion docker container may be running from another IP address - you will need to retrieve the virtual host IP as shown:
curl -X GET \
 'http://$(docker-machine ip default):1026/version'

Alternatively run all your cUrl commands from within the container network:

docker run --network fiware_default --rm appropriate/curl -s \
 -X GET 'http://orion:1026/version'

Creating Context Data

At its heart, FIWARE is a system for managing context information, so lets add some context data into the system by creating two new entities (stores in Berlin). Any entity must have a id and type attributes, additional attributes are optional and will depend on the system being described. Each additional attribute should also have a defined type and a value attribute.

2️⃣ Request:

curl -iX POST \
  'http://localhost:1026/v2/entities' \
  -H 'Content-Type: application/json' \
  -d '
{
    "id": "urn:ngsi-ld:Store:001",
    "type": "Store",
    "address": {
        "type": "PostalAddress",
        "value": {
            "streetAddress": "Bornholmer Straße 65",
            "addressRegion": "Berlin",
            "addressLocality": "Prenzlauer Berg",
            "postalCode": "10439"
        },
        "metadata": {
            "verified": {
                "value": true,
                "type": "Boolean"
            }
        }
    },
    "location": {
        "type": "geo:json",
        "value": {
             "type": "Point",
             "coordinates": [13.3986, 52.5547]
        }
    },
    "name": {
        "type": "Text",
        "value": "Bösebrücke Einkauf"
    }
}'

3️⃣ Request:

Each subsequent entity must have a unique id for the given type

curl -iX POST \
  'http://localhost:1026/v2/entities' \
  -H 'Content-Type: application/json' \
  -d '
{
    "type": "Store",
    "id": "urn:ngsi-ld:Store:002",
    "address": {
        "type": "PostalAddress",
        "value": {
            "streetAddress": "Friedrichstraße 44",
            "addressRegion": "Berlin",
            "addressLocality": "Kreuzberg",
            "postalCode": "10969"
        },
        "metadata": {
            "verified": {
                "value": true,
                "type": "Boolean"
            }
        }
    },
    "location": {
        "type": "geo:json",
        "value": {
             "type": "Point",
             "coordinates": [13.3903, 52.5075]
        }
    },
    "name": {
        "type": "Text",
        "value": "Checkpoint Markt"
    }
}'

Data Model Guidelines

Although the each data entity within your context will vary according to your use case, the common structure within each data entity should be standardized order to promote reuse. The full Smart Data model guidelines can be found here. This tutorial demonstrates the usage of the following recommendations:

All terms are defined in American English

Although the value fields of the context data may be in any language, all attributes and types are written using the English language.

Entity type names must start with a Capital letter

In this case we only have one entity type - Store

Entity IDs should be a URN following NGSI-LD guidelines

NGSI-LD has recently been published as a full ETSI specification, the proposal is that each id is a URN follows a standard format: urn:ngsi-ld:<entity-type>:<entity-id>. This will mean that every id in the system will be unique

Data type names should reuse schema.org data types where possible

Schema.org is an initiative to create common structured data schemas. In order to promote reuse we have deliberately used the Text and PostalAddress type names within our Store entity. Other existing standards such as Open311 (for civic issue tracking) or Datex II (for transport systems) can also be used, but the point is to check for the existence of the same attribute on existing data models and reuse it.

Use camel case syntax for attribute names

The streetAddress, addressRegion, addressLocality and postalCode are all examples of attributes using camel casing

Location information should be defined using address and location attributes

  • We have used an address attribute for civic locations as per schema.org
  • We have used a location attribute for geographical coordinates.

Use GeoJSON for codifying geospatial properties

GeoJSON is an open standard format designed for representing simple geographical features. The location attribute has been encoded as a geoJSON Point location.

Attribute Metadata

Metadata is "data about data", it is additionl data used to describe properties of the attribute value itself like accuracy, provider, or a timestamp. Several built-in metadata attribute already exist and these names are reserved

  • dateCreated (type: DateTime): attribute creation date as an ISO 8601 string.
  • dateModified (type: DateTime): attribute modification date as an ISO 8601 string.
  • previousValue (type: any): only in notifications. The value of this
  • actionType (type: Text): only in notifications.

One element of metadata can be found within the address attribute. a verified flag indicates whether the address has been confirmed.

Querying Context Data

A consuming application can now request context data by making HTTP requests to the Orion Context Broker. The existing NGSI interface enables us to make complex queries and filter results.

At the moment, for the store finder demo all the context data is being added directly via HTTP requests, however in a more complex smart solution, the Orion Context Broker will also retrieve context directly from attached sensors associated to each entity.

Here are a few examples, in each case the options=keyValues query parameter has been used shorten the responses by stripping out the type elements from each attribute

Obtain entity data by ID

This example returns the data of urn:ngsi-ld:Store:001

4️⃣ Request:

curl -G -X GET \
   'http://localhost:1026/v2/entities/urn:ngsi-ld:Store:001' \
   -d 'options=keyValues'

Response:

Because of the use of the options=keyValues, the response consists of JSON only without the attribute type and metadata elements.

{
    "id": "urn:ngsi-ld:Store:001",
    "type": "Store",
    "address": {
        "streetAddress": "Bornholmer Straße 65",
        "addressRegion": "Berlin",
        "addressLocality": "Prenzlauer Berg",
        "postalCode": "10439"
    },
    "location": {
        "type": "Point",
        "coordinates": [13.3986, 52.5547]
    },
    "name": "Bösebrücke Einkauf"
}

Obtain entity data by type

This example returns the data of all Store entities within the context data The type parameter limits the response to store entities only.

5️⃣ Request:

curl -G -X GET \
    'http://localhost:1026/v2/entities' \
    -d 'type=Store' \
    -d 'options=keyValues'

Response:

Because of the use of the options=keyValues, the response consists of JSON only without the attribute type and metadata elements.

[
    {
        "id": "urn:ngsi-ld:Store:001",
        "type": "Store",
        "address": {
            "streetAddress": "Bornholmer Straße 65",
            "addressRegion": "Berlin",
            "addressLocality": "Prenzlauer Berg",
            "postalCode": "10439"
        },
        "location": {
            "type": "Point",
            "coordinates": [13.3986, 52.5547]
        },
        "name": "Bose Brucke Einkauf"
    },
    {
        "id": "urn:ngsi-ld:Store:002",
        "type": "Store",
        "address": {
            "streetAddress": "Friedrichstraße 44",
            "addressRegion": "Berlin",
            "addressLocality": "Kreuzberg",
            "postalCode": "10969"
        },
        "location": {
            "type": "Point",
            "coordinates": [13.3903, 52.5075]
        },
        "name": "Checkpoint Markt"
    }
]

Filter context data by comparing the values of an attribute

This example returns all stores with the name attribute Checkpoint Markt. Filtering can be done using the q parameter - if a string has spaces in it, it can be URL encoded and held within single quote characters ' = %27

6️⃣ Request:

curl -G -X GET \
    'http://localhost:1026/v2/entities' \
    -d 'type=Store' \
    -d 'q=name==%27Checkpoint%20Markt%27' \
    -d 'options=keyValues'

Response:

Because of the use of the options=keyValues, the response consists of JSON only without the attribute type and metadata elements.

[
    {
        "id": "urn:ngsi-ld:Store:002",
        "type": "Store",
        "address": {
            "streetAddress": "Friedrichstraße 44",
            "addressRegion": "Berlin",
            "addressLocality": "Kreuzberg",
            "postalCode": "10969"
        },
        "location": {
            "type": "Point",
            "coordinates": [13.3903, 52.5075]
        },
        "name": "Checkpoint Markt"
    }
]

Filter context data by comparing the values of a sub-attribute

This example returns all stores found in the Kreuzberg District.

Filtering can be done using the q parameter - sub-attributes are annotated using the dot syntax e.g. address.addressLocality

7️⃣ Request:

curl -G -X GET \
    'http://localhost:1026/v2/entities' \
    -d 'type=Store' \
    -d 'q=address.addressLocality==Kreuzberg' \
    -d 'options=keyValues'

Response:

Because of the use of the options=keyValues, the response consists of JSON only without the attribute type and metadata elements.

[
    {
        "id": "urn:ngsi-ld:Store:002",
        "type": "Store",
        "address": {
            "streetAddress": "Friedrichstraße 44",
            "addressRegion": "Berlin",
            "addressLocality": "Kreuzberg",
            "postalCode": "10969"
        },
        "location": {
            "type": "Point",
            "coordinates": [13.3903, 52.5075]
        },
        "name": "Checkpoint Markt"
    }
]

Filter context data by querying metadata

This example returns the data of all Store entities with a verified address.

Metadata queries can be made using the mq parameter.

8️⃣ Request:

curl -G -X GET \
    'http://localhost:1026/v2/entities' \
    -d 'type=Store' \
    -d 'mq=address.verified==true' \
    -d 'options=keyValues'

Response:

Because of the use of the options=keyValues, the response consists of JSON only without the attribute type and metadata elements.

[
    {
        "id": "urn:ngsi-ld:Store:001",
        "type": "Store",
        "address": {
            "streetAddress": "Bornholmer Straße 65",
            "addressRegion": "Berlin",
            "addressLocality": "Prenzlauer Berg",
            "postalCode": "10439"
        },
        "location": {
            "type": "Point",
            "coordinates": [13.3986, 52.5547]
        },
        "name": "Bösebrücke Einkauf"
    },
    {
        "id": "urn:ngsi-ld:Store:002",
        "type": "Store",
        "address": {
            "streetAddress": "Friedrichstraße 44",
            "addressRegion": "Berlin",
            "addressLocality": "Kreuzberg",
            "postalCode": "10969"
        },
        "location": {
            "type": "Point",
            "coordinates": [13.3903, 52.5075]
        },
        "name": "Checkpoint Markt"
    }
]

Filter context data by comparing the values of a geo:json attribute

This example return all Stores within 1.5km the Brandenburg Gate in Berlin (52.5162N 13.3777W)

9️⃣ Request:

curl -G -X GET \
  'http://localhost:1026/v2/entities' \
  -d 'type=Store' \
  -d 'georel=near;maxDistance:1500' \
  -d 'geometry=point' \
  -d 'coords=52.5162,13.3777' \
  -d 'options=keyValues'

Response:

Because of the use of the options=keyValues, the response consists of JSON only without the attribute type and metadata elements.

[
    {
        "id": "urn:ngsi-ld:Store:002",
        "type": "Store",
        "address": {
            "streetAddress": "Friedrichstraße 44",
            "addressRegion": "Berlin",
            "addressLocality": "Kreuzberg",
            "postalCode": "10969"
        },
        "location": {
            "type": "Point",
            "coordinates": [13.3903, 52.5075]
        },
        "name": "Checkpoint Markt"
    }
]

Next Steps

Want to learn how to add more complexity to your application by adding advanced features? You can find out by reading the other tutorials in this series:

  101. Getting Started
  102. Entity Relationships
  103. CRUD Operations
  104. Context Providers
  105. Altering the Context Programmatically
  106. Subscribing to Changes in Context

  201. Introduction to IoT Sensors
  202. Provisioning an IoT Agent
  203. IoT over MQTT
  204. Using an alternative IoT Agent
  205. Creating a Custom IoT Agent
  250. Introduction to Fast-RTPS and Micro-RTPS

  301. Persisting Context Data using Apache Flume (MongoDB, MySQL, PostgreSQL)
  302. Persisting Context Data using Apache NIFI (MongoDB, MySQL, PostgreSQL)
  303. Querying Time Series Data (MongoDB)
  304. Querying Time Series Data (CrateDB)
  305. Big Data Analysis (Flink)

  401. Managing Users and Organizations
  402. Roles and Permissions
  403. Securing Application Access
  404. Securing Microservices with a PEP Proxy
  405. XACML Rules-based Permissions
  406. Administrating XACML via a PAP

  501. Creating Application Mashups
  503. Introduction to Media Streams
  507. Cloud-Edge Computing

  601. Introduction to Linked Data
  602. Linked Data Relationships and Data Models
  603. Traversing Linked Data Programmatically
  604. Linked Data Subscriptions and Registrations

The full documentation can be found here.

Iterative Development

The context of the store finder demo is very simple, it could easily be expanded to hold the whole of a stock management system by passing in the current stock count of each store as context data to the Orion Context Broker.

So far, so simple, but consider how this Smart application could be iterated:

  • Real-time dashboards could be created to monitor the state of the stock across each store using a visualization component. [Wirecloud]
  • The current layout of both the warehouse and store could be passed to the context broker so the location of the stock could be displayed on a map [Wirecloud]
  • User Management components [Wilma, AuthZForce, Keyrock] could be added so that only store managers are able to change the price of items
  • A threshold alert could be raised in the warehouse as the goods are sold to ensure the shelves are not left empty [publish/subscribe function of Orion Context Broker]
  • Each generated list of items to be loaded from the warehouse could be calculated to maximize the efficiency of replenishment [Complex Event Processing - CEP]
  • A motion sensor could be added at the entrance to count the number of customers [IDAS]
  • The motion sensor could ring a bell whenever a customer enters [IDAS]
  • A series of video cameras could be added to introduce a video feed in each store [Kurento]
  • The video images could be processed to recognize where customers are standing within a store [Kurento]
  • By maintaining and processing historical data within the system, footfall and dwell time can be calculated - establishing which areas of the store attract the most interest [connection through Cygnus to Apache Nifi]
  • Patterns recognizing unusual behaviour could be used to raise an alert to avoid theft [Kurento]
  • Data on the movement of crowds would be useful for scientific research - data about the state of the store could be published externally. [extensions to CKAN]

Each iteration adds value to the solution through existing components with standard interfaces and therefore minimizes development time.

License

MIT © 2018-2021 FIWARE Foundation e.V.

About

FIWARE 101: An Introduction to the FIWARE Platform

www.fiware.org/developers/

Resources

Readme

License

MIT license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

8 tags

Packages

No packages published

Languages

  • Shell 100.0%