Skip to content

Commit

Permalink
docs: Update references (#508)
Browse files Browse the repository at this point in the history
* Initial commit

* Update xref
  • Loading branch information
Techassi authored and fhennig committed Sep 21, 2023
1 parent 8a8dab8 commit e07f17d
Show file tree
Hide file tree
Showing 2 changed files with 32 additions and 16 deletions.
21 changes: 12 additions & 9 deletions docs/modules/nifi/pages/getting_started/installation.adoc
Original file line number Diff line number Diff line change
@@ -1,21 +1,23 @@
= Installation

On this page you will install the Stackable Operator for Apache NiFi and operators for its dependencies - ZooKeeper - as well as the commons and secret operator which are required by all Stackable Operators.
On this page you will install the Stackable Operator for Apache NiFi and operators for its dependencies - ZooKeeper -
as well as the commons and secret operator which are required by all Stackable Operators.

== Stackable Operators

There are two ways to install Stackable Operators:

1. Using xref:stackablectl::index.adoc[stackablectl]

2. Using Helm
. Using xref:management:stackablectl:index.adoc[stackablectl]
. Using Helm

=== stackablectl

The stackablectl command line tool is the recommended way to interact with operators and dependencies. Follow the xref:stackablectl::installation.adoc[installation steps] for your platform if you choose to work with stackablectl.
The `stackablectl` command line tool is the recommended way to interact with operators and dependencies. Follow the
xref:management:stackablectl:installation.adoc[installation steps] for your platform if you choose to work with
`stackablectl`.

After you have installed `stackablectl` and have a Kubernetes cluster up and
running, run the following command to install all operators necessary for NiFi:
After you have installed `stackablectl` and have a Kubernetes cluster up and running, run the following command to
install all operators necessary for NiFi:

[source,bash]
----
Expand All @@ -29,7 +31,7 @@ The output should contain the following lines
include::example$getting_started/install-operator-output.txt[tag=stackablectl-install-operators-output]
----

TIP: Consult the xref:stackablectl::quickstart.adoc[] to learn more about how to use `stackablectl`.
TIP: Consult the xref:management:stackablectl:quickstart.adoc[] to learn more about how to use `stackablectl`.

=== Helm

Expand All @@ -47,7 +49,8 @@ Then install the Stackable Operators:
include::example$getting_started/getting_started.sh[tag=helm-install-operators]
----

Helm will deploy the operators in a Kubernetes Deployment and apply the CRDs for the Apache NiFi service (as well as the CRDs for the required operators). You are now ready to deploy Apache NiFi in Kubernetes.
Helm will deploy the operators in a Kubernetes Deployment and apply the CRDs for the Apache NiFi service (as well as the
CRDs for the required operators). You are now ready to deploy Apache NiFi in Kubernetes.

== What's next

Expand Down
27 changes: 20 additions & 7 deletions docs/modules/nifi/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,30 +2,43 @@
:description: The Stackable Operator for Apache NiFi is a Kubernetes operator that can manage Apache NiFi clusters. Learn about its features, resources, dependencies and demos, and see the list of supported NiFi versions.
:keywords: k8s, Kubernetes, Stackable Operator, Apache NiFi, open source, operator, data science, data exploration, big data

This Operator manages https://nifi.apache.org/[Apache NiFi] clusters on Kubernetes.
Apache NiFi is an open-source data integration tool that provides a web-based interface for designing, monitoring and managing data flows between various systems and devices, using a visual programming approach. It supports a wide range of data sources, formats and features such as data provenance, security and clustering.
This Operator manages https://nifi.apache.org/[Apache NiFi] clusters on Kubernetes. Apache NiFi is an open-source data
integration tool that provides a web-based interface for designing, monitoring and managing data flows between various
systems and devices, using a visual programming approach. It supports a wide range of data sources, formats and features
such as data provenance, security and clustering.

== Getting started

Get started with Apache NiFi and the Stackable Operator by following the xref:getting_started/index.adoc[] guide. It will guide you through the xref:getting_started/installation.adoc[installation] process and xref:getting_started/first_steps.adoc[connect] to the NiFi web interface. Afterwards have a look at the xref:usage_guide/index.adoc[] to learn how to configure your NiFi instance to your needs or run some <<demos, demos>> to learn more about using NiFi with other components.
Get started with Apache NiFi and the Stackable Operator by following the xref:getting_started/index.adoc[] guide. It
will guide you through the xref:getting_started/installation.adoc[installation] process and
xref:getting_started/first_steps.adoc[connect] to the NiFi web interface. Afterwards have a look at the
xref:usage_guide/index.adoc[] to learn how to configure your NiFi instance to your needs or run some <<demos, demos>> to
learn more about using NiFi with other components.

== Operator Model

The Operator manages the _NifiCluster_ custom resource. NiFi only has a single process that it needs to run, so the NifiCluster has only a single xref:concepts:roles-and-role-groups.adoc[role]: `node`. This role can be divided in multiple role groups.
The Operator manages the _NifiCluster_ custom resource. NiFi only has a single process that it needs to run, so the
NifiCluster has only a single xref:concepts:roles-and-role-groups.adoc[role]: `node`. This role can be divided in
multiple role groups.

image::nifi_overview.drawio.svg[A diagram depicting the Kubernetes resources created by the Stackable Operator for Apache NiFi]

For every role group the Operator creates a ConfigMap and StatefulSet which can have multiple replicas (Pods). Every role group is accessible through it's own Service, and there is a Service for the whole Cluster.
For every role group the Operator creates a ConfigMap and StatefulSet which can have multiple replicas (Pods). Every
role group is accessible through it's own Service, and there is a Service for the whole Cluster.

== Dependencies

Apache NiFi depends on Apache ZooKeeper which you can run in Kubernetes with the xref:zookeeper:index.adoc[].

== [[demos]]Demos

NiFi is often a good choice as a first step in a data pipeline when it comes to fetching the data in various formats from various sources. The xref:stackablectl::demos/data-lakehouse-iceberg-trino-spark.adoc[] demo uses NiFi to fetch six different datasets in various formats. The data is then ingested into a Kafka topic. Apache Kafka is also xref:kafka:index.adoc[part of the Stackable platform].
NiFi is often a good choice as a first step in a data pipeline when it comes to fetching the data in various formats
from various sources. The xref:demos:data-lakehouse-iceberg-trino-spark.adoc[] demo uses NiFi to fetch six different
datasets in various formats. The data is then ingested into a Kafka topic. Apache Kafka is also
xref:kafka:index.adoc[part of the Stackable platform].

The xref:stackablectl::demos/nifi-kafka-druid-earthquake-data.adoc[] and xref:stackablectl::demos/nifi-kafka-druid-water-level-data.adoc[] demo use NiFi in the same way, both demos showcase downloading data from web APIs and ingesting it into Kafka.
The xref:demos:nifi-kafka-druid-earthquake-data.adoc[] and xref:demos:nifi-kafka-druid-water-level-data.adoc[] demo use
NiFi in the same way, both demos showcase downloading data from web APIs and ingesting it into Kafka.

== Supported Versions

Expand Down

0 comments on commit e07f17d

Please sign in to comment.