flyte nlp workflow

Author: BerndDoser

_quarto.yml

@@ -20,5 +20,5 @@ format:
     footer: "ML Workflow Orchestration (Bernd Doser, HITS)"
     slide-number: true
     highlight-style: a11y
-    code-block-height: 650px
+    # code-block-height: 650px
     # width: 1300

code/streamflow_slurm.yaml

@@ -1,5 +1,5 @@
 workflows:
-  (same as left)
+  # same as left
 
 deployments:
   ssh-cascade:
@@ -16,4 +16,4 @@ deployments:
           partition: cascade.p
           nodes: 1
           mem: 1gb
-    wraps: ssh-cascade
+    wraps: ssh-cascade

flyte.qmd

@@ -49,7 +49,25 @@ def mean(values: List[float]) -> float:
 
 ## Flyte Workflows
 
-todo ...
+- Workflows doesn’t perform computations it’s used to structure tasks 
+
+```python
+workflow_outputs = typing.NamedTuple(
+    "WorkflowOutputs",
+    simwords=Dict[str, float],
+    distance=float,
+    topics=Dict[int, List[str]],
+)
+
+@workflow
+def nlp_workflow(target_word: str = "computer") -> workflow_outputs:
+    corpus = generate_processed_corpus()
+    model_wv = train_word2vec_model(training_data=corpus, hyperparams=Word2VecModelHyperparams())
+    lda_topics = train_lda_model(corpus=corpus, hyperparams=LDAModelHyperparams())
+    similar_words = word_similarities(model_ser=model_wv.model, word=target_word)
+    distance = word_movers_distance(model_ser=model_wv.model)
+    return similar_words, distance, lda_topics
+```
 
 
 ## Containerized Tasks