How to set the output of other tasks to env

[9enki]

Is there a way to use the stdout of a task as an argument for another task?
I understand that I can use sh to dynamically switch arguments, but I don't like to manage commands in sh and want to use the results of other tasks' execution

version: '3'

tasks:
  default:
    cmds:
      - echo "execute command with parameter {{.PARAMETER}}"
    vars:
      PARAMETER:
        sh: echo "Instead of writing a shell command here, I want to set the result of executing get-parameter-from-api to PARAMETER"
        # task: get-parameter-from-api <- When I wrote and executed the left without sh, PARAMETER was empty.

  get-parameter-from-api:
    cmds:
      # It is actually a process of hitting the API with curl
      - echo "Parameters I want to use"

[ghostsquad]

There's currently no built in way besides sh in vars to pass data around.

The problem with tracking the output of an entire task as you are requesting, is that there is a lot of ambiguity in terms of whether or not the output of that tasks dependencies should be included, how output should be concatenated when a task has multiple commands in cmds and what happens if that task is "up-to-date".

Is there more information you can provide as to why writing a script in sh of a var is not optimal for you?

[9enki]

Thank you for your answer. For example, the following uses Task to control sqs, an aws queue service. To operate queue that has already been created, we need the URL of the queue. Since I have task: get-queue-url to retrieve it, I do not want to write the same script again with task: delete-queue or task: receive-message sh.

  get-queue-url:
    cmds:
      - '{{.AWSCLI}} sqs list-queues | jq ".QueueUrls[]|select(contains(\"{{.QUEUE_NAME}}\"))"'

  delete-queue:
    cmds:
      - '{{.AWSCLI}} sqs delete-queue --queue-url {{.QUEUE_URL}}'
    vars:
      QUEUE_URL:
        sh: '{{.AWSCLI}} sqs list-queues | jq ".QueueUrls[]|select(contains(\"{{.QUEUE_NAME}}\"))"'

  receive-message:
    cmds:
      - '{{.AWSCLI}} sqs receive-message --queue-url {{.QUEUE_URL}}'
    vars:
      QUEUE_URL:
        sh: '{{.AWSCLI}} sqs list-queues | jq ".QueueUrls[]|select(contains(\"{{.QUEUE_NAME}}\"))"'

I understand from your response that this feature cannot be implemented until the specifications are finalized for some of the issues that are ambiguous. I will save scripts that are referenced multiple times in sh as one-liner shell scripts outside of Taskfile.yml.

Thank you very much.

[ssbarnea]

That is fixable by adopting a lazy variable lookup, like jinja2 does. Basically the expansion on templated variables should happen at the last point where they are used and the result is not to be cached.

Think about $(cmd) from bash, one that would run every time you try to use the variable, and not at the point where you define the variable.

Optionally you can adopt a syntax that would allow opt-in caching of values, as not all commands are expected to be dynamic.

[andreynering]

Lazy variable lookup is planned for v4

[ghostsquad]

That is fixable by adopting a lazy variable lookup, like jinja2 does. Basically the expansion on templated variables should happen at the last point where they are used and the result is not to be cached.

Think about $(cmd) from bash, one that would run every time you try to use the variable, and not at the point where you define the variable.

Optionally you can adopt a syntax that would allow opt-in caching of values, as not all commands are expected to be dynamic.

foo=$(cmd) does not rerun every time I use reference foo elsewhere in a script. But with that aside, the ask here is most certainly that of a lazy evaluation, as opposed to eager evaluation. There are 2 forms of laziness: a memoized function (aka lazy variable) and an unmemoized function. A simple example of this would be the difference between these:

files is a essentially function (it's run every time it's used)

alias files=ls

if [ ! -z "${print_files}" ]; then
   # files retrieved here, but the result is not memoized
   echo "conditional print: $(files)"
fi

# files are retrieved here again, could get a different result this time
echo "always print: $(files)"

files is an eagerly evaluated variable

# files are retrieved here, even if they are not needed
files="$(ls)"

if [ ! -z "${print_files}" ]; then
   echo "conditional print: ${files}"
fi

files is a memoized function (aka lazy variable)

memoized_files=""

files () {
   if [ -z "memoized_files" ]; then
      memoized_files="$(ls)"
   fi
   
   echo "${memoized_files}"
}

if [ ! -z "${print_files}" ]; then
   echo "conditional print: $(files)"
fi

# result will be the same, regardless of the number of times it is referenced
echo "always print: $(files)"

Once I realized what was going on, I realized that what's needed is "first class functions" (like in Go) that can be memoized, with explicit return values, thus why I started development of https://github.com/go-fn/fn.

One hurdle I encountered which also was a reason for starting with a clean slate is that the interface for accessing "lazy variables" changes due to limitations in the go text templating syntax in how functions can be called which is one of the 2 variants below:

invoke a method or function

{{ .Vars "foo" }}
# or
{{ Vars "foo" }}

In the example above, since the names of variables are unknown until runtime, I must declare a function that can be easily called allowing an argument (the variable name) to be passed in.

The alternative, which is somewhat similar to current syntax, but I believe has some very big user experience downsides is this:

{{ call .Foo }}

where . is actually a map, return a function (which as the key of "Foo"). The function though is not invoked automatically, and so thus you must use call.

I believe the later syntax is inferior because the value returned by .Foo can coerced into a string without error. It would just be complete nonsense (as it would be a pointer / an address in memory). Additionally, there's no way to intercept this and check to see if the code is written correctly (and return a helpful error). It would just turn into confusion and/or a bug report.

This could be introduced in Task in a backwards compatible way, but I feel it would dilute existing the API and generally would make Task more complicated and less user friendly.

[justiceamoh]

Hi. Thanks again for Taskfile -- amazing software and I now use it in all my projects.

I am in a similar situation where using lazy evaluation of variables can significantly improve performance. Consider the Taskfile below:

version: '3'

vars:
  HOST: '{{.HOST | default "test-ec2-vm"}}'
  ID:
    sh: >
      aws ec2 describe-instances --filters "Name=tag:Name,Values={{.HOST}}"
      --query "Reservations[*].Instances[*].InstanceId"
      --output text --no-cli-page
  IP:
    sh: >
      aws ec2 describe-instances --filters "Name=tag:Name,Values={{.HOST}}"
      --query "Reservations[*].Instances[*].PublicIpAddress"
      --output text --no-cli-page

tasks:
  start: aws ec2 start-instances --instance-ids --no-cli-page {{.ID}}

  stop: aws ec2 stop-instances --instance-ids --no-cli-page {{.ID}}

  demo: echo "Testing mic"

The sh variables IP and ID take a few seconds to execute, affecting even the execution of an independent task like demo. Ideally, I'd have a separate task that first retrieves ID and IP so I can use that only in the start and stop tasks. But since there's no way to return values from tasks, the only options are to keep them globally or repeat them for all the necessary tasks.

Is there any progress on the lazy evaluation of variables? I see a PR #1231 opened, but there's been no update since June.