Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement commands expansion logic #229

Open
wants to merge 24 commits into
base: master
Choose a base branch
from

Conversation

QU3B1M
Copy link
Member

@QU3B1M QU3B1M commented Jan 17, 2025

Description

Detect the commands which target type is group, search in the .agents index for the agents of each group and generate the corresponding Orders.

Renamed the Document class to Orders, replacing the previously unused Orders class.

Working validations

  • Command expansion

    Group used for the test: group000

     % curl http://127.0.0.1:9200/.agents/_search | grep "group000"
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100 15337  100 15337    0     0  1799k      0 --:--:-- --:--:-- --:--:-- 1872k
        "groups": ["group000","group001","group003"],
        "groups": ["group000","group002","group002"],
    • Request

      {
        "commands": [
          {
            "source": "Engine",
            "user": "user53",
            "target": {
              "id": "group000",
              "type": "group"
            },
            "action": {
              "name": "restart",
              "args": {
                "arg1": "/path/to/executable/arg6"
              },
              "version": "v4"
            },
            "timeout": 30
          }
        ]
      }
    • Response

      {
        "_index": ".commands",
        "_orders": [
          {
            "_id": "WS5fjpQBDZQg8-hA8AYd"
          },
          {
            "_id": "Wi5fjpQBDZQg8-hA8AYe"
          }
        ],
        "result": "OK"
      }
    • Command creation log

      [2025-01-22T11:17:19,646][INFO ][c.w.c.i.CommandIndex     ] [integTest-0] Adding command with id [WS5fjpQBDZQg8-hA8AYd] to the bulk request
      [2025-01-22T11:17:19,646][INFO ][c.w.c.i.CommandIndex     ] [integTest-0] Adding command with id [Wi5fjpQBDZQg8-hA8AYe] to the bulk request
    • Command indexed

      % curl http://127.0.0.1:9200/.commands/_search                
      {"took":2,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":2,"relation":"eq"},"max_score":1.0,"hits":[{"_index":".commands","_id":"WS5fjpQBDZQg8-hA8AYd","_score":1.0,"_source":{"agent":{"groups":["group000","group001","group003"]},"@timestamp":"2025-01-22T14:17:19Z","delivery_timestamp":"2025-01-22T14:17:49Z","command":{"action":{"args":{"arg1":"/path/to/executable/arg6"},"name":"restart","version":"v4"},"source":"Engine","user":"user53","order_id":"WC5fjpQBDZQg8-hA8AYP","request_id":"Vy5fjpQBDZQg8-hA8AYP","timeout":30,"target":{"id":"group000","type":"group"},"status":"failure"}}},{"_index":".commands","_id":"Wi5fjpQBDZQg8-hA8AYe","_score":1.0,"_source":{"agent":{"groups":["group000","group002","group002"]},"@timestamp":"2025-01-22T14:17:19Z","delivery_timestamp":"2025-01-22T14:17:49Z","command":{"action":{"args":{"arg1":"/path/to/executable/arg6"},"name":"restart","version":"v4"},"source":"Engine","user":"user53","order_id":"WC5fjpQBDZQg8-hA8AYP","request_id":"Vy5fjpQBDZQg8-hA8AYP","timeout":30,"target":{"id":"group000","type":"group"},"status":"failure"}}}]}}
  • Command expansion - Group assigned to the same agent more than once

    Using the group 001

    % curl http://127.0.0.1:9200/.agents/_search | grep "group001"
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100 15451  100 15451    0     0  2741k      0 --:--:-- --:--:-- --:--:-- 3017k
        "groups": ["group001"],
        "groups": ["group004","group004","group001","group004"],
        "groups": ["group001","group005","group000","group001"],
        "groups": ["group004","group000","group003","group001","group002"],
        "groups": ["group004","group000","group001"],
    • Request
      {
        "commands": [
          {
            "source": "Engine",
            "user": "user53",
            "target": {
              "id": "group001",
              "type": "group"
            },
            "action": {
              "name": "restart",
              "args": {
                "arg1": "/path/to/executable/arg6"
              },
              "version": "v4"
            },
            "timeout": 30
          }
        ]
      }
    • Response
      {
        "_index": ".commands",
        "_orders": [
          {
            "_id": "wP9ojpQBpOpp5jjTDc0s"
          },
          {
            "_id": "wf9ojpQBpOpp5jjTDc0t"
          },
          {
            "_id": "wv9ojpQBpOpp5jjTDc0t"
          },
          {
            "_id": "w_9ojpQBpOpp5jjTDc0t"
          },
          {
            "_id": "xP9ojpQBpOpp5jjTDc0t"
          }
        ],
        "result": "OK"
      }
    • Search on the index
      % curl http://127.0.0.1:9200/.commands/_search       
      {
          "took": 2,
          "timed_out": false,
          "_shards": {
              "total": 1,
              "successful": 1,
              "skipped": 0,
              "failed": 0
          },
          "hits": {
              "total": {
                  "value": 5,
                  "relation": "eq"
              },
              "max_score": 1.0,
              "hits": [
                  {
                      "_index": ".commands",
                      "_id": "wP9ojpQBpOpp5jjTDc0s",
                      "_score": 1.0,
                      "_source": {
                          "agent": {
                              "groups": [
                                  "group001"
                              ]
                          },
                          "@timestamp": "2025-01-22T14:26:11Z",
                          "delivery_timestamp": "2025-01-22T14:26:41Z",
                          "command": {
                              "action": {
                                  "args": {
                                      "arg1": "/path/to/executable/arg6"
                                  },
                                  "name": "restart",
                                  "version": "v4"
                              },
                              "source": "Engine",
                              "user": "user53",
                              "order_id": "v_9ojpQBpOpp5jjTDc0h",
                              "request_id": "vv9ojpQBpOpp5jjTDc0h",
                              "timeout": 30,
                              "target": {
                                  "id": "group001",
                                  "type": "group"
                              },
                              "status": "failure"
                          }
                      }
                  },
                  {
                      "_index": ".commands",
                      "_id": "wf9ojpQBpOpp5jjTDc0t",
                      "_score": 1.0,
                      "_source": {
                          "agent": {
                              "groups": [
                                  "group001",
                                  "group005",
                                  "group000",
                                  "group001"
                              ]
                          },
                          "@timestamp": "2025-01-22T14:26:11Z",
                          "delivery_timestamp": "2025-01-22T14:26:41Z",
                          "command": {
                              "action": {
                                  "args": {
                                      "arg1": "/path/to/executable/arg6"
                                  },
                                  "name": "restart",
                                  "version": "v4"
                              },
                              "source": "Engine",
                              "user": "user53",
                              "order_id": "v_9ojpQBpOpp5jjTDc0h",
                              "request_id": "vv9ojpQBpOpp5jjTDc0h",
                              "timeout": 30,
                              "target": {
                                  "id": "group001",
                                  "type": "group"
                              },
                              "status": "failure"
                          }
                      }
                  },
                  {
                      "_index": ".commands",
                      "_id": "wv9ojpQBpOpp5jjTDc0t",
                      "_score": 1.0,
                      "_source": {
                          "agent": {
                              "groups": [
                                  "group004",
                                  "group000",
                                  "group001"
                              ]
                          },
                          "@timestamp": "2025-01-22T14:26:11Z",
                          "delivery_timestamp": "2025-01-22T14:26:41Z",
                          "command": {
                              "action": {
                                  "args": {
                                      "arg1": "/path/to/executable/arg6"
                                  },
                                  "name": "restart",
                                  "version": "v4"
                              },
                              "source": "Engine",
                              "user": "user53",
                              "order_id": "v_9ojpQBpOpp5jjTDc0h",
                              "request_id": "vv9ojpQBpOpp5jjTDc0h",
                              "timeout": 30,
                              "target": {
                                  "id": "group001",
                                  "type": "group"
                              },
                              "status": "failure"
                          }
                      }
                  },
                  {
                      "_index": ".commands",
                      "_id": "w_9ojpQBpOpp5jjTDc0t",
                      "_score": 1.0,
                      "_source": {
                          "agent": {
                              "groups": [
                                  "group004",
                                  "group004",
                                  "group001",
                                  "group004"
                              ]
                          },
                          "@timestamp": "2025-01-22T14:26:11Z",
                          "delivery_timestamp": "2025-01-22T14:26:41Z",
                          "command": {
                              "action": {
                                  "args": {
                                      "arg1": "/path/to/executable/arg6"
                                  },
                                  "name": "restart",
                                  "version": "v4"
                              },
                              "source": "Engine",
                              "user": "user53",
                              "order_id": "v_9ojpQBpOpp5jjTDc0h",
                              "request_id": "vv9ojpQBpOpp5jjTDc0h",
                              "timeout": 30,
                              "target": {
                                  "id": "group001",
                                  "type": "group"
                              },
                              "status": "failure"
                          }
                      }
                  },
                  {
                      "_index": ".commands",
                      "_id": "xP9ojpQBpOpp5jjTDc0t",
                      "_score": 1.0,
                      "_source": {
                          "agent": {
                              "groups": [
                                  "group004",
                                  "group000",
                                  "group003",
                                  "group001",
                                  "group002"
                              ]
                          },
                          "@timestamp": "2025-01-22T14:26:11Z",
                          "delivery_timestamp": "2025-01-22T14:26:41Z",
                          "command": {
                              "action": {
                                  "args": {
                                      "arg1": "/path/to/executable/arg6"
                                  },
                                  "name": "restart",
                                  "version": "v4"
                              },
                              "source": "Engine",
                              "user": "user53",
                              "order_id": "v_9ojpQBpOpp5jjTDc0h",
                              "request_id": "vv9ojpQBpOpp5jjTDc0h",
                              "timeout": 30,
                              "target": {
                                  "id": "group001",
                                  "type": "group"
                              },
                              "status": "failure"
                          }
                      }
                  }
              ]
          }
      }
  • Basic command creation

    Using agent id: agent94

    • Request
      {
        "commands": [
          {
            "source": "Engine",
            "user": "user53",
            "target": {
              "id": "agent94",
              "type": "agent"
            },
            "action": {
              "name": "restart",
              "args": {
                "arg1": "/path/to/executable/arg6"
              },
              "version": "v4"
            },
            "timeout": 30
          }
        ]
      }
    • Response
      {
        "_index": ".commands",
        "_orders": [
          {
            "_id": "SQJ1jpQBrXbQNgi-ZGvv"
          }
        ],
        "result": "OK"
      }
    • Command indexed
      % curl http://127.0.0.1:9200/.commands/_search                    
      {
          "took": 2,
          "timed_out": false,
          "_shards": {
              "total": 1,
              "successful": 1,
              "skipped": 0,
              "failed": 0
          },
          "hits": {
              "total": {
                  "value": 1,
                  "relation": "eq"
              },
              "max_score": 1.0,
              "hits": [
                  {
                      "_index": ".commands",
                      "_id": "SQJ1jpQBrXbQNgi-ZGvv",
                      "_score": 1.0,
                      "_source": {
                          "agent": {
                              "groups": [
                                  "group001",
                                  "group002",
                                  "group002"
                              ]
                          },
                          "@timestamp": "2025-01-22T14:40:45Z",
                          "delivery_timestamp": "2025-01-22T14:41:15Z",
                          "command": {
                              "action": {
                                  "args": {
                                      "arg1": "/path/to/executable/arg6"
                                  },
                                  "name": "restart",
                                  "version": "v4"
                              },
                              "source": "Engine",
                              "user": "user53",
                              "order_id": "SAJ1jpQBrXbQNgi-ZGve",
                              "request_id": "RwJ1jpQBrXbQNgi-ZGve",
                              "timeout": 30,
                              "target": {
                                  "id": "agent94",
                                  "type": "agent"
                              },
                              "status": "failure"
                          }
                      }
                  }
              ]
          }
      }

Issues Resolved

Resolves #88

@QU3B1M QU3B1M self-assigned this Jan 19, 2025
@QU3B1M QU3B1M changed the title Add getters to Target model Implement commands expansion logic Jan 19, 2025
@QU3B1M QU3B1M marked this pull request as ready for review January 22, 2025 14:43
@QU3B1M QU3B1M requested a review from a team as a code owner January 22, 2025 14:43
@f-galland f-galland self-requested a review January 27, 2025 11:37
@f-galland
Copy link
Member

Agents are part of group group000:

$ curl -s http://127.0.0.1:9200/wazuh-agents/_search | grep group000
    "groups": ["group000","group004"],
    "groups": ["group000","group005"],

Issuing a command to group group000:

$ curl -s http://localhost:9200/_plugins/_command_manager/commands -H 'Content-Type: application/json' -d '{
  "commands": [
    {
      "source": "Engine",
      "user": "user53",
      "target": {
        "id": "group000",
        "type": "group"
      },
      "action": {
        "name": "restart",
        "args": {
          "arg1": "/path/to/executable/arg6"
        },
        "version": "v4"
      },
      "timeout": 30
    }
  ]
}' | jq
{
  "_index": "wazuh-commands",
  "_orders": [
    {
      "_id": "6fSGp5QBXpIVW99KMN3m"
    },
    {
      "_id": "6vSGp5QBXpIVW99KMN3m"
    }
  ],
  "result": "OK"
}

Comand creation log:

[2025-01-27T08:29:36,998][INFO ][c.w.c.i.CommandIndex     ] [integTest-0] Adding command with id [6fSGp5QBXpIVW99KMN3m] to the bulk request
[2025-01-27T08:29:36,998][INFO ][c.w.c.i.CommandIndex     ] [integTest-0] Adding command with id [6vSGp5QBXpIVW99KMN3m] to the bulk request

Command indexed:

$ curl -s http://127.0.0.1:9200/wazuh-commands/_search | jq '.hits.hits[0]._source'
{
  "agent": {
    "groups": [
      "group000",
      "group004"
    ]
  },
  "@timestamp": "2025-01-27T11:29:36Z",
  "delivery_timestamp": "2025-01-27T11:30:06Z",
  "command": {
    "action": {
      "args": {
        "arg1": "/path/to/executable/arg6"
      },
      "name": "restart",
      "version": "v4"
    },
    "source": "Engine",
    "user": "user53",
    "order_id": "6PSGp5QBXpIVW99KMN3i",
    "request_id": "5_SGp5QBXpIVW99KMN3i",
    "timeout": 30,
    "target": {
      "id": "group000",
      "type": "group"
    },
    "status": "failure"
  }
}

Basic command creation:

$ curl -s http://localhost:9200/_plugins/_command_manager/commands -H 'Content-Type: application/json' -d '
{
  "commands": [
    {
      "source": "Engine",
      "user": "user53",
      "target": {
        "id": "agent92",
        "type": "agent"
      },
      "action": {
        "name": "restart",
        "args": {
          "arg1": "/path/to/executable/arg6"
        },
        "version": "v4"
      },
      "timeout": 30
    }
  ]
}'
{"_index":"wazuh-commands","_orders":[{"_id":"_KiPp5QBytahsG79nekW"}],"result":"OK"}

Command indexed:

$ curl -s http://localhost:9200/wazuh-commands/_search | jq '.hits.hits[] | select(._id == "_KiPp5QBytahsG79nekW") | ._source'
{
  "agent": {
    "groups": [
      "group002",
      "group003",
      "group005"
    ]
  },
  "@timestamp": "2025-01-27T11:39:54Z",
  "delivery_timestamp": "2025-01-27T11:40:24Z",
  "command": {
    "action": {
      "args": {
        "arg1": "/path/to/executable/arg6"
      },
      "name": "restart",
      "version": "v4"
    },
    "source": "Engine",
    "user": "user53",
    "order_id": "-6iPp5QBytahsG79nekT",
    "request_id": "-qiPp5QBytahsG79nekT",
    "timeout": 30,
    "target": {
      "id": "agent92",
      "type": "agent"
    },
    "status": "failure"
  }
}

Copy link
Member

@f-galland f-galland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested and working.

@f-galland
Copy link
Member

Agent assigned to a group more than once:

Group group004 is assigned multiple times to some agents:

$ curl -s http://localhost:9200/wazuh-agents/_search | grep group004
    "groups": ["group001","group004","group000","group004"],
    "groups": ["group004","group001","group000"],
    "groups": ["group004","group004"],

Indexing command:

curl -s http://localhost:9200/_plugins/_command_manager/commands -H 'Content-Type: application/json' -d '{
  "commands": [
    {
      "source": "Engine",
      "user": "user53",
      "target": {
        "id": "group004",
        "type": "group"
      },
      "action": {
        "name": "restart",
        "args": {
          "arg1": "/path/to/executable/arg6"
        },
        "version": "v4"
      },
      "timeout": 30
    }
  ]
}' | jq
{
  "_index": "wazuh-commands",
  "_orders": [
    {
      "_id": "_6iXp5QBytahsG79A-l6"
    },
    {
      "_id": "AKiXp5QBytahsG79A-p6"
    },
    {
      "_id": "AaiXp5QBytahsG79A-p6"
    }
  ],
  "result": "OK"
}

Only 3 commands are created:

$ curl -s http://localhost:9200/wazuh-commands/_search | jq '.hits.hits[] | select(._id == "_6iXp5QBytahsG79A-l6" or ._id == "AKiXp5QBytahsG79A-p6" or ._id == "AaiXp5QBytahsG79A-p6") | ._source'
{
  "agent": {
    "groups": [
      "group004",
      "group004"
    ]
  },
  "@timestamp": "2025-01-27T11:47:59Z",
  "delivery_timestamp": "2025-01-27T11:48:29Z",
  "command": {
    "action": {
      "args": {
        "arg1": "/path/to/executable/arg6"
      },
      "name": "restart",
      "version": "v4"
    },
    "source": "Engine",
    "user": "user53",
    "order_id": "_qiXp5QBytahsG79A-l3",
    "request_id": "_aiXp5QBytahsG79A-l3",
    "timeout": 30,
    "target": {
      "id": "group004",
      "type": "group"
    },
    "status": "failure"
  }
}
{
  "agent": {
    "groups": [
      "group001",
      "group004",
      "group000",
      "group004"
    ]
  },
  "@timestamp": "2025-01-27T11:47:59Z",
  "delivery_timestamp": "2025-01-27T11:48:29Z",
  "command": {
    "action": {
      "args": {
        "arg1": "/path/to/executable/arg6"
      },
      "name": "restart",
      "version": "v4"
    },
    "source": "Engine",
    "user": "user53",
    "order_id": "_qiXp5QBytahsG79A-l3",
    "request_id": "_aiXp5QBytahsG79A-l3",
    "timeout": 30,
    "target": {
      "id": "group004",
      "type": "group"
    },
    "status": "failure"
  }
}
{
  "agent": {
    "groups": [
      "group004",
      "group001",
      "group000"
    ]
  },
  "@timestamp": "2025-01-27T11:47:59Z",
  "delivery_timestamp": "2025-01-27T11:48:29Z",
  "command": {
    "action": {
      "args": {
        "arg1": "/path/to/executable/arg6"
      },
      "name": "restart",
      "version": "v4"
    },
    "source": "Engine",
    "user": "user53",
    "order_id": "_qiXp5QBytahsG79A-l3",
    "request_id": "_aiXp5QBytahsG79A-l3",
    "timeout": 30,
    "target": {
      "id": "group004",
      "type": "group"
    },
    "status": "failure"
  }
}

* @return an Orders object containing the generated orders.
*/
@SuppressWarnings("unchecked")
private static Orders commandsToOrders(NodeClient client, List<Command> commands) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commands expansion should be moved elsewhere, instead of RestPostCommandAction.java.

Probably Orders model, as it is the return type.

Comment on lines -127 to -134

/// Commands expansion
/// ==================
/// Transforms the array of commands to orders.
/// While commands can be targeted to groups of agents, orders are targeted to individual
// agents.
/// Given a group of agents A with N agents, a total of N orders are generated. One for each
// agent.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These comments in the code help to understand the data flow and the responsibilities of this endpoint. Please, add them back.

Comment on lines +171 to +184
private static List<Command> getCommandList(RestRequest request) throws IOException {
// Request parsing
XContentParser parser = request.contentParser();
List<Command> commands = new ArrayList<>();
ensureExpectedToken(XContentParser.Token.START_OBJECT, parser.nextToken(), parser);
parser.nextToken();
if (parser.nextToken() == XContentParser.Token.START_ARRAY) {
commands = Command.parseToArray(parser);
} else {
log.error("Token does not match {}", parser.currentToken());
}

return commands;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parsing should be done in the model classes only.

channel.sendResponse(
new BytesRestResponse(
RestStatus.BAD_REQUEST,
"Cannot generate orders. Invalid agent IDs or groups."));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In which case an Agent ID is invalid?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove unused methods.

channel.sendResponse(
new BytesRestResponse(
RestStatus.BAD_REQUEST,
"No valid commands detected in the request body."));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"No commands provided" is shorter and translates better to the condition being checked.

Copy link
Member

@AlexRuiz7 AlexRuiz7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

la expansión de comandos no es correcta.
En una prueba que he hecho, he enviado el comando:

The commands expansion is not working as intended.

I tested using this command:

{
  "commands": [
    {
      "action": {
        "args": {},
        "name": "restart",
        "version": "5.0.0"
      },
      "source": "Users/Services",
      "user": "Management API",
      "timeout": 100,
      "target": {
        "id": "group001",
        "type": "group"
      }
    }
  ]
}

One of the expanded commands is:

      {
        "_index": "wazuh-commands",
        "_id": "J4LK0JQBHI2QBvqR03xq",
        "_score": 1,
        "_source": {
          "agent": {
            "groups": [
              "group000",
              "group001"
            ]
          },
          "command": {
            "source": "Users/Services",
            "user": "Management API",
            "target": {
              "type": "group",
              "id": "group001"
            },
            "action": {
              "name": "restart",
              "args": {

              },
              "version": "5.0.0"
            },
            "timeout": 100,
            "status": "pending",
            "order_id": "JoLK0JQBHI2QBvqR03xU",
            "request_id": "JYLK0JQBHI2QBvqR03xU"
          },
          "@timestamp": "2025-02-04T11:49:00Z",
          "delivery_timestamp": "2025-02-04T11:50:40Z"
        }
      }

The result should be a command targeted to an specific agent, but it is not.

            "target": {
              "type": "agent",
              "id": "<agent-id>"
            }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement commands' expansion logic
3 participants