feat: audit logs

[Anirudhxx]

Opening this PR so it makes it easier to discuss adding audit logs to parseable.

Fixes #765.

What kind of events do we want to capture?

• Right now I am only tracking the Action events
• Since these are accessible in the auth middleware I have sent the actual audit log post request in the auth middleware itself
• This most likely add a performance penalty to our server and should be moved elsewhere (Looking for some suggestions)

Adding any other details to the audit log

• Right now I am only capturing details like <action, ip address, request type>

This PR has:

• been tested to ensure log ingestion and log query works.
• added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
• added documentation for new or modified features or behaviors.

[de-sh]

This could just as well be an http::HeaderMap

[de-sh]

Why is this hardcoded, and what is the intention behind using user-agent as type?

[de-sh]

Are you intending to use the addr of the client?

[de-sh]

I will work on the Schema, will get back to you in a week's time

[Anirudhxx]

Hey @de-sh, just following up. Did you get a chance to work on this?

[de-sh]

Hey, sorry that this got lost, here is what I have been able to come up with:

{
  "audit_id": "String: uuid",
  "timestamp": "String: Datetime",
  "actor": {
    "type": "Enum: user/sync",
    "user_id": "String: username",
    "authorization_method": "Enum: basic/oauth/jwt"
  },
  "request": {
    "method": "Enum: POST/GET/PUT",
    "url": "String: url",
    "headers": {
      "key": "value"
    },
    "protocol": "Enum: kafka/http"
  },
  "action": {
    "type": "Enum: create/delete/update",
    "target": "Enum: s3/hottier/local-store/user-list",
    "resource": "Enum: object/user"
  },
  "response": {
    "code": int
  }
}

[de-sh]

This most likely add a performance penalty to our server and should be moved elsewhere (Looking for some suggestions)

I feel like the right way to do this would involve a dedicated thread that handles the serialization and reqwest part of your code. This since audit logging shouldn't impact normal functioning of request in any-ways and hence should be done, removed from the data flow path. I am exploring the feasibility of using a specialized tracing subscriber, to do this for us, similar to how we log to stdout now and how otel loggers work.

[Anirudhxx]

I am exploring the feasibility of using a specialized tracing subscriber, to do this for us, similar to how we log to stdout now and how otel loggers work.

@de-sh I'll implement a audit macro and replace it with the current reqwest code and create a new tracing subscriber which will look for these audit spans and send the audit log to the relevant parseable server. lmk if this works?