feat: Add content filtering

[KavithaSiva]

Proposed Content filtering API:

Option 1: (Current approach)
This is based on the assumption that the orchestration service only respects the first configuration provided for a filter service provider.

client.chatCompletion({
    deploymentConfiguration: { deploymentId: 'id' },
      llmConfig: {
        model_name: 'gpt-35-turbo-16k',
        model_params: { max_tokens: 50, temperature: 0.1 }
      },
      prompt: {
        template: [{ role: 'user', content: 'Hello!' }],
        messages_history: [{ role: 'system', content: 'Test system message' }]
      },
      filterConfig: { 
        input: { azureContentSafety: { Hate: 0, SelfHarm: 2 }},
        output: { 
          azureContentSafety: { Sexual: 4, Violence: 6 },
          someOtherFilterServiceProvider: { Hate: 0, SelfHarm: 2 }
        },
      }
  })
);

Other options:
Option 2:

client.chatCompletion({
    deploymentConfiguration: { deploymentId: 'id' },
      llmConfig: {
        model_name: 'gpt-35-turbo-16k',
        model_params: { max_tokens: 50, temperature: 0.1 }
      },
      prompt: {
        template: [{ role: 'user', content: 'Hello!' }],
        messages_history: [{ role: 'system', content: 'Test system message' }]
      },
      filterConfig: { 
        input: { azureContentSafety: { Hate: 0, SelfHarm: 2 }},
        output: [
          { azureContentSafety: { Sexual: 4, Violence: 6 }},
          { someOtherFilterServiceProvider: { Hate: 0, SelfHarm: 2 }}
        ]
      }
  })
);

Option 3:

client.chatCompletion({
    deploymentConfiguration: { deploymentId: 'id' },
      llmConfig: {
        model_name: 'gpt-35-turbo-16k',
        model_params: { max_tokens: 50, temperature: 0.1 }
      },
      prompt: {
        template: [{ role: 'user', content: 'Hello!' }],
        messages_history: [{ role: 'system', content: 'Test system message' }]
      },
      filterConfig: { 
        input: { type: 'azureContentSafety', config: { Hate: 0, SelfHarm: 2}},
        output: [
          { type: 'azureContentSafety', config: { Sexual: 4, Violence: 6 }},
          { type: 'someOtherFilterServiceProvider', config: { Hate: 0, SelfHarm: 2 }}
        ]
      }
  })
);

[KavithaSiva]

With formatting though the proposed API still looks verbose, WDY think? is this an issue:

client.chatCompletion({
    deploymentConfiguration: { deploymentId: 'id' },
      llmConfig: {
        model_name: 'gpt-35-turbo-16k',
        model_params: { max_tokens: 50, temperature: 0.1 }
      },
      prompt: {
        template: [{ role: 'user', content: 'Hello!' }],
        messages_history: [{ role: 'system', content: 'Test system message' }]
      },
      filterConfig: {
        input: {
          AzureContentSafety: {
            Hate: 0,
            SelfHarm: 2
          }
        },
        output: [
          {
            AzureContentSafety: {
              Sexual: 4,
              Violence: 6
            }
          },
          {
            SomeOtherFilterServiceProvider: {
              Hate: 0,
              SelfHarm: 2,
              Sexual: 4,
              Violence: 6
            }
          }
        ]
      }
  })
);

[ChristophMatthies]

Suggestion:

filterConfig: { 
    input: { azureContentSafety: { Hate: 0, SelfHarm: 2 }},
    output: {
      azureContentSafety: { Sexual: 4, Violence: 6 },
      someOtherFilterServiceProvider: { Hate: 0, SelfHarm: 2 }
    }
}

[marikaner]

To be honest I am not happy with either of the options. If we simplify whatever is passed in the config, I think it is confusing if it is this close to the original, but still different. That opens potential for errors. I don't know whether this was already discussed before, but I would suggest one of the following options:

• Keep the original spec from the orchestration service. This has the disadvantage, that it might be too verbose and difficult to use.
• Breakout from the object structure and pass it through a more high level API. I could imagine something in this direction:

createFilterConfig({
  input: createInputFilter({
    type: 'azureContentSafety',
    config: { Hate: 0, SelfHarm: 2}
  }),
 output: createOutputFilter({
    type: 'azureContentSafety',
    config: { Sexual: 4, Violence: 6 }
  })
})

or

createFilterConfig(
  createInputFilter({
    type: 'azureContentSafety',
    config: { Hate: 0, SelfHarm: 2}
  ),
 createOutputFilter({
    type: 'azureContentSafety',
    config: { Sexual: 4, Violence: 6 }
  })
)

• allow both (my favorite)

[jjtang1985]

I would have the following:

• [flexibility] Vanilla content filter payload structure (ref), which should be used by SDK internally and this internal function might also be used by the user, if they like.
• [convenience] High level API, so user can pass a ContentFilterConfig object. (we also need to offer one or multiple ways to build the object, like constructor/factory)

This looks similar to what Marika mentioned.

[KavithaSiva]

I would have the following:

• Vanilla orchestration service payload structure (ref), which should be used by SDK internally and this internal function might also be used by the user, if they like.
• High level API, so user can pass a ContentFilterConfig object. (we also need to offer one or multiple ways to build the object)

This looks similar to what Marika mentioned.

I am assuming then that we can add separate config objects for both llmConfig and prompt as well.
Clarified in call, we just look at the content filtering module in this PR.

[MatKuhr]

I think this is an issue, because only setting one of the two, or even setting none is allowed. But the current code leave the filter config empty, if one of the two is not defined, right?

[MatKuhr]

We could consider:

Suggested change

	input?: FilterServiceProvider;
	input?: FilterServiceProvider | [FilterServiceProvider];

That would also allow:

filterConfig: {
        input: [] as FilterServiceProvider,

Not sure how to best implement this internally, but using if (Array.isArray(config.input)) was what copilot suggested me 😉

Only downside I can think of is that the type signature becomes a bit more complex.

[MatKuhr]

Is this really generated code? I am wondering why it is being added just now?

[KavithaSiva]

The latest state as per Junjie's recommendations are #58