Python wrappers for source creation

[pyalex]

Summary

Adding functions to the Python SDK for source objects creation. This is a fully backward-compatible change. Users can continue to use both thrift-based classes and new Python wrappers.

Why / Goal

The primary motivation is to enable the addition of extra attributes at the source level. Similarly to how it's done in GroupBy and Join: all extra arguments are stored in the customJson attribute in Thrift.
Sources can have all sorts of metadata, ie bootstrap.server for Kafka source, which can be helpful for a streaming job.

Additional benefits:

• Less verbose API
  before:

my_source = ttypes.Source(
    events=ttypes.EventSource(
          table=...
    )
)

after:

my_source = source.EventSource(
    table=...
)

• Improving API consistency: existing Python wrappers (ie, GroupBy, Join) use Pythonic snake case for parameter names, whereas code generated from Thrift uses camel case (ie, snapshotTable in EntitySource)
• Omitting a required attribute will produce a more meaningful error

Test Plan

• Added Unit Tests
• [ x ] Covered by existing CI
• Integration tested

Checklist

• Documentation update

Reviewers

[nikhil-zlai]

I like the source wrappers. Can you edit docs also please?

Given that there is already a way to specify stream config alongside the topic, can we skip the api change to add customJson?

For specifying streaming params - we use topicInfo:

chronon/online/src/main/scala/ai/chronon/online/DataStreamBuilder.scala

Line 32 in bf9497b

// kafka://topic_name/schema=my_schema/host=X/port=Y should parse into TopicInfo(topic_name, kafka, {schema: my_schema, host: X, port Y})

case class TopicInfo(name: String, topicType: String, params: Map[String, String])

 topic="kafka://topic_name/schema=my_schema/host=X/port=Y" will parse into 

TopicInfo(topic_name, kafka, {schema: my_schema, host: X, port Y})

[pyalex]

Hey @nikhil-zlai , thanks for the review!
There's more use for those extra attributes, than just Kafka host and port. For example, I want to store the Avro JSON schema near the source definition and attach it to the source. Or specify all kinds of Kafka consumer properties.

TopicInfo has limited usage since it makes / and = special symbols, and if I were to add anything encoded with base64 to this topic string, it would simply break.

[pyalex]

Updated docs

[nikhil-zlai]

I want to store the Avro JSON schema near the source definition and attach it to the source. Or specify all kinds of Kafka consumer properties.

I see. That definitely justifies the change.

[nikhil-zlai]

one minor nit. but lgtm!

[pyalex]

@hzding621, please take another look

[pyalex]

Ping @hzding621

[pengyu-hou]

can we change it to use the MetaData, because we already have customJson in MetaData. It is easier to wrap everything in Metadata so we don't have to add new fields in the future

Suggested change

	5: optional string customJson
	5: optional MetaData metaData

[pengyu-hou]

Ditto, use MetaData instead of creating a new customJson field.

Suggested change

	5: optional string customJson
	5: optional MetaData metaData