Distributed-logging-storage-practice

분산 로깅 스토리지 작성 예제

Test

1. Run server

go run cmd/server/main.go

2. Request produce to server

❯ curl -X POST localhost:8080 -d \
> '{"record": {"value": "X5gTdUBdJ3sa"}}'
{"offset":0}

❯ curl -X POST localhost:8080 -d \
'{"record": {"value": "X5gTdUBdJ4sa"}}'
{"offset":1}

❯ curl -X POST localhost:8080 -d \
'{"record": {"value": "cr7gTdUBdJ5sa"}}'
illegal base64 data at input byte 12

❯ curl -X POST localhost:8080 -d \
'{"record": {"value": "cr7gTdUBd5sa"}}'
{"offset":2}

3. Request consume to server

❯ curl -X GET localhost:8080 -d \
'{"offset": 1}'
{"record":{"value":"X5gTdUBdJ4sa","offset":1}}

❯ curl -X GET localhost:8080 -d \
'{"offset": 3}'
offset not found

❯ curl -X GET localhost:8080 -d \
'{"offset": 0}'
{"record":{"value":"X5gTdUBdJ3sa","offset":0}}

4. Install GRPC presets(Dev)

# On ubuntu
sudo apt install -y protobuf-compiler
# Install grpc presets
go install google.golang.org/protobuf/cmd/protoc-gen-go
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc
# Update PATH
export PATH="$PATH:$(go env GOPATH)/bin"

Implements

• Prototype
• Protocol buffer with struct
  • Schema
  • Domain
• Logging package
  • Storage
  • Indexing
  • Segment
  • Log(Set of segment)
• gRPC request
  • Service define
  • Server testing
  • Client testing
• Security
  • TLS
  • ACL
• Tracing
  • Metrics
  • Tracing
• Distributed service
  • Service discovery
  • Concensus
  • Multiplexer
  • Client load balancing
  • Service discovery with LB
• Deployments
  • Kubernetes
    • FQDN Advertising
  • Helm chart

Ref

• https://github.com/travisjeffery/proglog/tree/master/WriteALogPackage/
• https://github.com/gorilla/mux
• https://github.com/hashicorp/serf
• https://github.com/golang/protobuf