redpanda not listening external service

[dhgokul]

have configured redpanda in local and a cloud machine with following properties:

/etc/redpanda/redpanda.yaml

config_file: /etc/redpanda/redpanda.yaml
node_uuid: SBpLkJrLzpiuYraMi5Vrcw8oZzMN67RGKQE4wbe7iy6T2iWKH
#-------
pandaproxy:
  # A list of address and port to listen for Kafka REST API requests.
  # Default: 0.0.0.0:8082
  pandaproxy_api: 
  - address: 0.0.0.0
    name: internal
    port: 8082
  - address: 0.0.0.0
    name: external
    port: 8083

  # A list of TLS configurations for the REST API.
  # Default: null
  pandaproxy_api_tls:
  - name: external
    # Whether to enable TLS.
    enabled: false
    # Require client authentication
    require_client_auth: false
    # The path to the server certificate PEM file.
    cert_file: ""
    # The path to the server key PEM file
    key_file: ""
    # The path to the truststore PEM file. Only required if client
    # authentication is enabled.
    truststore_file: ""
  - name: internal
    enabled: false

  # A list of address and port for the REST API to publish to client
  # Default: from pandaproxy_api
  advertised_pandaproxy_api:
    - address: 0.0.0.0
      name: internal
      port: 8082
    - address: 0.0.0.0
      name: external
      port: 8083

# The REST API client
pandaproxy_client:
  # List of address and port of the brokers
  # Default: "127.0.0.1:9092
  brokers:
   - address: 127.0.0.1
     port: 9092

  # TLS configuration for the brokers
  broker_tls:
    # Whether to enable TLS.
    enabled: false
    # Require client authentication
    require_client_auth: false
    # The path to the server certificate PEM file.
    cert_file: ""
    # The path to the server key PEM file
    key_file: ""
    # The path to the truststore PEM file. Only required if client authentication
    # is enabled.
    truststore_file: ""

  # Number of times to retry a request to a broker
  # Default: 5
  retries: 5

  # Delay (in milliseconds) for initial retry backoff
  # Default: 100ms
  retry_base_backoff_ms: 100

  # Number of records to batch before sending to broker
  # Default: 1000
  produce_batch_record_count: 1000

  # Number of bytes to batch before sending to broker
  # Defautl 1MiB
  produce_batch_size_bytes: 1048576

  # Delay (in milliseconds) to wait before sending batch
  # Default: 100ms
  produce_batch_delay_ms: 100

  # Interval (in milliseconds) for consumer request timeout
  # Default: 100ms
  consumer_request_timeout_ms: 100

  # Max bytes to fetch per request
  # Default: 1MiB
  consumer_request_max_bytes: 1048576
      
  # Timeout (in milliseconds) for consumer session
  # Default: 10s
  consumer_session_timeout_ms: 10000

  # Timeout (in milliseconds) for consumer rebalance
  # Default: 2s
  consumer_rebalance_timeout_ms: 2000

  # Interval (in milliseconds) for consumer heartbeats
  # Default: 500ms
  consumer_heartbeat_interval_ms: 500

  # SASL mechanism to use for authentication
  # Supported: SCRAM-SHA-{256,512}
  # Default: ""
  # Support for SASL is disabled when no mechanism is specified.
  sasl_mechanism: ""

  # Username for SCRAM authentication mechanisms
  # Default: ""
  scram_username: ""

  # Password for SCRAM authentication mechanisms
  # Default: ""
  scram_password: ""


#pandaproxy: {}
#-------
redpanda:
  admin:
  - address: 0.0.0.0
    port: 9644
  auto_create_topics_enabled: true
  data_directory: /var/lib/redpanda/data
  developer_mode: true
  kafka_api:
  - address: 0.0.0.0
    port: 9092
  node_id: 1
  rpc_server:
    address: 0.0.0.0
    port: 33145
    #-----------
  rpc_server_tls:
    # Whether to enable TLS for the RPC server.
    enabled: true
    # Require client authentication
    require_client_auth: false
    # The path to the server certificate PEM file.
    cert_file: ""
    # The path to the server key PEM file
    key_file: ""
    # The path to the truststore PEM file. Only required if client authentication
    # is enabled.
    truststore_file: ""

  # Address of RPC endpoint published to other cluster members.
  # Default: 0.0.0.0:33145
  advertised_rpc_api:
    address: 0.0.0.0
    port: 33145
  
  # Multiple listeners are also supported as per KIP-103.
  # The names must match those in advertised_kafka_api
  kafka_api:
  - address: 0.0.0.0
    name: internal
    port: 9092
  - address: 0.0.0.0
    name: external
    port: 9093

  # A list of TLS configurations for the Kafka API listeners.
  # Default: null

  # Multiple listeners are also supported as per KIP-103.
  # The names must match those in kafka_api
  advertised_kafka_api:
  - address: 0.0.0.0
    name: internal
    port: 9092
  - address: 0.0.0.0
    name: external
    port: 9093
  
    #------------
  seed_servers: []
rpk:
  coredump_dir: /var/lib/redpanda/coredump
  enable_memory_locking: false
  enable_usage_stats: true
  overprovisioned: false
  tune_aio_events: false
  tune_clocksource: false
  tune_coredump: false
  tune_cpu: false
  tune_disk_irq: false
  tune_disk_nomerges: false
  tune_disk_scheduler: false
  tune_disk_write_cache: false
  tune_fstrim: false
  tune_network: false
  tune_swappiness: false
  tune_transparent_hugepages: false

when i tried to create a topic and and post message, not created any topic in cloud setup.
used cmd:

rpk topic create --brokers cloud_public_ip:9092

no topic created in a cloud server.
Checked both ping and telnet for verification, no issue in that.
What is the actual missing one in the setup?

[emaxerrno]

hi @dhgokul - let's turn this into a discussion.

your configuration doesn't make sense, it is using 0.0.0.0 vs the advertised ip

  # Address of RPC endpoint published to other cluster members.
  # Default: 0.0.0.0:33145
  advertised_rpc_api:
    address: 0.0.0.0
    port: 33145
  
  # Multiple listeners are also supported as per KIP-103.
  # The names must match those in advertised_kafka_api
  kafka_api:
  - address: 0.0.0.0
    name: internal
    port: 9092
  - address: 0.0.0.0
    name: external
    port: 9093

Have you tried using this? https://github.com/vectorizedio/deployment-automation

[emaxerrno]

@0x5d can we help @dhgokul w/ a minimal conf.

[0x5d]

@dhgokul looks like you took the config from https://vectorized.io/docs/configuration/. The code there is only meant to be a reference, but will not work if used literally. To get a minimal configuration, please run rpk config init (you may need to run that as root, depending on your current user's permissions). Then you can set the advertised Kafka IP to your machine's IP.

[emaxerrno]

@bmansheim - we should have that rpk config init tip in the docs.