IP Camera

DEPRECATION This page contains documentation of in deprecated API. Use with care. The CameraSource API was replaced by the CameraController API.

iOS 10 added support for IP Camera to HomeKit. The IP Camera protocol is mostly based on RTP with SRTP. HAP-NodeJS updated to support IP Camera with v0.4.0.

To use IP Camera with HAP-NodeJS, you will need construct a camera object and invoke Accessory.configureCameraSource(camera) to inform HAP-NodeJS the accessory supports IP Camera. You can use Camera.ts as the reference and modify from there.

Please notice that due to protocol limitation, it's not possible to expose multiple cameras via a single HAP Endpoint. Trying to add a camera to a bridged accessory may result unexpected behavior. (since v0.5.1 bridged cameras are fully supported)

High Level Overview of Camera Object

A camera object in HAP-NodeJS must implements the following:

• services

Array of Service

iOS expects a CameraControl service and a CameraRTPStreamManagement service. The accessory can provide multiple CameraRTPStreamManagement services to iOS. To simplify the stream parameter negotiation, HAP-NodeJS provides a helper class StreamController which handles the stream request from iOS device. For more information about StreamController, please refer to section StreamController.

The number of CameraRTPStreamManagement services determines the number of concurrent streams the accessory can support. Apple recommends accessory to support at least two concurrent streams.

• handleSnapshotRequest(request, callback(error, snapshot))

This method will be invoked when iOS device requests snapshot from the accessory. The request contains width and height of the requested snapshot. The callback is a function that expects error or snapshot image buffer.

• handleCloseConnection(connectionID)

This method will be invoked when the underlying connection between iOS device and HAP-NodeJS is closed. You should clean up the streams that's associated with the connection at here.

iOS expects the snapshot to be under JPEG format. Please try to honor the requested width and height and not send a huge snapshot back to callback. HomeKit on iOS may crash if snapshot is too big.

StreamController

StreamController is a helper class which manages CameraRTPStreamManagement service and handles stream negotiation from iOS device. The constructor takes in Identifier, Options and the StreamDelegate object.

Identifier

Identifier is used to differentiate CameraRTPStreamManagement services. It must be unique for each StreamController.

Options

Options is used to determine the accessory's streaming capabilities.

{
  proxy: bool, // Accessory needs RTP/RTCP MUX Proxy
  srtp: bool, // Accessory supports SRTP AES_CM_128_HMAC_SHA1_80, iOS 10.1 requires this value to be true.
  video: {
    resolutions: [
      [1920, 1080, 30], // Width, Height, framerate
      [320, 240, 15], // Apple Watch requires this configuration
      ...
    ],
    codec: {
      profiles: [0, 1, 2], // Enum, please refer StreamController.VideoCodecParamProfileIDTypes
      levels: [0, 1, 2] // Enum, please refer StreamController.VideoCodecParamLevelTypes
    }
  },
  audio: {
    codecs: [
      {
        type: "OPUS", // Audio Codec, as of iOS 10.1, HomeKit only supports OPUS and AAC-eld
        samplerate: 24 // Sample Rate, Apple Watch only supports 16KHz
      },
      {
        type: "AAC-eld",
        samplerate: 16
      }
    ]
  }
}

StreamDelegate

StreamController expects a delegate object that handles the actual streaming request to iOS device. The delegate object needs to implement the following:

• prepareStream(request, callback(response))

This method will be invoked when iOS device tries to start video stream. The request contains information like sessionID, targetAddress and other informations that will be used to setup RTP/RTCP stream. The callback expects a response which contains information supplied by accessory to setup RTP/RTCP stream. For more information about request and response, please refer to section Prepare Stream.

• handleStreamRequest(request)

This method will be invoked when iOS device requests accessory to start, stop or reconfigure the RTP stream. The request will contain basic information like sessionID along with other parameters that will be helpful for streaming. For more information about handling stream request, please refer to section Handle Stream Request.

Prepare Stream

Prepare stream stage allows accessory to setup RTP server. Depends on whether the accessory needs RTP/RTCP proxy, the request may differs a little bit.

Request

{
  sessionID: Buffer, // Contains UUID of the session.
  targetAddress: String, // IP address for where the stream should be send to.
  video: {
    port: Int, // RTP/RTCP port which the video stream should be send to. This field may not present if accessory requires proxy.

    proxy_rtp: Int, // RTP port which the video stream should be send to. This field will only present if accessory requires proxy.
    proxy_rtcp: Int, // RTCP port which the video stream should be send to. This field will only present if accessory requires proxy.

    srtp_key: Buffer, // SRTP Key buffer. This field will only present if accessory supports SRTP.
    srtp_salt: Buffer, // SRTP Salt buffer. This field will only present if accessory supports SRTP.
  },
  audio: {
    port: Int, // RTP/RTCP port which the audio stream should be send to. This field may not present if accessory requires proxy.

    proxy_rtp: Int, // RTP port which the audio stream should be send to. This field will only present if accessory requires proxy.
    proxy_rtcp: Int, // RTCP port which the audio stream should be send to. This field will only present if accessory requires proxy.

    srtp_key: Buffer, // SRTP Key buffer. This field will only present if accessory supports SRTP.
    srtp_salt: Buffer, // SRTP Salt buffer. This field will only present if accessory supports SRTP.
  }
}

Response

{
  address: { // Optional if accessory requires proxy.
    address: String, // IP address of where the stream will coming from
    type: String // Type of IP address, `v4` or `v6`.
  },
  video: {
    port: Int // RTP/RTCP port of streaming server. Optional if accessory requires proxy.
    ssrc: Int // Synchronization source of the stream. Optional if accessory requires proxy.

    srtp_key: Buffer // SRTP Key. Required if accessory supports SRTP.
    srtp_salt: Buffer // SRTP Salt. Required if accessory supports SRTP.

    proxy_pt: Int // Payload Type of input stream. Required only if accessory requires proxy.
    proxy_server_address: String // IP address of RTP server. Required only if accessory requires proxy.
    proxy_server_rtp: Int // RTP port. Required only if accessory requires proxy.
    proxy_server_rtcp: Int // RTCP port. Required only if accessory requires proxy.
  },
  audio: {
    port: Int // RTP/RTCP port of streaming server. Optional if accessory requires proxy.
    ssrc: Int // Synchronization source of the stream. Optional if accessory requires proxy.

    srtp_key: Buffer // SRTP Key. Required if accessory supports SRTP.
    srtp_salt: Buffer // SRTP Salt. Required if accessory supports SRTP.

    proxy_pt: Int // Payload Type of input stream. Required only if accessory requires proxy.
    proxy_server_address: String // IP address of RTP server. Required only if accessory requires proxy.
    proxy_server_rtp: Int // RTP port. Required only if accessory requires proxy.
    proxy_server_rtcp: Int // RTCP port. Required only if accessory requires proxy.
  }
}

Handle Stream Request

After prepare stream finishes, iOS device will send command to actually start streaming, reconfigure the streaming parameters and stop streaming. All those requests will be handled by this function. Depends on the type, the request may have different fields.

Request

All requests will have at least the following fields.

{
  sessionID: Buffer // Contains UUID of the session.
  type: String // Type of the request, can be `start`, `stop`, or `reconfigure`.
  ...
}

Start

{
  video: {
    profile: Int, // Enum, refer StreamController.VideoCodecParamProfileIDTypes
    level: Int, // Enum, refer StreamController.VideoCodecParamLevelTypes
    width: Int, // Width of the stream
    height: Int, // Height of the stream
    fps: Int, // Framerate of the stream
    ssrc: Int, // Synchronization source of incoming stream
    pt: Int, // Payload Type
    max_bit_rate: Int, // Max Bit Rate
    rtcp_interval: Int, // RTCP Interval
    mtu: Int // MTU
  },
  audio: {
    codec: Int, // Enum, refer StreamController.AudioCodecTypes
    channel: Int, // Number of Channels
    bit_rate: Int, // Bit Rate
    sample_rate: Int, // Sample Rate, kHz
    packet_time: Int, // Packet Time
    ssrc: Int, // Synchronization source of incoming stream
    pt: Int, // Payload Type
    max_bit_rate: Int, // Max Bit Rate
    rtcp_interval: Int, // RTCP Interval
    comfort_pt: Int, // Payload Type for comfort noise
  }
}

Reconfigure

iOS device will request reconfigure stream when network condition changes. It has similar format as Start but only has video section. Not all fields will be present in video section under reconfigure request.

Stop

iOS device will send stop request when it wants to stop streaming. It only has the basic request payload.

HomeKit integrations

Currently HomeKit offers two integrations for IP Camera. If you place the camera accessory and a doorbell in the same room, when the doorbell event occurs, iOS will show a rich notification. The rich notification allows to you control other accessories under the same room. (Like a door lock)

Secondly, if you place the camera accessory and a motion sensor in the same room, when the motion sensor sends out motion detected event, iOS will show a notification containing the snapshot from the IP Camera.