Skip to content

Commit f80aa31

Browse files
committed
ARTEMIS-5797 Updated docs for AMQP federation
Add more detailed documentation for AMQP federation convering more of the configuration options and example configurations.
1 parent 6700541 commit f80aa31

14 files changed

+496
-87
lines changed
Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
= Address Federation
2+
:idprefix:
3+
:idseparator: -
4+
:docinfo: shared
5+
6+
Address federation can be thought of as full multicast over the connected brokers.
7+
Every message sent to address on `Broker-1` will be delivered to every queue bound to that address on that broker, but also will be delivered to the matching address on `Broker-2`, `Broker-3` ... `Broker-N` and all the queues bound to that address.
8+
9+
Address federation creates consumers that read messages from the matching address on the remote peer when demand is detected on a local address that matches a specific federation address policy. On the remote peer a queue binding is created on the federated address which will store and forward messages sent to that address back to the initiating broker. Once a message arrives at the federation address consumer on the initiating broker it is then published into the local address which routes the message into each of the queues bound to that address.
10+
11+
Address federation can be configured to operate in a single direction between peers or can configured to operate in both directions between peers allowing message to flow in both directions delivering messages to consumers on either broker that were sent to the matching address on the opposing broker instance.
12+
13+
image::images/amqp-federation-address-concept.png[]
14+
15+
== Common Address Federation topologies
16+
17+
The sections that follow discuss some common address federation topologies and some configuration considerations specific to those topologies.
18+
19+
=== Two way Address Federation
20+
21+
This is a common case where consumers on either broker should receive messages sent from producers both on the local broker and on the remote broker. Messages only cross the federation connection if a consumer exists on the remote address otherwise messages do not leave the local broker.
22+
23+
image::images/amqp-federation-address-bi-directional.png[]
24+
25+
This configuration can be setup by adding a federation policy on both broker instances that configure the address to federate and a connection URI pointing to the opposing broker.
26+
27+
[,xml]
28+
----
29+
<broker-connections>
30+
<amqp-connection uri="tcp://HOST:PORT" name="federation-example">
31+
<federation>
32+
<local-address-policy name="example-local-address-policy">
33+
<include address-match="a.federated-address.#" />
34+
</local-address-policy>
35+
</federation>
36+
</amqp-connection>
37+
</broker-connections>
38+
----
39+
40+
It is also possible to configure this on only one of the two brokers using a remote address federation policy in the broker connection configuration.
41+
42+
[,xml]
43+
----
44+
<broker-connections>
45+
<amqp-connection uri="tcp://HOST:PORT" name="federation-example">
46+
<federation>
47+
<local-address-policy name="example-local-address-policy">
48+
<include address-match="a.federated-address.#" />
49+
</local-address-policy>
50+
<remote-address-policy name="example-remote-address-policy">
51+
<include address-match="a.federated-address.#" />
52+
</remote-address-policy>
53+
</federation>
54+
</amqp-connection>
55+
</broker-connections>
56+
----
57+
58+
In either of these two configurations the federation implementation will protect itself from looping messages so unlike other implementations you do not need to configure a `max-hops` setting.
59+
60+
=== Three Broker Mesh
61+
62+
An expansion of the previous configuration is a three broker mesh where each broker creates an address federation for the same address on each of the other two brokers when local demand is present. In this scenario any message published on an address on one of the brokers will be sent to consumers on that address that are active on the other two brokers.
63+
64+
image::images/amqp-federation-address-mesh.png[]
65+
66+
Because this configuration now creates a loop with more than two brokers the configuration must add a `max-hops` value to ensure that the messages sent to one of the brokers does not propagate beyond the two closest neighbors in the loop otherwise the messages would loop forever amongst the three brokers.
67+
68+
[,xml]
69+
----
70+
<broker-connections>
71+
<amqp-connection uri="tcp://HOST-1:PORT" name="federation-example-to-1">
72+
<federation>
73+
<local-address-policy name="example-local-address-policy-1" max-hops="1">
74+
<include address-match="a.federated-address.#" />
75+
</local-address-policy>
76+
</federation>
77+
</amqp-connection>
78+
<amqp-connection uri="tcp://HOST-2:PORT" name="federation-example-to-2">
79+
<federation>
80+
<local-address-policy name="example-local-address-policy-2" max-hops="1">
81+
<include address-match="a.federated-address.#" />
82+
</local-address-policy>
83+
</federation>
84+
</amqp-connection>
85+
</broker-connections>
86+
----
87+
88+
By configuring each of the brokers with a `max-hops` value as above the message will be stopped from moving beyond the neighboring broker that receives it and no message looping occurs.
89+
90+
=== Address Federation Ring
91+
92+
Another less common configuration is a good demonstration of a case where using the `max-hops` settings is critical for preventing an infinite loop of message federation. This is also known as a ring and involves creating an address federation configuration that connects to a neighboring broker amongst three or more broker instances. In this scenario the configuration must set a `max-hops` value of N-1 where N is the number of brokers in the ring.
93+
94+
image::images/amqp-federation-address-ring.png[]
95+
96+
To achieve this configuration each broker would need to add a broker connection for its neighbor as follows. Note the `max-hops=4` in this case since there are five brokers in the diagram, this ensures a federated message never cross back through the originating broker and thus cannot create an message loop.
97+
98+
[,xml]
99+
----
100+
<broker-connections>
101+
<amqp-connection uri="tcp://HOST-1:PORT" name="federation-example-to-1">
102+
<federation>
103+
<local-address-policy name="example-local-address-policy-1" max-hops="4">
104+
<include address-match="a.federated-address.#" />
105+
</local-address-policy>
106+
</federation>
107+
</amqp-connection>
108+
</amqp-connection>
109+
</broker-connections>
110+
----
111+
112+
=== Hub and Spoke Address Federation
113+
114+
Another configuration that allows consumers on brokers to read messages produced on any other broker peer is a hub and spoke configuration. In this configuration spoke brokers connect to a central hub and message traffic between the spokes passes through that hub broker which centralizes communication and allows the spokes to only need to know about one other broker, the hub broker.
115+
116+
image::images/amqp-federation-address-hub-spoke.png[]
117+
118+
This is configured on the spoke brokers much the same as dual address federation where a local and remote policy is configured both sides to listen for demand on the given address and federate messages when demand is detected. This setup allows the hub to now sense demand and federate from the spoke as needed and since each spoke mirrors this configuration the message can travel in either direction based on which spoke a consumer has subscribed to the federated address and which spoke a producer sends to the address. There is no need to configure a `max-hops` value in this configuration since the federation implementation prevents a loop in this case in the same manner it does for dual federation configurations.
119+
120+
[,xml]
121+
----
122+
<broker-connections>
123+
<amqp-connection uri="tcp://HUB-HOST:PORT" name="federation-example">
124+
<federation>
125+
<local-address-policy name="example-local-address-policy">
126+
<include address-match="a.federated-address.#" />
127+
</local-address-policy>
128+
<remote-address-policy name="example-remote-address-policy">
129+
<include address-match="a.federated-address.#" />
130+
</remote-address-policy>
131+
</federation>
132+
</amqp-connection>
133+
</broker-connections>
134+
----
135+
136+
While this is not the only configuration that can create this setup it does represent a simple and flexible configuration where spoke can be added without making configuration changes on the hub.

docs/user-manual/amqp-broker-connections.adoc

Lines changed: 11 additions & 85 deletions
Original file line numberDiff line numberDiff line change
@@ -429,7 +429,7 @@ IMPORTANT: To avoid confusion it is recommended that `address` `name` and `queue
429429
== Federation
430430

431431
Broker federation allows the local broker to create remote receivers for addresses or queues that have local demand.
432-
Conversely, the broker connection can send federation configuration to the remote broker causing it to create receivers on the local broker based on remote demand on an address or queue over this same connection.
432+
Conversely, the broker connection can send connection scoped federation configuration to the remote broker causing it to create receivers on the local broker based on remote demand on an address or queue over this same connection.
433433

434434
Add a `<federation>` element within the `<amqp-connection>` element to configure federation to the broker instance.
435435
The `<amqp-connection>` contains all the configuration for authentication and reconnection handling.
@@ -461,97 +461,23 @@ The broker connection federation configuration consists of one or more policies
461461
</broker-connections>
462462
----
463463

464-
=== Local and remote address federation
464+
=== Address federation
465465

466-
Local or Remote address federation configures the local or remote broker to watch for demand on addresses.
467-
When demand exists it will create a consumer on the matching address on the opposing broker.
468-
Because the consumers are created on addresses on the opposing broker, the authentication credentials supplied to the broker connection must have sufficient access to read from (local address federation) or write to the matching address (remote address federation) on the opposing broker.
466+
Address federation can be thought of as full multicast over a set of loosely coupled brokers. Every message sent to address on `Broker-1` will be delivered to every queue bound to that address on that broker, but also will be delivered to the matching address on `Broker-2`, `Broker-3` ... `Broker-N` and all the queues bound to that address.
469467

470-
An example of address federation configuration is shown below.
471-
Local and remote address policies have identical configuration parameters, only the policy element names are different.
468+
.Address Federation
469+
image::images/amqp-federation-address-concept.png[]
472470

473-
[,xml]
474-
----
475-
<broker-connections>
476-
<amqp-connection uri="tcp://HOST:PORT" name="federation-example">
477-
<federation>
478-
<local-address-policy name="example-local-address-policy"
479-
auto-delete="false"
480-
auto-delete-delay="1"
481-
auto-delete-message-count="12"
482-
max-hops="1"
483-
enable-divert-bindings="true">
484-
<include address-match="local-address.#" />
485-
<exclude address-match="local-address.excluded" />
486-
</local-address-policy>
487-
</federation>
488-
</amqp-connection>
489-
</broker-connections>
490-
----
471+
For further details please see the xref:amqp-address-federation.adoc#address-federation[Address Federation] documentation.
491472

492-
name::
493-
The name of the policy, these names should be unique within a broker connection's federation policy elements.
494-
include::
495-
The address-match pattern to use to match included addresses, multiple of these can be set.
496-
If none are set all addresses are matched.
497-
exclude::
498-
The address-match pattern to use to match excluded addresses, multiple of these can be set, or it can be omitted if no excludes are needed.
499-
max-hops::
500-
The maximum number of hops that a message can perform to be federated.
501-
Default value is `0` and will work for most simple federation deployments.
502-
However, in certain topologies a higher value may be required to prevent looping.
503-
auto-delete::
504-
For address federation, a durable queue is created on the broker from which messages are being federated.
505-
Set this property to `true` to mark the queue for automatic deletion once the initiating broker disconnects and the delay and message count parameters are met.
506-
This is a useful option if you want to automate the cleanup of dynamically-created queues.
507-
The default value is `false`, which means that the queue is not automatically deleted.
508-
auto-delete-delay::
509-
The amount of time in milliseconds after the initiating broker has disconnected before the created queue can be eligible for `auto-delete`.
510-
The default value is `0` when option is omitted.
511-
auto-delete-message-count::
512-
After the initiating broker has disconnected, the maximum number of messages allowed in the remote queue for the queue to be eligible for automatic deletion.
513-
The default value is `0`.
514-
enable-divert-bindings::
515-
Setting to `true` enables divert bindings to be listened-to for demand.
516-
If a divert binding with an address matches the included addresses for the address policy, any queue bindings that match the forwarding address of the divert creates demand. The default value is `false`.
473+
=== Queue federation
517474

518-
=== Local and remote queue federation
475+
Queue federation offers a means of load balancing message queues across multiple broker instances. Messages sent to a queue on `Broker-1` will be consumed and sent to the matching queue on `Broker-2` if there is no local consumer available to consume the message.
519476

520-
Local or Remote queue federation configures the local or remote broker to watch for demand on queues and when demand exists it will create a consumer on the matching queue on the opposing broker.
521-
Because the consumers are created on queues on the opposing broker, the authentication credentials supplied to the broker connection must have sufficient access to read from (local queue federation) or write to the matching queue (remote queue federation) on the opposing broker.
477+
.Queue Federation
478+
image::images/amqp-federation-queue-concept.png[]
522479

523-
An example of queue federation configuration is shown below.
524-
Local and remote queue policies have identical configuration parameters, only the policy element names are different.
525-
526-
[,xml]
527-
----
528-
<broker-connections>
529-
<amqp-connection uri="tcp://HOST:PORT" name="federation-example">
530-
<federation>
531-
<local-queue-policy name="example-local-queue-policy">
532-
<include address-match="#" queue-match="remote-queue" />
533-
<exclude address-match="excluded.#" queue-match="remote-queue-excluded" />
534-
</local-queue-policy>
535-
</federation>
536-
</amqp-connection>
537-
</broker-connections>
538-
----
539-
540-
name::
541-
The name of the policy.
542-
These names should be unique within a broker connection's federation policy elements.
543-
include::
544-
The queue-match pattern to use to match included queues.
545-
Multiple of these can be set.
546-
If none are set all queues are matched.
547-
exclude::
548-
The queue-match pattern to use to match excluded queues.
549-
Multiple of these can be set, or it can be omitted if no excludes are needed.
550-
priority-adjustment::
551-
When federation consumers are created this value can be used to ensure that those federation consumers have a lower priority value than other local consumers on the same queue.
552-
The default is `-1`.
553-
include-federated::
554-
Controls if consumers on a queue which come from federation instances should be counted when observing a queue for demand, by default this value is `false` and federation consumers are not counted.
480+
For further details please see the xref:amqp-queue-federation.adoc#queue-federation[Queue Federation] documentation.
555481

556482
== Bridges
557483

0 commit comments

Comments
 (0)