Welcome to the EnMasse documentation, where you can find information to help you learn about EnMasse and start exploring its features.

From getting started to trying out more advanced installation and configuration, these resources provide what you need to set up and manage EnMasse as a service administrator or a messaging tenant.

1. Overview

EnMasse is an open source project for managed, self-service messaging on Kubernetes. EnMasse can run on your own infrastructure or in the cloud, and simplifies running a messaging infrastructure for your organization.

The service admin can deploy and manage messaging infrastructure, while messaging tenants can request messaging resources, both using both cloud-native APIs and tools.

1.1. Features

  • Built-in authentication and authorization of clients and identity management

  • Runs on Kubernetes: deploy on-premise or in the cloud

  • Different messaging patterns such as request-response, publish-subscribe and events

  • Decouple operation of infrastructure from configuration and use by applications

EnMasse can be used for many purposes, such as moving your messaging infrastructure to the cloud without depending on a specific cloud provider, building a scalable messaging backbone for IoT, or just as a cloud-ready version of a message broker.

EnMasse can provision different types of messaging depending on your use case. A user can request messaging resources by creating an address space.

EnMasse currently supports a standard and a brokered address space type, each with different semantics.

1.1.1. Standard address space

The standard address space type is the default type in EnMasse, and is focused on scaling in the number of connections and the throughput of the system. It supports AMQP and MQTT protocols, with more to come in the future. This address space type is based on other open source projects such as Apache ActiveMQ Artemis and Apache Qpid Dispatch Router and provides elastic scaling of these components. This image illustrates the high-level architecture of the standard address space:

Standard Address Space

1.1.2. Brokered address space

The brokered address space type is the "classic" message broker in the cloud which supports AMQP, CORE, OpenWire, and MQTT protocols. It supports JMS with transactions, message groups, selectors on queues and so on. These features are useful for building complex messaging patterns. This address space is also more lightweight as it features only a single broker and a management console. This image illustrates the high-level architecture of the brokered address space:

Brokered Address Space

2. Getting Started

2.1. EnMasse on OpenShift

This guide will walk through the process of setting up EnMasse on OpenShift with clients for sending and receiving messages.

Prerequisites
  • To install EnMasse, the OpenShift client tools are required. You can download the OpenShift Origin client from OpenShift Origin. EnMasse has been tested to work with the latest stable release of the OpenShift Origin Client.

  • An OpenShift cluster is required. If you do not have an OpenShift cluster available, see Minishift for an example of how to run a local instance of OpenShift on your machine.

  • A user on the OpenShift cluster with cluster-admin permissions is required, in order to set up the required cluster roles and API services.

2.1.1. Downloading EnMasse

Procedure

2.1.2. Installing EnMasse using a YAML bundle

The simplest way to install EnMasse is to use the predefined YAML bundles.

Procedure
  1. Log in as a user with cluster-admin privileges:

    oc login -u system:admin
  2. (Optional) Replace namespace if you want to deploy to something other than enmasse-infra (and substitute enmasse-infra in subsequent steps):

    sed -i 's/enmasse-infra/my-namespace/' install/bundles/enmasse-with-standard-authservice/*.yaml
  3. Create the project where you want to deploy EnMasse:

    oc new-project enmasse-infra
  4. Deploy using the enmasse-with-standard-authservice bundle:

    oc apply -f install/bundles/enmasse-with-standard-authservice

2.1.3. Creating address spaces using the command line

Procedure
  1. Log in as a messaging tenant:

    oc login -u developer
  2. Create the project for the messaging application:

    oc new-project myapp
  3. Create an address space definition:

    apiVersion: enmasse.io/v1alpha1
    kind: AddressSpace
    metadata:
      name: myspace
    spec:
      type: standard
      plan: standard-unlimited
  4. Create the address space:

    oc create -f standard-address-space.yaml
  5. Check the status of the address space:

    oc get addressspace myspace -o jsonpath={.status.isReady}

    The address space is ready for use when the above command outputs true.

2.1.4. Creating addresses using the command line

Procedure
  1. Create an address definition:

    apiVersion: enmasse.io/v1alpha1
    kind: Address
    metadata:
        name: myspace.myqueue
    spec:
        address: myqueue
        type: queue
        plan: standard-small-queue
    Note
    Prefixing the name with the address space name is required to ensure addresses from different address spaces do not collide.
  2. Create the address:

    oc create -f standard-small-queue.yaml
  3. List the addresses:

    oc get addresses -o yaml

2.1.5. Creating users using the command line

In EnMasse users can be created using standard command-line tools.

Prerequisites
Procedure
  1. Save the user definition to a file:

    apiVersion: user.enmasse.io/v1alpha1
    kind: MessagingUser
    metadata:
      name: myspace.user1
    spec:
      username: user1
      authentication:
        type: password
        password: cGFzc3dvcmQ= # Base64 encoded
      authorization:
        - addresses: ["myqueue", "queue1", "queue2", "topic*"]
          operations: ["send", "recv"]
        - addresses: ["anycast1"]
          operations: ["send"]
  2. Create the user and associated user permissions:

    oc create -f user-example1.yaml
Additional resources

2.1.6. Sending and receiving messages

Prerequisites
  • Installed Apache Qpid Proton Python bindings.

  • An address space named myspace must be created.

  • An address named myqueue must be created.

  • A user named user1 with password "password" must be created.

Procedure
  1. Save Python client example to a file:

    from __future__ import print_function, unicode_literals
    import optparse
    from proton import Message
    from proton.handlers import MessagingHandler
    from proton.reactor import Container
    
    class HelloWorld(MessagingHandler):
        def __init__(self, url):
            super(HelloWorld, self).__init__()
            self.url = url
    
        def on_start(self, event):
            event.container.create_receiver(self.url)
            event.container.create_sender(self.url)
    
        def on_sendable(self, event):
            event.sender.send(Message(body="Hello World!"))
            event.sender.close()
    
        def on_message(self, event):
            print("Received: " + event.message.body)
            event.connection.close()
    
    parser = optparse.OptionParser(usage="usage: %prog [options]")
    parser.add_option("-u", "--url", default="amqps://localhost:5672/myqueue",
                      help="url to use for sending and receiving messages")
    opts, args = parser.parse_args()
    
    try:
        Container(HelloWorld(opts.url)).run()
    except KeyboardInterrupt: pass
  2. Retrieve the address space messaging endpoint host name:

    oc get addressspace myspace -o jsonpath={.status.endpointStatuses[?(@.name==\'messaging\')].externalHost}

    Use the output as the host name in the following step.

  3. Run the client:

    python client-example1.py -u amqps://user1:password@messaging.example1.com:443/myqueue

We have seen how to set up EnMasse on OpenShift, and how to communicate with it using an AMQP client.

2.2. EnMasse on Kubernetes

This guide will walk through the process of setting up EnMasse on Kubernetes with clients for sending and receiving messages.

Prerequisite
  • To install EnMasse, you must have Kubernetes installed. You can use minikube if you want to install EnMasse on your laptop.

  • A user on the Kubernetes cluster with cluster-admin permissions is required, in order to set up the required cluster roles and API services.

2.2.1. Downloading EnMasse

Procedure

2.2.2. Installing EnMasse using a YAML bundle

The simplest way to install EnMasse is to use the predefined YAML bundles.

Procedure
  1. Create the namespace where you want to deploy EnMasse:

    kubectl create namespace enmasse-infra
    kubectl config set-context $(kubectl config current-context) --namespace=enmasse-infra
  2. Create a certificate to use with the standard authentication service. For testing purposes, you can create a self-signed certificate:

    mkdir -p standard-authservice-cert
    openssl req -new -x509 -batch -nodes -days 11000 -subj "/O=io.enmasse/CN=standard-authservice.enmasse-infra.svc.cluster.local" -out standard-authservice-cert/tls.crt -keyout standard-authservice-cert/tls.key
  3. Create a secret with the standard authentication service certificate:

    kubectl create secret tls standard-authservice-cert --cert=standard-authservice-cert/tls.crt --key=standard-authservice-cert/tls.key
  4. Create a certificate to use with the API server. For testing purposes, you can create a self-signed certificate:

    mkdir -p api-server-cert/
    openssl req -new -x509 -batch -nodes -days 11000 -subj "/O=io.enmasse/CN=api-server.enmasse-infra.svc.cluster.local" -out api-server-cert/tls.crt -keyout api-server-cert/tls.key
  5. Create a secret containing the API server certificate:

    kubectl create secret tls api-server-cert --cert=api-server-cert/tls.crt --key=api-server-cert/tls.key
  6. Deploy using the enmasse-with-standard-authservice bundle:

    kubectl apply -f install/bundles/enmasse-with-standard-authservice

2.2.3. Creating address spaces using the command line

Procedure
  1. Create an address space definition:

    apiVersion: enmasse.io/v1alpha1
    kind: AddressSpace
    metadata:
      name: myspace
    spec:
      type: standard
      plan: standard-unlimited
  2. Create the address space:

    kubectl create -f standard-address-space.yaml
  3. Check the status of the address space:

    kubectl get addressspace myspace -o jsonpath={.status.isReady}

    The address space is ready for use when the above command outputs true.

2.2.4. Creating addresses using the command line

Procedure
  1. Create an address definition:

    apiVersion: enmasse.io/v1alpha1
    kind: Address
    metadata:
        name: myspace.myqueue
    spec:
        address: myqueue
        type: queue
        plan: standard-small-queue
    Note
    Prefixing the name with the address space name is required to ensure addresses from different address spaces do not collide.
  2. Create the address:

    kubectl create -f standard-small-queue.yaml
  3. List the addresses:

    kubectl get addresses -o yaml

2.2.5. Creating users using the command line

In EnMasse users can be created using standard command-line tools.

Prerequisites
Procedure
  1. Save the user definition to a file:

    apiVersion: user.enmasse.io/v1alpha1
    kind: MessagingUser
    metadata:
      name: myspace.user1
    spec:
      username: user1
      authentication:
        type: password
        password: cGFzc3dvcmQ= # Base64 encoded
      authorization:
        - addresses: ["myqueue", "queue1", "queue2", "topic*"]
          operations: ["send", "recv"]
        - addresses: ["anycast1"]
          operations: ["send"]
  2. Create the user and associated user permissions:

    kubectl create -f user-example1.yaml
Additional resources

2.2.6. Sending and receiving messages

Prerequisites
  • Installed Apache Qpid Proton Python bindings.

  • An address space named myspace must be created.

  • An address named myqueue must be created.

  • A user named user1 with password "password" must be created.

Procedure
  1. Save Python client example to a file:

    from __future__ import print_function, unicode_literals
    import optparse
    from proton import Message
    from proton.handlers import MessagingHandler
    from proton.reactor import Container
    
    class HelloWorld(MessagingHandler):
        def __init__(self, url):
            super(HelloWorld, self).__init__()
            self.url = url
    
        def on_start(self, event):
            event.container.create_receiver(self.url)
            event.container.create_sender(self.url)
    
        def on_sendable(self, event):
            event.sender.send(Message(body="Hello World!"))
            event.sender.close()
    
        def on_message(self, event):
            print("Received: " + event.message.body)
            event.connection.close()
    
    parser = optparse.OptionParser(usage="usage: %prog [options]")
    parser.add_option("-u", "--url", default="amqps://localhost:5672/myqueue",
                      help="url to use for sending and receiving messages")
    opts, args = parser.parse_args()
    
    try:
        Container(HelloWorld(opts.url)).run()
    except KeyboardInterrupt: pass
  2. Retrieve the address space messaging endpoint host name:

    kubectl get addressspace myspace -o jsonpath={.status.endpointStatuses[?(@.name==\'messaging\')].externalHost}

    Use the output as the host name in the following step.

  3. Run the client:

    python client-example1.py -u amqps://user1:password@messaging.example1.com:443/myqueue

We have seen how to set up EnMasse on Kubernetes, and how to communicate with it using an AMQP client.

3. Service Admin Guide

The service administrator guide provides resources on how to set up and manage EnMasse, infrastructure configuration and plans.

3.1. Installing EnMasse

3.1.1. Installing EnMasse on OpenShift

EnMasse can be installed using automated Ansible playbooks, the OpenShift template, or the manual steps.

Prerequisites
  • To install EnMasse, the OpenShift client tools are required. You can download the OpenShift Origin client from OpenShift Origin. EnMasse has been tested to work with the latest stable release of the OpenShift Origin Client.

  • An OpenShift cluster is required. If you do not have an OpenShift cluster available, see Minishift for an example of how to run a local instance of OpenShift on your machine.

  • A user on the OpenShift cluster with cluster-admin permissions is required, in order to set up the required cluster roles and API services.

Downloading EnMasse
Procedure
Installing EnMasse using a YAML bundle

The simplest way to install EnMasse is to use the predefined YAML bundles.

Procedure
  1. Log in as a user with cluster-admin privileges:

    oc login -u system:admin
  2. (Optional) Replace namespace if you want to deploy to something other than enmasse-infra (and substitute enmasse-infra in subsequent steps):

    sed -i 's/enmasse-infra/my-namespace/' install/bundles/enmasse-with-standard-authservice/*.yaml
  3. Create the project where you want to deploy EnMasse:

    oc new-project enmasse-infra
  4. Deploy using the enmasse-with-standard-authservice bundle:

    oc apply -f install/bundles/enmasse-with-standard-authservice
Installing EnMasse using OpenShift template

Installing EnMasse using the OpenShift template is useful for evaluating EnMasse. For a production setup, it is recommended to use one of the following installation methods instead:

Procedure
  1. Log in as as a user with cluster-admin privileges:

    oc login -u system:admin
  2. Create the project where you want to deploy EnMasse:

    oc new-project enmasse-infra
  3. Deploy using enmasse-with-standard-authservice.yaml template:

    oc process -f install/templates/enmasse-with-standard-authservice.yaml NAMESPACE=enmasse-infra | oc apply -f -
Installing EnMasse using Ansible

Installing EnMasse using Ansible requires creating an inventory file with the variables for configuring the system. Example inventory files can be found in the ansible/inventory folder.

An example inventory file that enables both the API server and service broker integration:

[enmasse]
localhost ansible_connection=local

[enmasse:vars]
namespace=enmasse-infra
enable_rbac=False
api_server=True
service_catalog=True
register_api_server=True
keycloak_admin_password=admin
authentication_services=["standard"]
enable_monitoring=False

The following Ansible configuration settings are supported:

Table 1. Ansible Configuration Settings

Name

Description

Default value

Required

namespace

Specifies the namespace where EnMasse is installed.

Not applicable

yes

enable_rbac

Specifies whether to enable RBAC authentication of REST APIs.

True

no

service_catalog

Specifies whether to enable integration with the Service Catalog.

False

no

authentication_services

Specifies the list of authentication services to deploy. none and standard are supported.

none

no

keycloak_admin_password

Specifies the admin password to use for the standard authentication service Keycloak instance.

Not applicable

yes (if standard authentication service is enabled)

api_server

Specifies whether to enable the REST API server.

True

no

register_api_server

Specifies whether to register the API server with Kubernetes master.

False

no

secure_api_server

Specifies whether to enable mutual TLS for the API server.

False

no

enable_monitoring

Specifies whether to enable the monitoring services

Not applicable

no

smtp_server

Specifies the SMTP server used to send alert emails

Not applicable

no

smtp_username

Specifies the username used to authenticate to the SMTP server

Not applicable

no

smtp_password

Specifies the password used to authenticate to the SMTP server

Not applicable

no

smtp_from_address

Specifies the from address displayed in alert emails

Not applicable

no

sysadmin_email

Specifies the email address to which alerts are sent

Not applicable

no

Procedure
  1. (Optional) Create an inventory file.

  2. Run the Ansible playbook:

    ansible-playbook -i inventory-file ansible/playbooks/openshift/deploy_all.yml
Installing EnMasse manually

The manual deployment procedure can be performed on any platform supporting the OpenShift client.

Creating the project for EnMasse
Procedure
  • Create the enmasse-infra project:

    oc new-project enmasse-infra
Deploying authentication services

EnMasse requires at least one authentication service to be deployed. The authentication service can be none (allow all), standard (Keycloak), or external (not managed by EnMasse).

Deploying the none authentication service
Procedure
  1. Create the none authentication service:

    oc create -f ./install/components/none-authservice
Deploying the standard authentication service
Procedure
  1. (Optional) Replace namespace if you want to deploy to something other than enmasse-infra:

    sed -i 's/enmasse-infra/my-namespace/' install/components/standard-authservice/*RoleBinding*.yaml
  2. Create the standard authentication service:

    oc create -f ./install/components/standard-authservice
Deploying the Address Space Controller

The Address Space Controller is responsible for creating the infrastructure used by address spaces.

Procedure
  1. (Optional) Replace namespace if you want to deploy to something other than enmasse-infra:

    sed -i 's/enmasse-infra/my-namespace/' install/components/address-space-controller/*.yaml
  2. Deploy the Address Space Controller

    oc apply -f install/components/address-space-controller
Deploying the API server

The API server provides a REST API for creating address spaces and addresses. It can also serve as an aggregated API server if it is registered as an API service.

Procedure
  1. (Optional) Replace namespace if you want to deploy to something other than enmasse-infra (and substitute enmasse-infra in subsequent steps):

    sed -i 's/enmasse-infra/my-namespace/' install/components/api-service/*.yaml
    sed -i 's/enmasse-infra/my-namespace/' install/components/api-server/*.yaml
  2. Register API service

    oc apply -f install/components/api-service
  3. Deploy the API Server

    oc apply -f install/components/api-server/
(Optional) Deploying the service broker

The service broker provides an implementation of the Open Service Broker API that integrates with the OpenShift Service Catalog.

Prerequisites
  • The service broker requires the standard authentication service to be deployed.

Procedure
  1. (Optional) Replace namespace if you want to deploy to something other than enmasse-infra:

    sed -i 's/enmasse-infra/my-namespace/' install/components/service-broker/*.yaml
    sed -i 's/enmasse-infra/my-namespace/' install/components/cluster-service-broker/*.yaml
  2. Deploy the Service Broker:

    oc apply -f install/components/service-broker
  3. Register the Service Broker with the OpenShift Service Catalog:

    oc apply -f install/components/cluster-service-broker

3.1.2. Installing EnMasse on Kubernetes

These steps follow the manual deployment procedure and work on any platform supporting the kubectl command-line client.

Prerequisite
  • To install EnMasse, you must have Kubernetes installed. You can use minikube if you want to install EnMasse on your laptop.

  • A user on the Kubernetes cluster with cluster-admin permissions is required, in order to set up the required cluster roles and API services.

Downloading EnMasse
Procedure
Installing EnMasse using a YAML bundle

The simplest way to install EnMasse is to use the predefined YAML bundles.

Procedure
  1. Create the namespace where you want to deploy EnMasse:

    kubectl create namespace enmasse-infra
    kubectl config set-context $(kubectl config current-context) --namespace=enmasse-infra
  2. Create a certificate to use with the standard authentication service. For testing purposes, you can create a self-signed certificate:

    mkdir -p standard-authservice-cert
    openssl req -new -x509 -batch -nodes -days 11000 -subj "/O=io.enmasse/CN=standard-authservice.enmasse-infra.svc.cluster.local" -out standard-authservice-cert/tls.crt -keyout standard-authservice-cert/tls.key
  3. Create a secret with the standard authentication service certificate:

    kubectl create secret tls standard-authservice-cert --cert=standard-authservice-cert/tls.crt --key=standard-authservice-cert/tls.key
  4. Create a certificate to use with the API server. For testing purposes, you can create a self-signed certificate:

    mkdir -p api-server-cert/
    openssl req -new -x509 -batch -nodes -days 11000 -subj "/O=io.enmasse/CN=api-server.enmasse-infra.svc.cluster.local" -out api-server-cert/tls.crt -keyout api-server-cert/tls.key
  5. Create a secret containing the API server certificate:

    kubectl create secret tls api-server-cert --cert=api-server-cert/tls.crt --key=api-server-cert/tls.key
  6. Deploy using the enmasse-with-standard-authservice bundle:

    kubectl apply -f install/bundles/enmasse-with-standard-authservice
Installing EnMasse manually

The manual deployment procedure can be performed on any platform supporting the Kubernetes client.

Creating the project for EnMasse
Procedure
  1. Create the enmasse-infra namespace:

    kubectl create namespace enmasse-infra
  2. Set the enmasse-infra namespace as the default namespace:

    kubectl config set-context $(kubectl config current-context) --namespace=enmasse-infra
Deploying authentication services

EnMasse requires at least one authentication service to be deployed. The authentication service can be none (allow all), standard (Keycloak), or external (not managed by EnMasse).

Deploying the none authentication service
Procedure
  1. Create a certificate to use with the none authentication service. For testing purposes, you can create a self-signed certificate:

    mkdir -p none-authservice-cert
    openssl req -new -x509 -batch -nodes -days 11000 -subj "/O=io.enmasse/CN=none-authservice.enmasse-infra.svc.cluster.local" -out none-authservice-cert/tls.crt -keyout none-authservice-cert/tls.key
  2. Create a secret with the none authentication service certificate:

    kubectl create secret tls none-authservice-cert --cert=none-authservice-cert/tls.crt --key=none-authservice-cert/tls.key
  3. Create the none authentication service:

    kubectl create -f ./install/components/none-authservice
Deploying the standard authentication service
Procedure
  1. (Optional) Replace namespace if you want to deploy to something other than enmasse-infra:

    sed -i 's/enmasse-infra/my-namespace/' install/components/standard-authservice/*RoleBinding*.yaml
  2. Create a certificate to use with the standard authentication service. For testing purposes, you can create a self-signed certificate:

    mkdir -p standard-authservice-cert
    openssl req -new -x509 -batch -nodes -days 11000 -subj "/O=io.enmasse/CN=standard-authservice.enmasse-infra.svc.cluster.local" -out standard-authservice-cert/tls.crt -keyout standard-authservice-cert/tls.key
  3. Create a secret with the standard authentication service certificate:

    kubectl create secret tls standard-authservice-cert --cert=standard-authservice-cert/tls.crt --key=standard-authservice-cert/tls.key
  4. Create the standard authentication service:

    kubectl create -f ./install/components/standard-authservice
Deploying the Address Space Controller

The Address Space Controller is responsible for creating the infrastructure used by address spaces.

Procedure
  1. (Optional) Replace namespace if you want to deploy to something other than enmasse-infra:

    sed -i 's/enmasse-infra/my-namespace/' install/components/address-space-controller/*.yaml
  2. Deploy the Address Space Controller

    kubectl apply -f install/components/address-space-controller
Deploying the API server

The API server provides a REST API for creating address spaces and addresses. It can also serve as an aggregated API server if it is registered as an API service.

Procedure
  1. (Optional) Replace namespace if you want to deploy to something other than enmasse-infra (and substitute enmasse-infra in subsequent steps):

    sed -i 's/enmasse-infra/my-namespace/' install/components/api-service/*.yaml
    sed -i 's/enmasse-infra/my-namespace/' install/components/api-server/*.yaml
  2. Register API service

    kubectl apply -f install/components/api-service
  3. Create a certificate to use with the API server. For testing purposes, you can create a self-signed certificate:

    mkdir -p api-server-cert/
    openssl req -new -x509 -batch -nodes -days 11000 -subj "/O=io.enmasse/CN=api-server.enmasse-infra.svc.cluster.local" -out api-server-cert/tls.crt -keyout api-server-cert/tls.key
  4. Create a secret containing the API server certificate:

    kubectl create secret tls api-server-cert --cert=api-server-cert/tls.crt --key=api-server-cert/tls.key
  5. Deploy the API Server

    kubectl apply -f install/components/api-server/

3.2. Configuring EnMasse

3.2.1. Address space plans

Address space plans are used to configure quotas and control the resources consumed by address spaces. Address space plans are configured by the EnMasse service operator and are selected when creating an address space.

EnMasse includes a default set of plans that are sufficient for most use cases.

Plans are configured as custom resources. The following example shows a plan for the standard address space:

apiVersion: admin.enmasse.io/v1alpha1
kind: AddressSpacePlan
metadata:
  name: restrictive-plan
  labels:
    app: enmasse
  annotations:
    enmasse.io/defined-by: default
displayName: Restrictive Plan
displayOrder: 0
shortDescription: A plan with restrictive quotas
longDescription: A plan with restrictive quotas for the standard address space
uuid: 74b9a40e-117e-11e8-b4e1-507b9def37d9
addressSpaceType: standard
addressPlans:
- small-queue
- small-anycast
resources:
- name: router
  min: 0.0
  max: 2.0
- name: broker
  min: 0.0
  max: 2.0
- name: aggregate
  min: 0.0
  max: 2.0

The following fields are required:

  • metadata.name

  • resources

  • addressPlans

  • addressSpaceType

The other fields are used by the EnMasse Console UI. Note the annotation enmasse.io/defined-by, which points to an infrastructure configuration that must exist when an address space using this plan is created. For more information on infrastructure configurations, see Infrastructure configuration.

Creating address space plans on OpenShift
Procedure
  1. Log in as a service admin:

    oc login -u system:admin
  2. Select the project where EnMasse is installed:

    oc project enmasse-infra
  3. Create an address space plan definition:

    apiVersion: admin.enmasse.io/v1alpha1
    kind: AddressSpacePlan
    metadata:
      name: restrictive-plan
      labels:
        app: enmasse
      annotations:
        enmasse.io/defined-by: default
    displayName: Restrictive Plan
    displayOrder: 0
    shortDescription: A plan with restrictive quotas
    longDescription: A plan with restrictive quotas for the standard address space
    uuid: 74b9a40e-117e-11e8-b4e1-507b9def37d9
    addressSpaceType: standard
    addressPlans:
    - small-queue
    - small-anycast
    resources:
    - name: router
      min: 0.0
      max: 2.0
    - name: broker
      min: 0.0
      max: 2.0
    - name: aggregate
      min: 0.0
      max: 2.0
  4. Create the address space plan:

    oc create -f restrictive-plan.yaml
  5. Verify that schema has been updated and contains the plan:

    oc get addressspaceschema standard -o yaml
Creating address space plans on Kubernetes
Procedure
  1. Create an address space plan definition:

    apiVersion: admin.enmasse.io/v1alpha1
    kind: AddressSpacePlan
    metadata:
      name: restrictive-plan
      labels:
        app: enmasse
      annotations:
        enmasse.io/defined-by: default
    displayName: Restrictive Plan
    displayOrder: 0
    shortDescription: A plan with restrictive quotas
    longDescription: A plan with restrictive quotas for the standard address space
    uuid: 74b9a40e-117e-11e8-b4e1-507b9def37d9
    addressSpaceType: standard
    addressPlans:
    - small-queue
    - small-anycast
    resources:
    - name: router
      min: 0.0
      max: 2.0
    - name: broker
      min: 0.0
      max: 2.0
    - name: aggregate
      min: 0.0
      max: 2.0
  2. Create the address space plan:

    kubectl create -f restrictive-plan.yaml
  3. Verify that schema has been updated and contains the plan:

    kubectl get addressspaceschema standard -o yaml

3.2.2. Address plans

Address plans specify the expected resource usage of a given address. The sum of the resource usage for all resource types determines the amount of infrastructure provisioned for an address space. A single router and broker pod has a maximum usage of one. If a new address requires additional resources and the resource consumption is within the address space plan limits, a new pod will be created automatically to handle the increased load.

Address plans are configured by the EnMasse service operator and are selected when creating an address.

EnMasse includes a default set of address plans that are sufficient for most use cases.

In the Address space plans section, the address space plan references two address plans: small-queue and small-anycast. These address plans are stored as custom resources and are defined as follows:

apiVersion: admin.enmasse.io/v1alpha1
kind: AddressPlan
metadata:
  name: small-queue
  labels:
    app: enmasse
displayName: Small queue plan
displayOrder: 0
shortDescription: A plan for small queues
longDescription: A plan for small queues that consume little resources
uuid: 98feabb6-1183-11e8-a769-507b9def37d9
addressType: queue
requiredResources:
- name: router
  credit: 0.2
- name: broker
  credit: 0.3

The following fields are required:

  • metadata.name

  • requiredResources

  • addressType

A single router can support five instances of addresses and broker can support three instances of addresses with this plan. If the number of addresses with this plan increases to four, another broker is created. If it increases further to six, another router is created as well.

Note, however, that although the address space plan allows two routers and two brokers to be deployed, it only allows two pods to be deployed in total. This means that the address space is restricted to three addresses with the small-queue plan.

The small-anycast plan does not consume any broker resources, and can provision two routers at the expense of not being able to create any brokers:

apiVersion: admin.enmasse.io/v1alpha1
kind: AddressPlan
metadata:
  name: small-anycast
  labels:
    app: enmasse
displayName: Small anycast plan
displayOrder: 0
shortDescription: A plan for small anycast addresses
longDescription: A plan for small anycast addresses that consume little resources
uuid: cb61f440-1184-11e8-adda-507b9def37d9
addressType: anycast
requiredResources:
- name: router
  credit: 0.2

With this plan, up to 10 addresses can be created.

Creating address plans on OpenShift
Procedure
  1. Log in as a service admin:

    oc login -u system:admin
  2. Select the project where EnMasse is installed:

    oc project enmasse-infra
  3. Create an address plan definition:

    apiVersion: admin.enmasse.io/v1alpha1
    kind: AddressPlan
    metadata:
      name: small-anycast
      labels:
        app: enmasse
    displayName: Small anycast plan
    displayOrder: 0
    shortDescription: A plan for small anycast addresses
    longDescription: A plan for small anycast addresses that consume little resources
    uuid: cb61f440-1184-11e8-adda-507b9def37d9
    addressType: anycast
    requiredResources:
    - name: router
      credit: 0.2
  4. Create the address plan:

    oc create -f small-anycast-plan.yaml
  5. Verify that schema has been updated and contains the plan:

    oc get addressspaceschema standard -o yaml
Creating address plans on Kubernetes
Procedure
  1. Create an address plan definition:

    apiVersion: admin.enmasse.io/v1alpha1
    kind: AddressPlan
    metadata:
      name: small-anycast
      labels:
        app: enmasse
    displayName: Small anycast plan
    displayOrder: 0
    shortDescription: A plan for small anycast addresses
    longDescription: A plan for small anycast addresses that consume little resources
    uuid: cb61f440-1184-11e8-adda-507b9def37d9
    addressType: anycast
    requiredResources:
    - name: router
      credit: 0.2
  2. Create the address plan:

    kubectl create -f small-anycast-plan.yaml
  3. Verify that schema has been updated and contains the plan:

    kubectl get addressspaceschema standard -o yaml

3.2.3. Infrastructure configuration

EnMasse creates infrastructure components such as routers, brokers, and consoles. These components can be configured while the system is running, and EnMasse automatically updates the components with the new settings. The EnMasse service operator can edit the EnMasse default infrastructure configuration or create new configurations.

Infrastructure configurations can be referred to from one or more address space plans. For more information about address space plans, see Address space plans.

Infrastructure configuration can be managed for both brokered and standard infrastructure using BrokeredInfraConfig and StandardInfraConfig resources.

Brokered infrastructure configuration

BrokeredInfraConfig resources is used to configure infrastructure deployed by brokered address spaces. The brokered infra configuration is referenced by address space plans using a enmasse.io/defined-by annotation. For more information, see Address space plans.

apiVersion: admin.enmasse.io/v1alpha1
kind: BrokeredInfraConfig
metadata:
  name: brokered-infra-config-example
spec:
  version: 0.23.0
  admin:
    resources:
      memory: 256Mi
  broker:
    resources:
      memory: 2Gi
      storage: 100Gi
    addressFullPolicy: PAGE

The version field specifies the EnMasse version used. When upgrading, EnMasse uses this field whether to upgrade the infrastructure to the requested version.

The admin object specifies the settings you can configure for the admin components.

The broker object specifies the settings you can configure for the broker components. Changing the .broker.resources.storage setting does not configure existing broker storage size.

Standard infrastructure configuration

StandardInfraConfig resources is used to configure infrastructure deployed by standard address spaces. The standard infra configuration is referenced by address space plans using a enmasse.io/defined-by annotation. For more information, see Address space plans.

apiVersion: admin.enmasse.io/v1alpha1
kind: StandardInfraConfig
metadata:
  name: myconfig
spec:
  version: 0.23.0
  admin:
    resources:
      memory: 256Mi
  broker:
    resources:
      memory: 2Gi
      storage: 100Gi
    addressFullPolicy: PAGE
  router:
    resources:
      memory: 256Mi
    linkCapcity: 1000

The version field specifies the EnMasse version used. When upgrading, EnMasse uses this field whether to upgrade the infrastructure to the requested version.

The admin object specifies the settings you can configure for the admin components.

The broker object specifies the settings you can configure for the broker components. Changing the .broker.resources.storage setting does not configure existing broker storage size.

The router object specifies the settings you can configure for the router components.

Applying infrastructure configuration on OpenShift

You can edit existing configurations or create new ones.

Prerequisites
Procedure
  1. Log in as a service operator:

    oc login -u developer
  2. Select the project where EnMasse is installed:

    oc project enmasse
  3. Create infrastructure configuration:

    apiVersion: admin.enmasse.io/v1alpha1
    kind: StandardInfraConfig
    metadata:
      name: myconfig
    spec:
      version: 0.23.0
      admin:
        resources:
          memory: 256Mi
      broker:
        resources:
          memory: 2Gi
          storage: 100Gi
        addressFullPolicy: PAGE
      router:
        resources:
          memory: 256Mi
        linkCapcity: 1000
  4. Apply the configuration changes:

    oc apply -f standard-infra-config-example.yaml
  5. Monitor the pods while they are restarted:

    oc get pods -w

    The configuration changes will be applied within a couple of minutes.

Applying infrastructure configuration on Kubernetes

You can edit existing configurations or create new ones.

Prerequisites
Procedure
  1. Create infrastructure configuration:

    apiVersion: admin.enmasse.io/v1alpha1
    kind: StandardInfraConfig
    metadata:
      name: myconfig
    spec:
      version: 0.23.0
      admin:
        resources:
          memory: 256Mi
      broker:
        resources:
          memory: 2Gi
          storage: 100Gi
        addressFullPolicy: PAGE
      router:
        resources:
          memory: 256Mi
        linkCapcity: 1000
  2. Apply the configuration changes:

    kubectl apply -f standard-infra-config-example.yaml
  3. Monitor the pods while they are restarted:

    kubectl get pods -w

    The configuration changes will be applied within a couple of minutes.

3.3. Upgrading EnMasse

3.3.1. Upgrading EnMasse on OpenShift

EnMasse supports upgrades between minor versions using cloud native tools and the same mechanism used to apply configuration changes. When upgrading, the updated infrastructure configuration of the new version will trigger the upgrade to start.

Upgrading EnMasse is done by applying the YAML files for the new version.

Upgrading EnMasse on OpenShift using a YAML bundle
Prerequisites
Procedure
  1. Log in as a service operator:

    oc login -u system:admin
  2. Select the project where EnMasse is installed:

    oc project enmasse-infra
  3. Apply the new release bundle:

    oc apply -f install/bundles/enmasse-with-standard-authservice
  4. Monitor pods while they are restarted:

    oc get pods -w

    The upgrade should cause all pods to be restarted within a couple of minutes.

Upgrading EnMasse on OpenShift using Template
Prerequisites
Procedure
  1. Log in as a service operator:

    oc login -u system:admin
  2. Select the project where EnMasse is installed:

    oc project enmasse-infra
  3. Apply the new release template:

    oc process -f install/templates/enmasse-with-standard-authservice.yaml NAMESPACE=enmasse-infra | oc apply -f -
  4. Monitor pods while they are restarted:

    oc get pods -w

    The upgrade should cause all pods to be restarted within a couple of minutes.

Upgrading EnMasse on OpenShift using Ansible
Prerequisites
Procedure
  1. Log in as a service operator:

    oc login -u system:admin
  2. Run the Ansible playbook from the new release:

    ansible-playbook -i inventory-file ansible/playbooks/openshift/deploy_all.yml
  3. Monitor pods while they are restarted:

    oc get pods -w

    The pods restart and become active within several minutes.

3.3.2. Upgrading EnMasse on OpenShift

EnMasse supports upgrades between minor versions using cloud native tools and the same mechanism used to apply configuration changes. When upgrading, the updated infrastructure configuration of the new version will trigger the upgrade to start.

Upgrading EnMasse is done by applying the YAML files for the new version.

Upgrading EnMasse on Kubernetes using a YAML bundle
Prerequisites
Procedure
  1. Select the namespace where EnMasse is installed:

    kubectl config set-context $(kubectl config current-context) --namespace=enmasse-infra
  2. Apply the new release bundle:

    kubectl apply -f install/bundles/enmasse-with-standard-authservice
  3. Monitor pods while they are restarted:

    kubectl get pods -w

    The upgrade should cause all pods to be restarted within a couple of minutes.

3.4. Monitoring EnMasse

3.4.1. Monitoring EnMasse on OpenShift

Monitoring services run frequent health checks on EnMasse and send alerts when health checks fail. Health is assessed using Prometheus and kube-state-metrics. Alerting is implemented with Alertmanager. Grafana is also configured to provide a dashboard of the current status of health checks.

Deploying Prometheus
Procedure
  1. Replace the namespace with the namespace EnMasse is currently deployed to:

    sed -i 's/enmasse-infra/_my-namespace_/' install/components/prometheus/*.yaml
  2. Create the Prometheus deployment:

    oc apply -f ./install/components/prometheus
Deploying kube-state-metrics
Procedure
  1. Replace the namespace with the namespace EnMasse is currently deployed to:

    sed -i 's/enmasse-infra/_my-namespace_/' install/components/kube-state-metrics/*.yaml
  2. Create the kube-state-metrics deployment:

    oc apply -f ./install/components/kube-state-metrics
Deploying Alertmanager
Prerequisites
  • You must have already installed Prometheus and kube-state-metrics.

Procedure
  1. Replace the namespace with the namespace EnMasse is currently deployed to:

    sed -i 's/enmasse-infra/_my-namespace_/' install/components/alertmanager/*.yaml
  2. Create the Alertmanager deployment:

    oc apply -f ./install/components/alertmanager
  3. (Optional) Configure Alertmanager to send emails using a custom configmap with the relevant SMTP details:

    oc apply -f ./alertmanager-configmap.yaml

    An example Alertmanager config map:

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app: enmasse
  name: alertmanager-config
data:
  alertmanager.yml: |
    global:
      resolve_timeout: 5m
      smtp_smarthost: localhost
      smtp_from: alerts@localhost
      smtp_auth_username: admin
      smtp_auth_password: password
    route:
      group_by: ['alertname']
      group_wait: 60s
      group_interval: 60s
      repeat_interval: 1h
      receiver: 'sysadmins'
    receivers:
    - name: 'sysadmins'
      email_configs:
      - to: sysadmin@localhost
    inhibit_rules:
      - source_match:
          severity: 'critical'
        target_match:
          severity: 'warning'
        equal: ['alertname']
Deploying Grafana
Prerequisites
  • You must have already installed Prometheus and kube-state-metrics.

Procedure
  1. Replace the namespace with the namespace {ProcuctName} is currently deployed to:

    sed -i 's/enmasse-infra/_my-namespace_/' install/components/grafana/*.yaml
  2. Create the Grafana deployment:

    oc apply -f ./install/components/grafana
Restarting a component
Procedure
  • Delete the component’s pod:

    oc delete pod -l name=component

The component will be automatically restarted in a new pod.

Viewing router logs
Procedure
  1. List all router pods and choose the pod for the relevant address space:

    oc get pods -l name=qdrouterd -o go-template --template '{{range .items}}{{.metadata.name}}{{"\t"}}{{.metadata.annotations.addressSpace}}{{"\n"}}{{end}}'
  2. Display the logs for the pod:

    oc logs pod -c router
Viewing broker logs
Procedure
  1. List all broker pods and choose the pod for the relevant address space:

    oc get pods -l name=broker -o go-template --template '{{range .items}}{{.metadata.name}}{{"\t"}}{{.metadata.annotations.addressSpace}}{{"\n"}}{{end}}'
  2. Display the logs for the pod:

    oc logs pod
Viewing message and connection statistics using the EnMasse Console
Prerequisites
  • You must be logged into the EnMasse Console.

Table 2. Message statistics reference table
To view…​ On the Addresses page see…​

Address status

The first column (the symbol preceding the address name)

Address type

The third column

Address plan

The fourth column

Message ingress rate (during the last 5 minutes)

Messages In

Message egress rate (during the last 5 minutes)

Messages Out

Number of senders attached

Senders

Number of receivers attached

Receivers

Queue and topic address types only: Number of stored messages on the broker or brokers

Stored

Standard address space only: Message deliveries per second

For the desired address, expand the twisty on the left to show the Senders table; see the Delivery Rate column.

Table 3. Connection statistics reference table
To view…​ On the Connections page see…​

Total number of messages received as long the connection has existed

Messages In

Standard address space only: Total number of messages sent as long the connection has existed

Messages Out

Total number of messages delivered

For the desired connection, expand the twisty on the left to show the Senders and Receivers tables; see the Deliveries columns.

Standard address space only: Username used by the client to connect

The third column

3.4.2. Monitoring EnMasse on Kubernetes

Monitoring services run frequent health checks on EnMasse and send alerts when health checks fail. Health is assessed using Prometheus and kube-state-metrics. Alerting is implemented with Alertmanager. Grafana is also configured to provide a dashboard of the current status of health checks.

Deploying Prometheus
Procedure
  1. Replace the namespace with the namespace EnMasse is currently deployed to:

    sed -i 's/enmasse-infra/_my-namespace_/' install/components/prometheus/*.yaml
  2. Create the Prometheus deployment:

    kubectl apply -f ./install/components/prometheus
Deploying kube-state-metrics
Procedure
  1. Replace the namespace with the namespace EnMasse is currently deployed to:

    sed -i 's/enmasse-infra/_my-namespace_/' install/components/kube-state-metrics/*.yaml
  2. Create the kube-state-metrics deployment:

    kubectl apply -f ./install/components/kube-state-metrics
Deploying Alertmanager
Prerequisites
  • You must have already installed Prometheus and kube-state-metrics.

Procedure
  1. Replace the namespace with the namespace EnMasse is currently deployed to:

    sed -i 's/enmasse-infra/_my-namespace_/' install/components/alertmanager/*.yaml
  2. Create the Alertmanager deployment:

    kubectl apply -f ./install/components/alertmanager
  3. (Optional) Configure Alertmanager to send emails using a custom configmap with the relevant SMTP details:

    kubectl apply -f ./alertmanager-configmap.yaml

    An example Alertmanager config map:

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app: enmasse
  name: alertmanager-config
data:
  alertmanager.yml: |
    global:
      resolve_timeout: 5m
      smtp_smarthost: localhost
      smtp_from: alerts@localhost
      smtp_auth_username: admin
      smtp_auth_password: password
    route:
      group_by: ['alertname']
      group_wait: 60s
      group_interval: 60s
      repeat_interval: 1h
      receiver: 'sysadmins'
    receivers:
    - name: 'sysadmins'
      email_configs:
      - to: sysadmin@localhost
    inhibit_rules:
      - source_match:
          severity: 'critical'
        target_match:
          severity: 'warning'
        equal: ['alertname']
Deploying Grafana
Prerequisites
  • You must have already installed Prometheus and kube-state-metrics.

Procedure
  1. Replace the namespace with the namespace {ProcuctName} is currently deployed to:

    sed -i 's/enmasse-infra/_my-namespace_/' install/components/grafana/*.yaml
  2. Create the Grafana deployment:

    kubectl apply -f ./install/components/grafana
Restarting a component
Procedure
  • Delete the component’s pod:

    kubectl delete pod -l name=component

The component will be automatically restarted in a new pod.

Viewing router logs
Procedure
  1. List all router pods and choose the pod for the relevant address space:

    kubectl get pods -l name=qdrouterd -o go-template --template '{{range .items}}{{.metadata.name}}{{"\t"}}{{.metadata.annotations.addressSpace}}{{"\n"}}{{end}}'
  2. Display the logs for the pod:

    kubectl logs pod -c router
Viewing broker logs
Procedure
  1. List all broker pods and choose the pod for the relevant address space:

    kubectl get pods -l name=broker -o go-template --template '{{range .items}}{{.metadata.name}}{{"\t"}}{{.metadata.annotations.addressSpace}}{{"\n"}}{{end}}'
  2. Display the logs for the pod:

    kubectl logs pod

3.4.3. Using qdstat

You can use qdstat to monitor the EnMasse service.

Viewing router connections using qdstat

You can view the router connections using qdstat.

Procedure
  • On the command line, run the following command:

qdstat -c

Connections
  id   host                 container                             role    dir  security                              authentication                tenant
  =========================================================================================================================================================
  3    172.17.0.9:34998     admin-78794c68c8-9jdd6                normal  in   TLSv1.2(ECDHE-RSA-AES128-GCM-SHA256)  CN=admin,O=io.enmasse(x.509)
  12   172.30.188.174:5671  27803a14-42d2-6148-9491-a6c1e69e875a  normal  out  TLSv1.2(ECDHE-RSA-AES128-GCM-SHA256)  x.509
  567  127.0.0.1:43546      b240c652-82df-48dd-b54e-3b8bbaef16c6  normal  in   no-security                           PLAIN
Viewing router addresses using qdstat

You can view the router addresses using qdstat.

Procedure
  • On the command line, run the following command:

qdstat -a

Router Addresses
  class     addr                   phs  distrib       in-proc  local  remote  cntnr  in     out    thru  to-proc  from-proc
  ===========================================================================================================================
  local     $_management_internal       closest       1        0      0       0      0      0      0     588      588
  link-in   $lwt                        linkBalanced  0        0      0       0      0      0      0     0        0
  link-out  $lwt                        linkBalanced  0        0      0       0      0      0      0     0        0
  mobile    $management            0    closest       1        0      0       0      601    0      0     601      0
  local     $management                 closest       1        0      0       0      2,925  0      0     2,925    0
  local     qdhello                     flood         1        0      0       0      0      0      0     0        5,856
  local     qdrouter                    flood         1        0      0       0      0      0      0     0        0
  topo      qdrouter                    flood         1        0      0       0      0      0      0     0        196
  local     qdrouter.ma                 multicast     1        0      0       0      0      0      0     0        0
  topo      qdrouter.ma                 multicast     1        0      0       0      0      0      0     0        0
  local     temp.VTXOKyyWsq7OEei        balanced      0        1      0       0      0      0      0     0        0
  local     temp.k2RGQNPe6sDMvz4        balanced      0        1      0       0      0      3,511  0     0        3,511
  local     temp.xg+y8I_Tr4Y94LA        balanced      0        1      0       0      0      5      0     0        5

You can view the router links using qdstat.

Procedure
  • On the command line, run the following command:

qdstat -l

Router Links
  type      dir  conn id  id  peer  class   addr                  phs  cap  undel  unsett  del   presett  psdrop  acc   rej  rel  mod  admin    oper
  ====================================================================================================================================================
  endpoint  in   3        8                                            250  0      0       3829  0        0       3829  0    0    0    enabled  up
  endpoint  out  3        9         local   temp.k2RGQNPe6sDMvz4       250  0      0       3829  3829     0       0     0    0    0    enabled  up
  endpoint  in   12       10                                           250  0      0       5     0        0       5     0    0    0    enabled  up
  endpoint  out  12       11        local   temp.xg+y8I_Tr4Y94LA       250  0      0       5     5        0       0     0    0    0    enabled  up
  endpoint  in   645      26        mobile  $management           0    50   0      0       1     0        0       1     0    0    0    enabled  up
  endpoint  out  645      27        local   temp.0BrHJ1O+fi6whyg       50   0      0       0     0        0       0     0    0    0    enabled  up

You can view the link routes using qdstat.

Procedure
  • On the command line, run the following command:

qdstat --linkroutes

Link Routes
  address  dir  distrib       status
  ======================================
  $lwt     in   linkBalanced  inactive
  $lwt     out  linkBalanced  inactive

4. Tenant Guide

The tenant guide provides resources on how to manage address spaces, addresses,and users as a messaging tenant.

4.1. Address space

An address space is a group of addresses that can be accessed through a single connection (per protocol). This means that clients connected to the endpoints of an address space can send messages to or receive messages from any authorized address within that address space. An address space can support multiple protocols, as defined by the address space type.

EnMasse has two types of address spaces:

  • Standard

  • Brokered

4.2. Address

An address is part of an address space and represents a destination for sending and receiving messages. An address has a type, which defines the semantics of sending messages to and receiving messages from that address.

The types of addresses available in EnMasse depend on the address space type.

4.3. Standard address space

The standard address space is the default address space in EnMasse. It consists of an AMQP router network in combination with attachable storage units. Clients connect to a message router, which forwards messages to or from one or more message brokers. This address space type is appropriate when you have many connections and addresses. However, the standard address space has the following limitations:

  • No transaction support

  • No message ordering

  • No selectors on queues

  • No browsing on queues

  • No message groups

Clients connect and send and receive messages in this address space using the AMQP or MQTT protocols. Note that MQTT does not support qos2 or retained messages.

4.3.1. Standard address types

The standard address space supports five different address types:

  • queue

  • topic

  • anycast

  • multicast

  • subscription

Queue

The queue address type is a store-and-forward queue. This address type is appropriate for implementing a distributed work queue, handling traffic bursts, and other use cases where you want to decouple the producer and consumer. A queue can be sharded across multiple storage units; however, message order is no longer guaranteed.

Topic

The topic address type supports the publish-subscribe messaging pattern where there are 1..N producers and 1..M consumers. Each message published to a topic address is forwarded to all subscribers for that address. A subscriber can also be durable, in which case messages are kept until the subscriber has acknowledged them.

Hierarchical topics and wildcards

A client receiving from a topic address can specify a wildcard address with the topic address as the root. The wildcard behavior follows the MQTT syntax:

  • / is a separator

  • + matches one level

  • # matches one or more levels

So, for example:

  • a/#/b matches a/foo/b, a/bar/b, and a/foo/bar/b

  • a/+/b matches a/foo/b and a/bar/b, but would not match a/foo/bar

In the standard address space, the first level must always be a defined topic address; that is, # and + are not valid as the first characters of a subscribing address.

Anycast

The anycast address type is a scalable direct address for sending messages to one consumer. Messages sent to an anycast address are not stored, but are instead forwarded directly to the consumer. This method makes this address type ideal for request-reply (RPC) uses or even work distribution. This is the cheapest address type as it does not require any persistence.

Multicast

The multicast address type is a scalable direct address for sending messages to multiple consumers. Messages sent to a multicast address are forwarded to all consumers receiving messages on that address. Because message acknowledgments from consumers are not propagated to producers, only pre-settled messages can be sent to multicast addresses.

Subscription

The subscription address type allows a subscription to be created for a topic that holds messages published to the topic even if the subscriber is not attached. The subscription is accessed by the consumer using <topic-address>::<subscription-address>. For example, for a subscription mysub on a topic mytopic the consumer consumes from the address mytopic::mysub.

4.4. Brokered address space

The brokered address space is designed to support broker-specific features, at the cost of limited scale in terms of the number of connections and addresses. This address space supports JMS transactions, message groups, and selectors on queues and topics.

Clients can connect and send and receive messages in this address space using the AMQP, CORE, OpenWire, and MQTT protocols.

4.4.1. Brokered address types

The brokered address space supports two address types:

  • queue

  • topic

Queue

The queue address type is a store-and-forward queue. This address type is appropriate for implementing a distributed work queue, handling traffic bursts, and other use cases where you want to decouple the producer and consumer. A queue in the brokered address spaces supports selectors, message groups, transactions, and other JMS features.

Topic

The topic address type supports the publish-subscribe messaging pattern in which there are 1..N producers and 1..M consumers. Each message published to a topic address is forwarded to all subscribers for that address. A subscriber can also be durable, in which case messages are kept until the subscriber has acknowledged them.

Hierarchical topics and wildcards

A client receiving from a topic address can specify a wildcard address with the topic address as the root. The wildcard behavior follows the MQTT syntax:

  • / is a separator

  • + matches one level

  • # matches one or more levels

So, for example:

  • a/#/b matches a/foo/b, a/bar/b, a/foo/bar/b

  • a/+/b matches a/foo/b and a/bar/b, but would not match a/foo/bar

4.5. Managing address spaces on OpenShift

EnMasse is configured to support managing address spaces using the Kubernetes and OpenShift command-line tools.

4.5.1. Minimal example

The minimal example shows the smallest set of options needed to create an AddressSpace.

apiVersion: enmasse.io/v1alpha1
kind: AddressSpace
metadata:
  name: myspace
spec:
  type: standard (1)
  plan: standard-unlimited (2)
  1. The address space type can be either brokered or standard.

  2. The address space plan depends on the address space type and what has been configured by the EnMasse admin.

4.5.2. Advanced example

The advanced example show how you can configure the authentication service and endpoints of an AddressSpace.

apiVersion: enmasse.io/v1alpha1
kind: AddressSpace
metadata:
  name: myspace
spec:
  type: standard (1)
  plan: standard-unlimited (2)
  authenticationService:
    type: standard (3)
  endpoints: (4)
    - name: messaging
      service: messaging
      expose: (5)
        type: route
        routeHost: messaging.example.com
        routeTlsTermination: passthrough
        routeServicePort: amqps
      cert: (6)
        provider: certBundle
        tlsKey: ZXhhbXBsZSAtbgo=
        tlsCert: ZXhhbXBsZSAtbgo=
    - name: mqtt
      service: mqtt
      expose:
        type: loadbalancer
        loadBalancerPorts:
        - mqtt
        - mqtts
      cert:
        provider: selfsigned
    - name: console
      service: console
      expose:
        type: route
        routeTlsTermination: reencrypt
        routeServicePort: https
      cert:
        provider: openshift
  1. The address space type can be either brokered or standard.

  2. The address space plan depends on the address space type and what has been configured by the EnMasse admin.

  3. The authentication service type can be none, standard or external. External requires an additional field, details, including host and port information.

  4. The endpoints that must be configured for the AddressSpace. Endpoints can specity either messaging, console or mqtt services. However, the mqtt service is supported for the standard address space type only.

  5. Expose the configuration of an endpoint. Endpoints can be kept internal to the cluster, or exposed using an OpenShift route or a load-balancer service.

  6. The certificate configuration can be used to specify how the endpoint certificate is provided. The provider can be selfsigned (default), wildcard (requires it to be enabled by the EnMasse admin), openshift (signed by the OpenShift cluster CA), or certBundle (base64-encoded PEM key and certificate).

4.5.3. Example address space status output

The AddressSpace resource contains a status field that can be used to retrieve information about its state and endpoints. The following output is an example of the output you can get from running oc get addressspace myspace -o yaml:

apiVersion: enmasse.io/v1alpha1
kind: AddressSpace
metadata:
  name: myspace
spec:
  ...
status:
  isReady: false (1)
  messages:
    - "One or more deployments are not ready: "
  endpointStatuses: (2)
    - name: messaging
      cert: aGVsbG8= (3)
      serviceHost: messaging-123.enmasse-infra.svc (4)
      servicePorts: (5)
        - name: amqp
          port: 5672
        - name: amqps
          port: 5671
      externalHost: messaging.example.com (6)
      externalPorts: (7)
        - name: amqps
          port: 443
  1. The status.isReady field can be either true or false.

  2. The status.endpointStatuses field provides information about available endpoints for this address space.

  3. The cert field contains the base64-encoded certificate for a given endpoint.

  4. The serviceHost field contains the cluster-internal host name for a given endpoint.

  5. The servicePorts field contains the available ports for the cluster-internal host.

  6. The externalHost field contains the external host name for a given endpoint.

  7. The externalPorts field contains the available ports for the external host.

4.5.4. Creating address spaces using the command line

Procedure
  1. Log in as a messaging tenant:

    oc login -u developer
  2. Create the project for the messaging application:

    oc new-project myapp
  3. Create an address space definition:

    apiVersion: enmasse.io/v1alpha1
    kind: AddressSpace
    metadata:
      name: myspace
    spec:
      type: standard
      plan: standard-unlimited
  4. Create the address space:

    oc create -f standard-address-space.yaml
  5. Check the status of the address space:

    oc get addressspace myspace -o jsonpath={.status.isReady}

    The address space is ready for use when the above command outputs true.

4.5.5. Example commands for retrieving address space information

  • Retrieving the EnMasse Console host name:

    oc get addressspace myspace -o jsonpath={.status.endpointStatuses[?(@.name==\'console\')].externalHost}
  • Retrieving the status of an address space:

    oc get addressspace myspace -o jsonpath={.status.isReady}
  • Retrieving the base64-encoded PEM certificate for the messaging endpoint:

    oc get addressspace myspace -o jsonpath={.status.endpointStatuses[?(@.name==\'messaging\')].cert
  • Retrieving the host name for the messaging endpoint:

    oc get addressspace myspace -o jsonpath={.status.endpointStatuses[?(@.name==\'messaging\')].externalHost

4.6. Managing address spaces on Kubernetes

EnMasse is configured to support managing address spaces using the Kubernetes and OpenShift command-line tools.

4.6.1. Minimal example

The minimal example shows the smallest set of options needed to create an AddressSpace.

apiVersion: enmasse.io/v1alpha1
kind: AddressSpace
metadata:
  name: myspace
spec:
  type: standard (1)
  plan: standard-unlimited (2)
  1. The address space type can be either brokered or standard.

  2. The address space plan depends on the address space type and what has been configured by the EnMasse admin.

4.6.2. Advanced example

The advanced example show how you can configure the authentication service and endpoints of an AddressSpace.

apiVersion: enmasse.io/v1alpha1
kind: AddressSpace
metadata:
  name: myspace
spec:
  type: standard (1)
  plan: standard-unlimited (2)
  authenticationService:
    type: standard (3)
  endpoints: (4)
    - name: messaging
      service: messaging
      expose: (5)
        type: route
        routeHost: messaging.example.com
        routeTlsTermination: passthrough
        routeServicePort: amqps
      cert: (6)
        provider: certBundle
        tlsKey: ZXhhbXBsZSAtbgo=
        tlsCert: ZXhhbXBsZSAtbgo=
    - name: mqtt
      service: mqtt
      expose:
        type: loadbalancer
        loadBalancerPorts:
        - mqtt
        - mqtts
      cert:
        provider: selfsigned
    - name: console
      service: console
      expose:
        type: route
        routeTlsTermination: reencrypt
        routeServicePort: https
      cert:
        provider: openshift
  1. The address space type can be either brokered or standard.

  2. The address space plan depends on the address space type and what has been configured by the EnMasse admin.

  3. The authentication service type can be none, standard or external. External requires an additional field, details, including host and port information.

  4. The endpoints that must be configured for the AddressSpace. Endpoints can specity either messaging, console or mqtt services. However, the mqtt service is supported for the standard address space type only.

  5. Expose the configuration of an endpoint. Endpoints can be kept internal to the cluster, or exposed using an OpenShift route or a load-balancer service.

  6. The certificate configuration can be used to specify how the endpoint certificate is provided. The provider can be selfsigned (default), wildcard (requires it to be enabled by the EnMasse admin), openshift (signed by the OpenShift cluster CA), or certBundle (base64-encoded PEM key and certificate).

4.6.3. Example address space status output

The AddressSpace resource contains a status field that can be used to retrieve information about its state and endpoints. The following output is an example of the output you can get from running oc get addressspace myspace -o yaml:

apiVersion: enmasse.io/v1alpha1
kind: AddressSpace
metadata:
  name: myspace
spec:
  ...
status:
  isReady: false (1)
  messages:
    - "One or more deployments are not ready: "
  endpointStatuses: (2)
    - name: messaging
      cert: aGVsbG8= (3)
      serviceHost: messaging-123.enmasse-infra.svc (4)
      servicePorts: (5)
        - name: amqp
          port: 5672
        - name: amqps
          port: 5671
      externalHost: messaging.example.com (6)
      externalPorts: (7)
        - name: amqps
          port: 443
  1. The status.isReady field can be either true or false.

  2. The status.endpointStatuses field provides information about available endpoints for this address space.

  3. The cert field contains the base64-encoded certificate for a given endpoint.

  4. The serviceHost field contains the cluster-internal host name for a given endpoint.

  5. The servicePorts field contains the available ports for the cluster-internal host.

  6. The externalHost field contains the external host name for a given endpoint.

  7. The externalPorts field contains the available ports for the external host.

4.6.4. Creating address spaces using the command line

Procedure
  1. Create an address space definition:

    apiVersion: enmasse.io/v1alpha1
    kind: AddressSpace
    metadata:
      name: myspace
    spec:
      type: standard
      plan: standard-unlimited
  2. Create the address space:

    kubectl create -f standard-address-space.yaml
  3. Check the status of the address space:

    kubectl get addressspace myspace -o jsonpath={.status.isReady}

    The address space is ready for use when the above command outputs true.

4.6.5. Example commands for retrieving address space information

  • Retrieving the EnMasse Console host name:

    kubectl get addressspace myspace -o jsonpath={.status.endpointStatuses[?(@.name==\'console\')].externalHost}
  • Retrieving the status of an address space:

    kubectl get addressspace myspace -o jsonpath={.status.isReady}
  • Retrieving the base64-encoded PEM certificate for the messaging endpoint:

    kubectl get addressspace myspace -o jsonpath={.status.endpointStatuses[?(@.name==\'messaging\')].cert
  • Retrieving the host name for the messaging endpoint:

    kubectl get addressspace myspace -o jsonpath={.status.endpointStatuses[?(@.name==\'messaging\')].externalHost

4.7. Managing addresses on OpenShift

EnMasse is configured to support managing address using the OpenShift command-line tools and the EnMasse Console.

4.7.1. Example

apiVersion: enmasse.io/v1alpha1
kind: Address
metadata:
    name: myspace.myqueue (1)
spec:
    address: myqueue (2)
    type: queue (3)
    plan: standard-small-queue (4)
  1. The address name must be prefixed with the address space name and a dot. Address names can only include alphanumeric characters.

  2. The address is the messaging address this address resource represents.

  3. The address type dictates the semantics of this address.

  4. The address plan describes the resource usage for the address.

4.7.2. Creating addresses using the command line

Procedure
  1. Create an address definition:

    apiVersion: enmasse.io/v1alpha1
    kind: Address
    metadata:
        name: myspace.myqueue
    spec:
        address: myqueue
        type: queue
        plan: standard-small-queue
    Note
    Prefixing the name with the address space name is required to ensure addresses from different address spaces do not collide.
  2. Create the address:

    oc create -f standard-small-queue.yaml
  3. List the addresses:

    oc get addresses -o yaml

4.7.3. Creating addresses using the EnMasse Console

You can create new addresses using the EnMasse Console. The type of addresses that you can create are determined by the type of address space.

Prerequisites
Procedure
  1. Log in to the EnMasse Console. For more information, see Accessing the EnMasse Console.

  2. Click New. The Create new address window opens.

  3. Type a name and select the address type. If selecting subscription, from the Topic list select the topic name to which you want to create a subscription.

  4. Click Next.

  5. Select a plan and click Next.

  6. Click Create. Your address is displayed in the EnMasse Console.

4.8. Managing addresses on Kubernetes

EnMasse is configured to support managing address using the Kubernetes command-line tools and the EnMasse Console.

4.8.1. Example

apiVersion: enmasse.io/v1alpha1
kind: Address
metadata:
    name: myspace.myqueue (1)
spec:
    address: myqueue (2)
    type: queue (3)
    plan: standard-small-queue (4)
  1. The address name must be prefixed with the address space name and a dot. Address names can only include alphanumeric characters.

  2. The address is the messaging address this address resource represents.

  3. The address type dictates the semantics of this address.

  4. The address plan describes the resource usage for the address.

4.8.2. Creating addresses using the command line

Procedure
  1. Create an address definition:

    apiVersion: enmasse.io/v1alpha1
    kind: Address
    metadata:
        name: myspace.myqueue
    spec:
        address: myqueue
        type: queue
        plan: standard-small-queue
    Note
    Prefixing the name with the address space name is required to ensure addresses from different address spaces do not collide.
  2. Create the address:

    kubectl create -f standard-small-queue.yaml
  3. List the addresses:

    kubectl get addresses -o yaml

4.8.3. Creating addresses using the EnMasse Console

You can create new addresses using the EnMasse Console. The type of addresses that you can create are determined by the type of address space.

Prerequisites
Procedure
  1. Log in to the EnMasse Console. For more information, see Accessing the EnMasse Console.

  2. Click New. The Create new address window opens.

  3. Type a name and select the address type. If selecting subscription, from the Topic list select the topic name to which you want to create a subscription.

  4. Click Next.

  5. Select a plan and click Next.

  6. Click Create. Your address is displayed in the EnMasse Console.

4.9. Accessing the EnMasse Console

Prerequisites
Procedure
  1. In a web browser, navigate to https://console-host-name where console-host-name is the EnMasse Console host name.

  2. Log in with your OpenShift user credentials.

4.10. Using the EnMasse Console address filtering

The address name filtering feature in the EnMasse Console uses regular expressions. Also, filters are cumulative.

Table 4. EnMasse Console address name filtering behavior
To match…​ Use…​ Results in…​

The beginning of an expression only

A caret followed by an expression: ^my

All addresses beginning with my

An expression

The matching string: my

All addresses containing my

The end of an expression only

An expression followed by the dollar sign: my$

All addresses ending with my

An exact expression

A caret followed by an expression and a dollar sign: ^my$

Only the address my

4.11. User model

You can create users to access messaging clients and the EnMasse Console through user definition files, which specify the authentication type and authorization policies.

Users are configured as MessagingUser resources. Users can only be created, deleted, read, updated and listed.

The following example shows the user-example1.yaml file:

apiVersion: user.enmasse.io/v1alpha1
kind: MessagingUser
metadata:
  name: myspace.user1
spec:
  username: user1
  authentication:
    type: password
    password: cGFzc3dvcmQ= # Base64 encoded
  authorization:
    - addresses: ["myqueue", "queue1", "queue2", "topic*"]
      operations: ["send", "recv"]
    - addresses: ["anycast1"]
      operations: ["send"]

The following fields are required:

  • metadata.name

  • metadata.namespace

  • spec.authentication

  • spec.authorization

The spec.authentication object defines how the user is authenticated, whereas spec.authorization defines the authorization policies for that user.

4.11.1. Authentication

The supported values for the authentication type are password and federated.

Note
In the default configuration, password authentication type users can only access messaging endpoints, whereas federated authentication type users can only access the EnMasse Console.
Password authentication type

For the password type, an additional field password must be set to a base64-encoded value of the password for that user. The password will not be printed out when reading the resource.

A password can be base64-encoded on the command line. To encode my-password, for example:

$ echo -n my-password | base64
bXktcGFzc3dvcmQ=
Federated authentication type

For the federated type, the provider field must also be specified. The supported values depend on the identity providers configured for the address space. The federatedUsername and federatedUserid fields must be set, and must map to the username and userid in the federated identity provider.

4.11.2. Authorization

In addition, authorization policies can be defined using operations and addresses. Valid operations are send, recv, view, and manage. If addresses are not specified, operations will be applied to all addresses.

Additional resources

4.12. Managing users on OpenShift

EnMasse user management is only supported when using the standard authentication service. On OpenShift, users can be managed using the OpenShift command-line tools.

Prerequisites

4.12.1. Creating users using the command line

In EnMasse users can be created using standard command-line tools.

Prerequisites
Procedure
  1. Save the user definition to a file:

    apiVersion: user.enmasse.io/v1alpha1
    kind: MessagingUser
    metadata:
      name: myspace.user1
    spec:
      username: user1
      authentication:
        type: password
        password: cGFzc3dvcmQ= # Base64 encoded
      authorization:
        - addresses: ["myqueue", "queue1", "queue2", "topic*"]
          operations: ["send", "recv"]
        - addresses: ["anycast1"]
          operations: ["send"]
  2. Create the user and associated user permissions:

    oc create -f user-example1.yaml
Additional resources

4.12.2. Deleting users using the command line

Users can be deleted using standard command-line tools.

Prerequisites
  • An address space must have been created.

  • A user must have been created.

Procedure
  1. List the current users:

    oc get messagingusers
  2. Delete the desired user:

    oc delete messaginguser myspace.user1
Additional resources

4.12.3. Managing user permissions using the command line

You can edit the permissions for an existing user using the command line.

Prerequisites
Procedure
  1. Retrieve the user whose permissions you want to edit:

    oc get messaginguser myspace.user1 -o yaml > user1-example.yaml
  2. Make the desired permissions change and save the file.

  3. From the command line, run the following command to apply the change:

    oc apply -f user-example1.yaml

    The new user permissions are applied.

Additional resources

4.13. Managing users on Kubernetes

EnMasse user management is only supported when using the standard authentication service. On Kubernetes, users can be managed using a Kubernetes native REST API and custom resources using the command-line tools.

Prerequisites

4.13.1. Creating users using the command line

In EnMasse users can be created using standard command-line tools.

Prerequisites
Procedure
  1. Save the user definition to a file:

    apiVersion: user.enmasse.io/v1alpha1
    kind: MessagingUser
    metadata:
      name: myspace.user1
    spec:
      username: user1
      authentication:
        type: password
        password: cGFzc3dvcmQ= # Base64 encoded
      authorization:
        - addresses: ["myqueue", "queue1", "queue2", "topic*"]
          operations: ["send", "recv"]
        - addresses: ["anycast1"]
          operations: ["send"]
  2. Create the user and associated user permissions:

    kubectl create -f user-example1.yaml
Additional resources

4.13.2. Deleting users using the command line

Users can be deleted using standard command-line tools.

Prerequisites
  • An address space must have been created.

  • A user must have been created.

Procedure
  1. List the current users:

    kubectl get messagingusers
  2. Delete the desired user:

    kubectl delete messaginguser myspace.user1
Additional resources

4.13.3. Managing user permissions using the command line

You can edit the permissions for an existing user using the command line.

Prerequisites
Procedure
  1. Retrieve the user whose permissions you want to edit:

    kubectl get messaginguser myspace.user1 -o yaml > user1-example.yaml
  2. Make the desired permissions change and save the file.

  3. From the command line, run the following command to apply the change:

    kubectl apply -f user-example1.yaml

    The new user permissions are applied.

Additional resources

4.14. Connecting applications to EnMasse

You can connect your application to EnMasse using one of the following client examples.

To connect to the messaging service from outside the OpenShift or Kubernetes cluster, TLS must be used with SNI set to specify the fully qualified host name for the address space. The port used is 443.

The messaging protocols supported depends on the type of address space used. For more information, see the address space section.

4.14.1. Client examples

4.14.2. Apache Qpid Proton Python example

You can use the following Apache Qpid Proton Python example to connect your application to EnMasse. This example assumes you have created an address of type queue named myqueue.

from future import print_function, unicode_literals
from proton import Message
from proton.handlers import MessagingHandler
from proton.reactor import Container

class HelloWorld(MessagingHandler):
    def init(self, server, address):
        super(HelloWorld, self).init()
        self.server = server
        self.address = address

    def on_start(self, event):
        conn = event.container.connect(self.server)
        event.container.create_receiver(conn, self.address)
        event.container.create_sender(conn, self.address)

    def on_sendable(self, event):
        event.sender.send(Message(body="Hello World!"))
        event.sender.close()

    def on_message(self, event):
        print(event.message.body)
        event.connection.close()

Container(HelloWorld("amqps://messaging-route-hostname:443", "myqueue")).run()
Apache Qpid JMS example

You can use the following Apache Qpid JMS example to connect your application to EnMasse. This example assumes you have created an address of type queue named myqueue.

package org.apache.qpid.jms.example;

import javax.jms.Connection;
import javax.jms.ConnectionFactory;
import javax.jms.DeliveryMode;
import javax.jms.Destination;
import javax.jms.ExceptionListener;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageConsumer;
import javax.jms.MessageProducer;
import javax.jms.Session;
import javax.jms.TextMessage;
import javax.naming.Context;
import javax.naming.InitialContext;

public class HelloWorld {
    public static void main(String[] args) throws Exception {
        try {
            // The configuration for the Qpid InitialContextFactory has been supplied in
            // a jndi.properties file in the classpath, which results in it being picked
            // up automatically by the InitialContext constructor.
            Context context = new InitialContext();

            ConnectionFactory factory = (ConnectionFactory) context.lookup("myFactoryLookup");
            Destination queue = (Destination) context.lookup("myQueueLookup");

            Connection connection = factory.createConnection(System.getProperty("USER"), System.getProperty("PASSWORD"));
            connection.setExceptionListener(new MyExceptionListener());
            connection.start();

            Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);

            MessageProducer messageProducer = session.createProducer(queue);
            MessageConsumer messageConsumer = session.createConsumer(queue);

            TextMessage message = session.createTextMessage("Hello world!");
            messageProducer.send(message, DeliveryMode.NON_PERSISTENT, Message.DEFAULT_PRIORITY, Message.DEFAULT_TIME_TO_LIVE);
            TextMessage receivedMessage = (TextMessage) messageConsumer.receive(2000L);

            if (receivedMessage != null) {
                System.out.println(receivedMessage.getText());
            } else {
                System.out.println("No message received within the given timeout!");
            }

            connection.close();
        } catch (Exception exp) {
            System.out.println("Caught exception, exiting.");
            exp.printStackTrace(System.out);
            System.exit(1);
        }
    }

    private static class MyExceptionListener implements ExceptionListener {
        @Override
        public void onException(JMSException exception) {
            System.out.println("Connection ExceptionListener fired, exiting.");
            exception.printStackTrace(System.out);
            System.exit(1);
        }
    }
}

with jndi.properties:

connectionfactory.myFactoryLookup = amqps://messaging-route-hostname:443?transport.trustAll=true&transport.verifyHost=false
queue.myQueueLookup = myqueue
Rhea JavaScript Client example

You can use the following Rhea JavaScript Client example to connect your application to EnMasse. This example assumes you have created an address of type queue named myqueue.

var container = require('rhea');
container.on('connection_open', function (context) {
    context.connection.open_receiver('myqueue');
    context.connection.open_sender('myqueue');
});
container.on('message', function (context) {
    console.log(context.message.body);
    context.connection.close();
});
container.on('sendable', function (context) {
    context.sender.send({body:'Hello World!'});
    context.sender.detach();
});
container.connect({username: 'username', password: 'password', port:443, host:'messaging-route-hostname', transport:'tls', rejectUnauthorized:false});
Rhea JavaScript Client example using WebSockets
var container = require('rhea');
var WebSocket = require('ws');

container.on('connection_open', function (context) {
    context.connection.open_receiver('myqueue');
    context.connection.open_sender('myqueue');
});
container.on('message', function (context) {
    console.log(context.message.body);
    context.connection.close();
});
container.on('sendable', function (context) {
    context.sender.send({body:'Hello World!'});
    context.sender.detach();
});

var ws = container.websocket_connect(WebSocket);
container.connect({username: 'username', password: 'password', connection_details: ws("wss://messaging-route-hostname:443", ["binary"], {rejectUnauthorized: false})});
Apache Qpid Proton C++ example

The C++ client has equivalent simple_recv and simple_send examples with the same options as Python. However, the C++ library does not perform the same level of processing on the URL; in particular it will not accept amqps:// to imply using TLS, so the example needs to be modified as follows:

#include <proton/connection.hpp>
#include <proton/container.hpp>
#include <proton/default_container.hpp>
#include <proton/delivery.hpp>
#include <proton/message.hpp>
#include <proton/messaging_handler.hpp>
#include <proton/ssl.hpp>
#include <proton/thread_safe.hpp>
#include <proton/tracker.hpp>
#include <proton/url.hpp>

#include <iostream>

#include "fake_cpp11.hpp"

class hello_world : public proton::messaging_handler {
  private:
    proton::url url;

  public:
    hello_world(const std::string& u) : url(u) {}

    void on_container_start(proton::container& c) OVERRIDE {
        proton::connection_options co;
        co.ssl_client_options(proton::ssl_client_options());
        c.client_connection_options(co);
        c.connect(url);
    }

    void on_connection_open(proton::connection& c) OVERRIDE {
        c.open_receiver(url.path());
        c.open_sender(url.path());
    }

    void on_sendable(proton::sender &s) OVERRIDE {
        proton::message m("Hello World!");
        s.send(m);
        s.close();
    }

    void on_message(proton::delivery &d, proton::message &m) OVERRIDE {
        std::cout << m.body() << std::endl;
        d.connection().close();
    }
};

int main(int argc, char **argv) {
    try {
        std::string url = argc > 1 ? argv[1] : "messaging-route-hostname:443/myqueue";

        hello_world hw(url);
        proton::default_container(hw).run();

        return 0;
    } catch (const std::exception& e) {
        std::cerr << e.what() << std::endl;
    }

    return 1;
}
AMQP.Net Lite example

You can use the following AMQP.Net Lite example to connect your application to EnMasse. This example assumes you have created an address of type queue named myqueue.

using System;
using Amqp;

namespace Test
{
    public class Program
    {
        public static void Main(string[] args)
        {
            String url = (args.Length > 0) ? args[0] : "amqps://messaging-route-hostname:443";
            String address = (args.Length > 1) ? args[1] : "myqueue";

            Connection.DisableServerCertValidation = true;
            Connection connection = new Connection(new Address(url));
            Session session = new Session(connection);
            SenderLink sender = new SenderLink(session, "test-sender", address);

            Message messageSent = new Message("Test Message");
            sender.Send(messageSent);

            ReceiverLink receiver = new ReceiverLink(session, "test-receiver", address);
            Message messageReceived = receiver.Receive(TimeSpan.FromSeconds(2));
            Console.WriteLine(messageReceived.Body);
            receiver.Accept(messageReceived);

            sender.Close();
            receiver.Close();
            session.Close();
            connection.Close();
        }
    }
}

Appendix A: REST API Reference

A.1. EnMasse REST API

A.1.1. Overview

This is the EnMasse API specification.

Version information

Version : 1.0.0

URI scheme

Schemes : HTTPS

Tags
  • addresses : Operating on Addresses.

  • addressplans : Operating on AddressPlans.

  • addressspaceplans : Operating on AddressSpacePlans.

  • addressspaces : Operate on AddressSpaces

  • brokeredinfraconfigs : Operating on BrokeredInfraConfigs.

  • messagingusers : Operating on MessagingUsers.

  • standardinfraconfigs : Operating on StandardInfraConfigs.

External Docs

Description : Find out more about EnMasse
URL : http://enmasse.io

A.1.2. Paths

POST /apis/admin.enmasse.io/v1alpha1/namespaces/{namespace}/addressspaceplans
Description

create an AddressSpacePlan

Parameters
Type Name Description Schema

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Body

body
required

Responses
HTTP Code Description Schema

200

OK

201

Created

401

Unauthorized

No Content

Consumes
  • application/json

Produces
  • application/json

Tags
  • addressspaceplan

  • admin

  • enmasse_v1alpha1

GET /apis/admin.enmasse.io/v1alpha1/namespaces/{namespace}/addressspaceplans
Description

list objects of kind AddressSpacePlan

Parameters
Type Name Description Schema

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Query

labelSelector
optional

A selector to restrict the list of returned objects by their labels. Defaults to everything.

string

Responses
HTTP Code Description Schema

200

OK

401

Unauthorized

No Content

Produces
  • application/json

Tags
  • addressspaceplan

  • admin

  • enmasse_v1alpha1

GET /apis/admin.enmasse.io/v1alpha1/namespaces/{namespace}/addressspaceplans/{name}
Description

read the specified AddressSpacePlan

Parameters
Type Name Description Schema

Path

name
required

Name of AddressSpacePlan to read.

string

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Responses
HTTP Code Description Schema

200

OK

401

Unauthorized

No Content

404

Not found

No Content

Consumes
  • application/json

Produces
  • application/json

Tags
  • addressspaceplan

  • admin

  • enmasse_v1alpha1

PUT /apis/admin.enmasse.io/v1alpha1/namespaces/{namespace}/addressspaceplans/{name}
Description

replace the specified AddressSpacePlan

Parameters
Type Name Description Schema

Path

name
required

Name of AddressSpacePlan to replace.

string

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Body

body
required

Responses
HTTP Code Description Schema

200

OK

201

Created

401

Unauthorized

No Content

Produces
  • application/json

Tags
  • addressspaceplan

  • admin

  • enmasse_v1alpha1

DELETE /apis/admin.enmasse.io/v1alpha1/namespaces/{namespace}/addressspaceplans/{name}
Description

delete an AddressSpacePlan

Parameters
Type Name Description Schema

Path

name
required

Name of AddressSpacePlan to delete.

string

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Responses
HTTP Code Description Schema

200

OK

401

Unauthorized

No Content

404

Not found

No Content

Produces
  • application/json

Tags
  • addressspaceplan

  • admin

  • enmasse_v1alpha1

POST /apis/enmasse.io/v1alpha1/namespaces/{namespace}/addresses
Description

create an Address

Parameters
Type Name Description Schema

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Body

body
required

Responses
HTTP Code Description Schema

200

OK

201

Created

401

Unauthorized

No Content

Consumes
  • application/json

Produces
  • application/json

Tags
  • addresses

  • enmasse_v1alpha1

GET /apis/enmasse.io/v1alpha1/namespaces/{namespace}/addresses
Description

list objects of kind Address

Parameters
Type Name Description Schema

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Query

labelSelector
optional

A selector to restrict the list of returned objects by their labels. Defaults to everything.

string

Responses
HTTP Code Description Schema

200

OK

401

Unauthorized

No Content

Produces
  • application/json

Tags
  • addresses

  • enmasse_v1alpha1

GET /apis/enmasse.io/v1alpha1/namespaces/{namespace}/addresses/{name}
Description

read the specified Address

Parameters
Type Name Description Schema

Path

name
required

Name of Address to read

string

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Responses
HTTP Code Description Schema

200

OK

401

Unauthorized

No Content

404

Not found

No Content

Consumes
  • application/json

Produces
  • application/json

Tags
  • addresses

  • enmasse_v1alpha1

PUT /apis/enmasse.io/v1alpha1/namespaces/{namespace}/addresses/{name}
Description

replace the specified Address

Parameters
Type Name Description Schema

Path

name
required

Name of Address to replace

string

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Body

body
required

Responses
HTTP Code Description Schema

200

OK

201

Created

401

Unauthorized

No Content

Produces
  • application/json

Tags
  • addresses

  • enmasse_v1alpha1

DELETE /apis/enmasse.io/v1alpha1/namespaces/{namespace}/addresses/{name}
Description

delete an Address

Parameters
Type Name Description Schema

Path

name
required

Name of Address to delete

string

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Responses
HTTP Code Description Schema

200

OK

401

Unauthorized

No Content

404

Not found

No Content

Produces
  • application/json

Tags
  • addresses

  • enmasse_v1alpha1

POST /apis/enmasse.io/v1alpha1/namespaces/{namespace}/addressspaces
Description

create an AddressSpace

Parameters
Type Name Description Schema

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Body

body
required

Responses
HTTP Code Description Schema

200

OK

201

Created

401

Unauthorized

No Content

Consumes
  • application/json

Produces
  • application/json

Tags
  • addressspaces

  • enmasse_v1alpha1

GET /apis/enmasse.io/v1alpha1/namespaces/{namespace}/addressspaces
Description

list objects of kind AddressSpace

Parameters
Type Name Description Schema

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Query

labelSelector
optional

A selector to restrict the list of returned objects by their labels. Defaults to everything.

string

Responses
HTTP Code Description Schema

200

OK

401

Unauthorized

No Content

Produces
  • application/json

Tags
  • addressspaces

  • enmasse_v1alpha1

POST /apis/enmasse.io/v1alpha1/namespaces/{namespace}/addressspaces/{addressSpace}/addresses
Description

create Addresses in an AddressSpace

Parameters
Type Name Description Schema

Path

addressSpace
required

Name of AddressSpace

string

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Body

body
required

AddressList object

Responses
HTTP Code Description Schema

200

OK

401

Unauthorized

No Content

404

Not found

No Content

Consumes
  • application/json

Produces
  • application/json

Tags
  • addressspace_addresses

  • enmasse_v1alpha1

GET /apis/enmasse.io/v1alpha1/namespaces/{namespace}/addressspaces/{addressSpace}/addresses
Description

list objects of kind Address in AddressSpace

Parameters
Type Name Description Schema

Path

addressSpace
required

Name of AddressSpace

string

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Responses
HTTP Code Description Schema

200

OK

401

Unauthorized

No Content

404

Not found

No Content

Produces
  • application/json

Tags
  • addressspace_addresses

  • enmasse_v1alpha1

GET /apis/enmasse.io/v1alpha1/namespaces/{namespace}/addressspaces/{addressSpace}/addresses/{address}
Description

read the specified Address in AddressSpace

Parameters
Type Name Description Schema

Path

address
required

Name of Address

string

Path

addressSpace
required

Name of AddressSpace

string

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Responses
HTTP Code Description Schema

200

OK

401

Unauthorized

No Content

404

Not found

No Content

Produces
  • application/json

Tags
  • addressspace_addresses

  • enmasse_v1alpha1

PUT /apis/enmasse.io/v1alpha1/namespaces/{namespace}/addressspaces/{addressSpace}/addresses/{address}
Description

replace Address in an AddressSpace

Parameters
Type Name Description Schema

Path

address
required

Name of address

string

Path

addressSpace
required

Name of AddressSpace

string

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Body

body
required

Address object

Responses
HTTP Code Description Schema

200

OK

201

Created

401

Unauthorized

No Content

404

Not found

No Content

Consumes
  • application/json

Produces
  • application/json

Tags
  • addressspace_addresses

  • enmasse_v1alpha1

DELETE /apis/enmasse.io/v1alpha1/namespaces/{namespace}/addressspaces/{addressSpace}/addresses/{address}
Description

delete an Address in AddressSpace

Parameters
Type Name Description Schema

Path

address
required

Name of Address

string

Path

addressSpace
required

Name of AddressSpace

string

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Responses
HTTP Code Description Schema

200

OK

401

Unauthorized

No Content

404

Not found

No Content

Produces
  • application/json

Tags
  • addressspace_addresses

  • enmasse_v1alpha1

GET /apis/enmasse.io/v1alpha1/namespaces/{namespace}/addressspaces/{name}
Description

read the specified AddressSpace

Parameters
Type Name Description Schema

Path

name
required

Name of AddressSpace to read

string

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Responses
HTTP Code Description Schema

200

OK

401

Unauthorized

No Content

404

Not found

No Content

Consumes
  • application/json

Produces
  • application/json

Tags
  • addressspaces

  • enmasse_v1alpha1

PUT /apis/enmasse.io/v1alpha1/namespaces/{namespace}/addressspaces/{name}
Description

replace the specified AddressSpace

Parameters
Type Name Description Schema

Path

name
required

Name of AddressSpace to replace

string

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Body

body
required

Responses
HTTP Code Description Schema

200

OK

201

Created

401

Unauthorized

No Content

Produces
  • application/json

Tags
  • addressspaces

  • enmasse_v1alpha1

DELETE /apis/enmasse.io/v1alpha1/namespaces/{namespace}/addressspaces/{name}
Description

delete an AddressSpace

Parameters
Type Name Description Schema

Path

name
required

Name of AddressSpace to delete

string

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Responses
HTTP Code Description Schema

200

OK

401

Unauthorized

No Content

404

Not found

No Content

Produces
  • application/json

Tags
  • addressspaces

  • enmasse_v1alpha1

POST /apis/user.enmasse.io/v1alpha1/namespaces/{namespace}/messagingusers
Description

create a MessagingUser

Parameters
Type Name Description Schema

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Body

body
required

Responses
HTTP Code Description Schema

200

OK

201

Created

401

Unauthorized

No Content

Consumes
  • application/json

Produces
  • application/json

Tags
  • auth

  • enmasse_v1alpha1

  • user

GET /apis/user.enmasse.io/v1alpha1/namespaces/{namespace}/messagingusers
Description

list objects of kind MessagingUser

Parameters
Type Name Description Schema

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Query

labelSelector
optional

A selector to restrict the list of returned objects by their labels. Defaults to everything.

string

Responses
HTTP Code Description Schema

200

OK

401

Unauthorized

No Content

Produces
  • application/json

Tags
  • auth

  • enmasse_v1alpha1

  • user

GET /apis/user.enmasse.io/v1alpha1/namespaces/{namespace}/messagingusers/{name}
Description

read the specified MessagingUser

Parameters
Type Name Description Schema

Path

name
required

Name of MessagingUser to read. Must include addressSpace and dot separator in the name (that is, 'myspace.user1').

string

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Responses
HTTP Code Description Schema

200

OK

401

Unauthorized

No Content

404

Not found

No Content

Consumes
  • application/json

Produces
  • application/json

Tags
  • auth

  • enmasse_v1alpha1

  • user

PUT /apis/user.enmasse.io/v1alpha1/namespaces/{namespace}/messagingusers/{name}
Description

replace the specified MessagingUser

Parameters
Type Name Description Schema

Path

name
required

Name of MessagingUser to replace. Must include addressSpace and dot separator in the name (that is, 'myspace.user1').

string

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Body

body
required

Responses
HTTP Code Description Schema

200

OK

201

Created

401

Unauthorized

No Content

Produces
  • application/json

Tags
  • auth

  • enmasse_v1alpha1

  • user

DELETE /apis/user.enmasse.io/v1alpha1/namespaces/{namespace}/messagingusers/{name}
Description

delete a MessagingUser

Parameters
Type Name Description Schema

Path

name
required

Name of MessagingUser to delete. Must include addressSpace and dot separator in the name (that is, 'myspace.user1').

string

Path

namespace
required

object name and auth scope, such as for teams and projects

string

Responses
HTTP Code Description Schema

200

OK

401

Unauthorized

No Content

404

Not found

No Content

Produces
  • application/json

Tags
  • auth

  • enmasse_v1alpha1

  • user

A.1.3. Definitions

ObjectMeta

ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.

Name Schema

name
required

string

namespace
optional

string

Status

Status is a return value for calls that do not return other objects.

Name Description Schema

code
optional

Suggested HTTP return code for this status, 0 if not set.

integer (int32)

io.enmasse.admin.v1alpha1.AddressPlan
Name Schema

addressType
required

string

apiVersion
required

enum (admin.enmasse.io/v1alpha1)

displayName
required

string

displayOrder
required

integer

kind
required

enum (AddressPlan)

longDescription
optional

string

metadata
required

requiredResources
required

< requiredResources > array

shortDescription
required

string

uuid
required

string

requiredResources

Name Schema

credit
required

number

name
required

string

io.enmasse.admin.v1alpha1.AddressPlanList
Name Schema

apiVersion
required

enum (admin.enmasse.io/v1alpha1)

items
required

kind
required

enum (AddressPlanList)

io.enmasse.admin.v1alpha1.AddressSpacePlan
Name Schema

addressPlans
optional

< string > array

addressSpaceType
required

string

apiVersion
required

enum (admin.enmasse.io/v1alpha1)

displayName
required

string

displayOrder
required

integer

kind
required

enum (AddressSpacePlan)

longDescription
optional

string

metadata
required

resources
required

< resources > array

shortDescription
required

string

uuid
required

string

resources

Name Schema

max
required

number

min
required

number

name
required

string

io.enmasse.admin.v1alpha1.AddressSpacePlanList
Name Schema

apiVersion
required

enum (admin.enmasse.io/v1alpha1)

items
required

kind
required

enum (AddressSpacePlanList)

io.enmasse.admin.v1alpha1.BrokeredInfraConfig
Name Schema

apiVersion
required

enum (admin.enmasse.io/v1alpha1)

kind
required

enum (BrokeredInfraConfig)

metadata
required

spec
required

io.enmasse.admin.v1alpha1.BrokeredInfraConfigList
Name Schema

apiVersion
required

enum (admin.enmasse.io/v1alpha1)

items
required

kind
required

enum (BrokeredInfraConfigList)

io.enmasse.admin.v1alpha1.BrokeredInfraConfigSpec
Name Schema

admin
optional

broker
optional

version
required

string

io.enmasse.admin.v1alpha1.BrokeredInfraConfigSpecAdmin
Name Schema

resources
optional

resources

Name Schema

memory
optional

string

io.enmasse.admin.v1alpha1.BrokeredInfraConfigSpecBroker
Name Schema

addressFullPolicy
optional

enum (PAGE, BLOCK, FAIL)

resources
optional

resources

Name Schema

memory
optional

string

storage
optional

string

io.enmasse.admin.v1alpha1.StandardInfraConfig
Name Schema

apiVersion
required

enum (admin.enmasse.io/v1alpha1)

kind
required

enum (StandardInfraConfig)

metadata
required

spec
required

io.enmasse.admin.v1alpha1.StandardInfraConfigList
Name Schema

apiVersion
required

enum (admin.enmasse.io/v1alpha1)

items
required

kind
required

enum (StandardInfraConfigList)

io.enmasse.admin.v1alpha1.StandardInfraConfigSpec
Name Schema

admin
optional

broker
optional

router
optional

version
required

string

io.enmasse.admin.v1alpha1.StandardInfraConfigSpecAdmin
Name Schema

resources
optional

resources

Name Schema

memory
optional

string

io.enmasse.admin.v1alpha1.StandardInfraConfigSpecBroker
Name Schema

addressFullPolicy
optional

enum (PAGE, BLOCK, FAIL)

resources
optional

resources

Name Schema

memory
optional

string

storage
optional

string

io.enmasse.admin.v1alpha1.StandardInfraConfigSpecRouter
Name Schema

linkCapacity
optional

string

resources
optional

resources

Name Schema

memory
optional

string

io.enmasse.user.v1alpha1.MessagingUser
Name Schema

apiVersion
required

enum (user.enmasse.io/v1alpha1)

kind
required

enum (MessagingUser)

metadata
required

spec
required

io.enmasse.user.v1alpha1.MessagingUserList
Name Schema

apiVersion
required

enum (user.enmasse.io/v1alpha1)

items
required

kind
required

enum (MessagingUserList)

io.enmasse.user.v1alpha1.UserSpec
Name Schema

authentication
optional

authorization
optional

< authorization > array

username
required

string

authentication

Name Description Schema

federatedUserid
optional

User id of the user to federate when 'federated' type is specified.

string

federatedUsername
optional

User name of the user to federate when 'federated' type is specified.

string

password
optional

Base64 encoded value of password when 'password' type is specified.

string

provider
optional

Name of provider to use for federated identity when 'federated' type is specified.

string

type
required

enum (password, federated)

authorization

Name Schema

addresses
optional

< string > array

operations
required

< enum (send, receive, view, manage) > array

io.enmasse.v1alpha1.Address
Name Schema

apiVersion
required

enum (enmasse.io/v1alpha1)

kind
required

enum (Address)

metadata
required

spec
required

status
optional

io.enmasse.v1alpha1.AddressList
Name Description Schema

apiVersion
required

Default : "enmasse.io/v1alpha1"

enum (enmasse.io/v1alpha1)

items
required

kind
required

enum (AddressList)

io.enmasse.v1alpha1.AddressSpace
Name Schema

apiVersion
required

enum (enmasse.io/v1alpha1)

kind
required

enum (AddressSpace)

metadata
required

spec
required

status
optional

io.enmasse.v1alpha1.AddressSpaceList
Name Description Schema

apiVersion
required

Default : "enmasse.io/v1alpha1"

enum (enmasse.io/v1alpha1)

items
required

kind
required

enum (AddressSpaceList)

io.enmasse.v1alpha1.AddressSpaceSpec
Name Schema

authenticationService
optional

endpoints
optional

< endpoints > array

plan
required

string

type
required

authenticationService

Name Schema

details
optional

object

type
optional

string

endpoints

Name Schema

cert
optional

host
optional

string

name
optional

string

service
optional

string

servicePort
optional

string

cert

Name Schema

provider
optional

string

secretName
optional

string

io.enmasse.v1alpha1.AddressSpaceStatus
Name Schema

endpointStatuses
optional

< endpointStatuses > array

isReady
optional

boolean

messages
optional

< string > array

endpointStatuses

Name Schema

host
optional

string

name
optional

string

port
optional

integer

serviceHost
optional

string

servicePorts
optional

< servicePorts > array

servicePorts

Name Schema

name
optional

string

port
optional

integer

io.enmasse.v1alpha1.AddressSpaceType

AddressSpaceType is the type of address space (standard, brokered). Each type supports different types of addresses and semantics for those types.

Type : enum (standard, brokered)

io.enmasse.v1alpha1.AddressSpec
Name Schema

address
required

string

plan
required

string

type
required

io.enmasse.v1alpha1.AddressStatus
Name Schema

isReady
optional

boolean

messages
optional

< string > array

phase
optional

enum (Pending, Configuring, Active, Failed, Terminating)

io.enmasse.v1alpha1.AddressType

Type of address (queue, topic, …). Each address type support different kinds of messaging semantics.

Type : enum (queue, topic, anycast, multicast)