How To: Elasticsearch

Use Elasticsearch instead of an RDBMS

The API Manager is configured (by default) to use JPA as the persistence technology for storing all of its data. But this isn’t the only persistence technology supported. Another option is to use Elasticsearch. This section details how to set up Apiman to use Elasticsearch instead of an RDBMS to store your API Manager data.

Overview

  1. Download and install Elasticsearch 7.x

  2. Make changes to apiman.properties to switch from JPA to Elasticsearch

  3. (Re)start Apiman!

  4. Perform standard admin configuration of Apiman (the database will of course be empty!)

Download and install Elasticsearch

This part is pretty easy - download the Elasticsearch software and get it running. The Elasticsearch documentation provides extensive guidance for this.

An embedded instance of Elasticsearch is no longer included with Apiman.

Running Elasticsearch using Docker is a very convenient way to get started.

docker run -p 19200:9200 -p 19300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:<ES 7 version>

You may use it as your API Manager persistence store.

Make changes to apiman.properties

Once Elasticsearch is running smoothly, you must make some changes to the apiman.properties file in order to tell Apiman to use ES instead of a database. You should modify the apiman.properties file to have the following properties set:

You can optionally also set the username and password - this is only useful if you are using something like Elasticsearch Shield to enable basic authentication. 
apiman.es.protocol=http
apiman.es.host=localhost
apiman.es.port=19200
apiman.es.username=
apiman.es.password=

apiman-manager.storage.type=es
apiman-manager.storage.es.protocol=${apiman.es.protocol}
apiman-manager.storage.es.host=${apiman.es.host}
apiman-manager.storage.es.port=${apiman.es.port}
apiman-manager.storage.es.username=${apiman.es.username}
apiman-manager.storage.es.password=${apiman.es.password}
apiman-manager.storage.es.initialize=true

Make sure you enter appropriate values for the apiman.es.protocol, apiman.es.host, and apiman.es.port properties. These values should reflect the settings of your Elasticsearch instance.

If you are using a standalone gateway instance you will need to configure it separately to point to the same datastore. For example, conf.json for Vert.x and standalone.xml for Servlet.

(Re)start Apiman

If Apiman was running, you should stop it now. Once everything is shutdown, and the changes to apiman.properties have been made, go ahead and start Apiman up again. It will pick up the new settings in apiman.properties and attempt to use Elasticsearch instead of the database!

Perform standard admin configuration

Note that the Apiman quickstart overlay ZIP comes pre-configured with a number of settings, including:

  • Installed policy definitions

  • Default configured roles (Organization Owner, API Developer, Application Developer)

  • A default configured Gateway

This built-in configuration will be lost when you switch from JPA to Elasticsearch. You will need to use the Apiman admin UI to reconfigure these settings. Refer to the "System Administration" section of the User Guide for more information on this.

Use standalone Elasticsearch instance/cluster

The Apiman quickstart overlay ZIP ships by default with an embedded instance of Elasticsearch. This is suitable for getting up and running quickly, but is not a good long term solution. Instead, users are encouraged to install a standalone instance of Elasticsearch and point Apiman to it.

High Level Overview

  1. Download and install Elasticsearch

  2. Make changes to "apiman.properties" to point to your standalone Elasticsearch instance

  3. (Re)start Apiman!

Download and install Elasticsearch

This part is pretty easy - download the Elasticsearch software and get it running. A very good resource for this can be found here:

www.elastic.co/guide/en/elasticsearch/guide/master/getting-started.html

Make changes to "apiman.properties"

Once Elasticsearch is running smoothly, you must make some minor changes to the apiman.properties file in order to tell Apiman where your Elasticsearch instance is located.

There are a set of global properties used for all Apiman components that use Elasticsearch to load data. These properties are:

apiman.es.protocol=http
apiman.es.host=localhost
apiman.es.port=19200
apiman.es.username=
apiman.es.password=

Make sure you enter appropriate values for this properties - they should reflect the settings of your Elasticsearch installation.

(Re)start Apiman

If Apiman was running, you should stop it now. Once everything is shutdown, and the changes to apiman.properties have been made, go ahead and start Apiman up again. It will pick up the new settings in apiman.properties and attempt to use Elasticsearch instead of Infinispan.