Java plugin development and deployment workflow

Introduction

Apama is an industry leading complex event processing (CEP) engine. The core component of Apama is known as ‘correlator‘ and is powered by a language called EPL (Event Processing Language) that has a syntax which would be familiar to Java programmers. The EPL is native to the core engine resembles Java like syntax. The correlator can also be programmed by general purpose languages like C++Java or Python using a plug-in mechanism. These are referred to as EPL plug-ins in general, for other types of plug-in refer the below note.

The Apama framework also supports another feature called ‘connectivity plug-ins’. The connectivity plug-ins facilitate integrating & transforming data feed generated by external systems like MQTT, Kafka, or even plain JSON feed. The current blog post describes native API support through Java.

This blog post is an introduction on how to develop an EPL Java plug-in in the Apama environment. The post explains various development features in the Apama designer along with deployment utilities available. You can also take a look at how to write an EPL Python plug-in.

Developing an EPL plug-in using Designer

Open the ‘Apama Designer’ tool installed as part of Apama installation. Select new project from “File -> New -> Apama Project“.

 

Name the project as needed, for e.g. “AnEPLJavaPlugin“, and click ‘Finish‘.

Adding Java Nature

The Java Nature is standard eclipse feature that allows all necessary Java JAR files accessible to the project’s java build path. The Apama Designer also comes with “Apama Java Nature” feature. By adding this feature to your project you enable the use of Apama’s Java API for the Java EPL plug-in to work. To add the Apama Java Nature to an existing project select the following, in the “Project Explorer, right-click on the project, select “Apama -> Add Apama Java Nature”.

Adding Java EPL plug-in

The Java EPL plug-in enables writing Java code then to be called from EPL script. The Apama Designer has features to add EPL Java plug-in to an existing Apama project. This can be added in the following way, right click on the project, select “New -> Java EPL Plugin“.

and provide ‘Plugin name‘, ‘Description‘ and ‘Apama Package‘ details, click ‘Finish‘.

The above action opens the Java file with two annotations, @Application and @EPLPlugin. Don’t remove these, you can update details like author, version, etc…

Integrating Java API with EPL Monitor Script

Adding EPL Monitor

Inorder to invoke the EPI Java API, we need to have EPL monitor script added to the project. On the Apama Designer, select the following to add a simple monitor script.

In the Apama project right click on ‘monitors’ and select ‘New -> EPL Monitor‘.

Provide appropriate name to the monitor script and click ‘Finish‘.

Integrating the Java API with EPL script

In the Java source add a static method EPL_API() to be invoked from the EPL script, an example is show below,

The EPL_API() will be called from JavaIntegration.mon EPL monitor as explained below.

At the global level (outside onload or any EPL action) add the following import,

import "demo.DemoEPLJavaPlugin" as eplJavaPlugin;

The import aliases the demo Java plug-in as eplJavaPlugin to be accessible from EPL code under any action; for example, add the following code inside the onload() action,

eplJavaPlugin.EPL_API();

This is all we need integrating the Java method to be called from an EPL script; the complete code is shown below.

We can also add complex Java code (including third party dependencies) with Apama EPL applications in the same way.

Building the project

Inside Apama Designer we don’t need to explicitly build the project as it is automatically built, however you can run the project by selecting “Run As -> Apama Application“. From the log messages you can see the message printed from Java method “EPLPlugin message from Java“.

Ant build & run

The user can also use the ant build system to build the project on the command line. This can be done by exporting the ant build file from Apama Designer by right click on the Apama project, select “export“, and search for ant, select “Apama Ant Export” as shown.

Click “Next” and provide directory path to which the build & run scripts to be written. For example, these are written to “AnEPLJavaPluginAnt“. The generated ant script takes care of building the JAR from Java sources. We can also write the ant build file manually, however it can be error prone.

You can run ‘ant‘ command available in the Apama environment to build the JAR from sources following by running the binaries.

Deploying the project

Apama SDK is shipped with few utilities useful for project deployment. One such tool is engine_deploy, you can run the help to see how to use the tool. Use the following command to deploy the current project to another directory called ‘AnEPLJavaPluginDeployed‘.

engine_deploy -d AnEPLJavaPluginDeployed C:\SoftwareAG_10.5\Designer_Workspace\AnEPLJavaPlugin

Ensure you don’t get any error in running the above command, and similar set of command also work for Linux/Windows platform. The deployed project artifacts are available under ‘AnEPLJavaPluginDeployed‘.

Running the project on command line

Inorder to run the EPL Java plug-in, the Apama correlator needs to know that Java code to be integrated, this can be done using -j option of the correlator. The Apama SDK comes with utilities to inject the build JAR files and monitor scripts into the running correlator. The engine_inject tool used in this regard.

Navigate to the AnEPLJavaPluginDeployed directory, and run the following commands,

AnEPLJavaPluginDeployed>correlator -j --config config\CorrelatorConfig.yaml – This starts the correlator emitting its log to the console.

The below two commands inject the JAR file followed by the monitor script to the running correlator.

AnEPLJavaPluginDeployed>engine_inject -j -v "C:\ SoftwareAG_10.5\Designer_Workspace\AnEPLJavaPluginDeployed\AnEPLJavaPlugin java application files\AnEPLJavaPlugin.jar"

AnEPLJavaPluginDeployed>engine_inject -v "C: \SoftwareAG_10.5\Designer_Workspace\AnEPLJavaPluginDeployed\monitors\JavaIntegration.mon"

Note: It is assumed that the correlator is running on default port, it can be overridden by using -p option including the engine_inject utility.

After inserting the EPL monitor script, the onload() action will be called as part of monitor initialization which in turn calls the EPL Java API. The correlator log can be inspected for messages emitted from Java API.

For more details on EPL Java plug-in refer to the section ‘Writing EPL Plug-ins in Java‘ in the Apama documentation.

A more complex example of parsing JSON data

The above example described how to call Java code from EPL without any third party dependency.

For demonstrating the next example, we will need third party Java JARs and we assume that all the required JAR files are downloaded into the project working directory. To be specific we need jackson-core-2.10.1.jar, jackson-databind-2.10.1.jar and jackson-annotations-2.10.1.jar files for this example. You can download them from Maven or other JAR repositories.

Apama EPL doesn’t having in-built functionality of parsing JSON data in string form into an equivalent EPL dictionary data structure. In the current example, we pass JSON data in the form of a string to the Java API, and the input string originates from EPL code. The Java code further decodes the string data into a JSON structure using the Jackson parser. In the process the JSON structure is converted into corresponding Java map, and the then formatted into a string form to be interpreted by the EPL dictionary parser. Refer to the below Java code for details,

And the corresponding EPL code to pass JSON data as string, interpret (parse) returned string as EPL dictionary is given below,

As shown in the code, we are embedding various currency exchange rates w.r.t. USD in the form of JSON data inside an EPL string (You can also consider this as external data coming via connectivity chain for processing inside correlator). The string is passed to Java API to construct a JSON object using the Jackson library. The map is converted back to a string for inter-operation between the EPL & the Java object, and the same string is parsed as a dictionary inside EPL. Without JSON parser in Java, we can’t convert the string like JSON into EPL dictionary data structure.

Configuring class path

As we can see the parsing requires Jackson library imports. These imports can be resolved by updating build paths. The project needs to be configured to update third party libraries like Jackson. As described earlier, download them to local project directory, and add them to project as explained below.

Right click on the project, select “Build path -> Configure Build Path

In the next window go to ‘Libraries‘ tab, and select ‘Add External JARs‘, navigate to project directory (or to the directory JARs stored) and select the JAR files downloaded to project directory.

This will resolve the import errors w.r.t. Jackson library.

 

These libraries can also be configured using classpath option of @Application annotation. An example,

classpath = "${env:PROJECT_HOME}/jackson-annotations-2.10.1.jar;${env:PROJECT_HOME}/jackson-core-2.10.1.jar;${env:PROJECT_HOME}/jackson-databind-2.10.1.jar"

The environment variable PROJECT_HOME should be set to the project directory or to the directory JARs stored.

Caveats

The Java code integrated with correlator can still suffer usual concurrency related issues. The plug-in developer needs to ensure no such bugs are introduced, if not the correlator can get into issues like deadlocks, etc…

What Next

Apama packages are available in two forms, a full installation which requires a valid license, and core installation with limited functionality. In the future, we will cover more application specific EPL Java plug-in examples. You can find more Java plug-in examples in the samples directory apama-samples/correlator_plugin/java.