Apama uses the concept of “bundles” to provide the ability to add connectivity such as Kafka or MQTT to the project, and also for adding EPL capabilities such as date/time formatting. Until now, the only way to create a project and add these bundles was using the Software AG Designer graphical environment, which is only supported on Windows.
However, now we have a new tool called apama_project that enables all this functionality from the command line, on Linux as well as Windows.
The tool is interoperable with Software AG Designer, allowing the user to move between the tools as required. Projects created with apama_project are also fully compatible with the engine_deploy tool for project deployment.
The diagram below shows a comparison between the traditional Software AG Designer application and the newly developed command-line tool apama_project .
It illustrates how we can use the tool to manage an Apama project and the bundles inside it.
Actions supported by the apama_project tool
Creating a project
The tool creates a new project directory with the specified name below the current directory and adds all the Apama project related files into the project directory.
The created project will be identical to one created within Software AG Designer.
1 2 3 4 |
> apama_project create ApamaProject Project created successfully at location: .../ApamaProject |
Creates a folder ApamaProject with the following layout:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
ApamaProject |-.dependencies |-queries |-config | |-CorrelatorConfig.yaml | |-dashboard_generation.xml | |-dataplayer_queries.xml | |-dashboard_deploy.xml |-logs |-events |-monitors |-eventdefinitions |-dashboards | |-EXTENSIONS.ini | |-JAAS.ini |-.project |
Once the project is successfully created, change the shell’s current directory to the new project directory before performing other commands such as listing bundles or adding/removing bundles.
Adding bundles
Adding a bundle is very simple, and can be done in a couple of ways. Firstly, we can pass the bundle display name, which gets displayed when we list the bundles (See below) in the project.
1 2 3 4 5 6 7 8 |
> apama_project add bundle Kafka Adding bundle "Kafka". "Kafka" bundle added successfully. You may need to configure instance files by editing the following files. ".../ApamaProject/config/connectivity/Kafka/Kafka.yaml" ".../ApamaProject/config/connectivity/Kafka/Kafka.properties" |
Some bundles support adding multiple instances. For example, we can add several instances of the HTTP client to connect to different HTTP servers.
We can supply an instance name for such bundles using --instance to give a name for the service we’re connecting to. If omitted a default instance name will be used E.G. HTTPClient, HTTPClient2, HTTPClient3 , and so on.
The instance name is used in various places within the connectivity configuration files and in some cases also in the EPL code for interacting with them. It cannot be changed once the bundle instance has been added, so it’s worth thinking about before adding.
Kafka is another bundle which can be used multiple times, above we added it with the default and below we use our own more descriptive name:
1 2 3 4 5 6 7 8 |
> apama_project add bundle Kafka --instance TwitterStream Adding bundle "Kafka". "Kafka" bundle added successfully. You may need to configure instance files by editing the following files. ".../ApamaProject/config/connectivity/TwitterStream/TwitterStream.yaml" ".../ApamaProject/config/connectivity/TwitterStream/TwitterStream.properties" |
We can also add a bundle using the index number that is shown in the bundle list (See Below). Since the index numbers are not fixed for any bundle type, and may change with the addition of new bundles, we should list the bundles before using this command to ensure we are using the correct index number.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
> apama_project list bundles Bundles that have already been added: Bundles that can be added: Standard bundles: .. ... Adapter bundles: .. ... Connectivity bundles: .. ... 43 User Connectivity > apama_project add bundle 43 Adding bundle "User Connectivity". "User Connectivity" bundle added successfully. You may need to configure instance files by editing the following files. ".../ApamaProject/config/connectivity/UserConnectivity/UserConnectivity.properties" ".../ApamaProject/config/connectivity/UserConnectivity/UserConnectivity.chains.yaml" ".../ApamaProject/config/connectivity/UserConnectivity/UserConnectivity.plugins.yaml" |
Listing bundles
The bundles list consists of two sections. The first section displays the bundles that are already added to the Apama project. The second section displays the available bundles that can be added to the project.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
> apama_project list bundles Bundles that have already been added: Kafka Kafka TwitterStream Bundles that can be added: Standard bundles: 1 Any Extractor .. ... 24 Time Format Adapter bundles: 25 Distributed MemoryStore .. ... 31 Web Services Client Adapter Connectivity bundles: 32 Kafka .. ... 43 User Connectivity |
Removing bundles
To delete a standard EPL bundle, we need to provide the bundle display name as shown in the list.
To delete a connectivity bundle, we need to provide the bundle instance name or the bundle display name as shown in the list. The default name of the first instance of a connectivity bundle is the display name of the bundle as shown in the list.
Consider the earlier scenario where we’ve added two Kafka instances – if we execute apama_project remove bundle Kafka only the default Kafka instance will be removed, while the other instance TwitterStream will remain unaffected.
1 2 3 4 5 |
> apama_project remove bundle Kafka Removing bundle instance: Kafka Successfully removed the bundle instance : Kafka |
To clean up another Kafka instance, we can pass the bundle instance name.
1 2 3 4 5 |
> apama_project remove bundle TwitterStream Removing bundle instance: KafkaInstanceFoo Successfully removed the bundle instance : TwitterStream |
Alternatively, we can simply input the bundle file name to remove the remaining non-matching instances of the Kafka bundle.
1 2 3 4 5 |
> apama_project remove bundle Kafka Removing bundle : Kafka Successfully removed the bundle : Kafka |
Deploying a tool-created project
The engine_deploy tool can deploy a project that’s created using apama_project or SoftwareAG Designer. It works on both windows and linux and we use it below in combination with apama_project to create an application using a connectivity plug-in to allow Apama to act as a simple HTTP server:
1 2 3 4 |
> apama_project create HTTPServerDemo Project created successfully at location: .../HTTPServerDemo |
Run
apama_project add bundle
command from within the project directory:
1 2 3 4 5 6 7 8 9 10 |
> cd HTTPServerDemo > apama_project add bundle "HTTP Server" Adding bundle "HTTP Server". "HTTP Server" bundle added successfully. You may need to configure instance files by editing the following files. "...\HTTPServerDemo\config\connectivity\HTTPServer\HTTPServer.properties" "...\HTTPServerDemo\config\connectivity\HTTPServer\HTTPServer.yaml" "...\HTTPServerDemo\config\connectivity\HTTPServer\swagger_HTTPServer.json" |
make changes to the contents inside the HTTPServer.yaml file:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
connectivityPlugins: httpServer: libraryName: connectivity-http-server class: HTTPServer dynamicChainManagers: httpServer: transport: httpServer managerConfig: port: 443 bindAddress: localhost dynamicChains: httpServer: - apama.eventMap: defaultEventType: HelloWorld defaultChannel: HelloWorldChannel - jsonCodec - stringCodec - httpServer: automaticResponses: true allowedMethods: [PUT] |
Add a new monitor file in the monitors directory, E.G. server.mon :
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
event HelloWorld {} monitor HelloWorldMonitor { action onload() { monitor.subscribe("HelloWorldChannel"); on all HelloWorld () { log "Hello World" at INFO; } // signal that we are ready to receive events com.softwareag.connectivity.ConnectivityPlugins.onApplicationInitialized(); } } |
Now, use the engine_deploy tool to create a deployment directory from the project:
1 2 3 |
> engine_deploy.exe --outputDeployDir HTTPServerDemo-Deployment HTTPServerDemo |
And run the application using the following command:
1 2 3 4 5 |
> correlator --config HTTPServerDemo-Deployment ... 2019-02-22 00:49:00.261 INFO [3364] - <connectivity.httpServer.manager> Binding to http://localhost:443 |
You should now be able to attach with a browser to see “Hello World”.