July 1, 2020

How to Implement Custom Expression Language in Camel 3.x

by Eugene Berman in DataSonnet , OSS , Tips and Tricks 2 comments

In this blog post, I’m going to explain the process of implementing a custom expression language support in detail as well as highlight some specifics of the DataSonnet implementation. But first, let me explain how this got started. 

Prelude to a post

Shortly after we released the first version of DataSonnet, our open source data transformation platform, a colleague of mine working on the next version of the PortX platform sent me this Slack message: 

“Hey, can I use DataSonnet as an expression language in Apache Camel?”

And, I keyed:

“No”

But just as my thumb was reaching for ‘Send’, intuition (and curiosity) arrested it. “Well, hmm. Maybe.” So I backspaced over No, responded to my coworker instead with a boiler-plate, time-buying equivocation, opened DuckDuckGo, and searched for…

“how to implement a custom expression language in Camel”

I was expecting a bunch of results with links to tutorials. To my surprise, nothing useful. It was the same with my Google search, nada. At this point, I realized I would need to do a private investigation and get some help from my colleagues as well.

The Solution

So, after a few days of bouncing ideas off coworkers and playing around with a couple of different approaches, we found a solution. Without further adieu, here’s what we came up with. 

Create an empty Maven project

Add the following snippet to the <build><plugins> section:

<plugin>

   <groupId>org.apache.camel</groupId>

   <artifactId>camel-package-maven-plugin</artifactId>

   <version>${camel.version}</version>

   <executions>

       <execution>

           <id>generate</id>

           <phase>process-classes</phase>

           <goals>

               <goal>generate-languages-list</goal>

           </goals>

       </execution>

   </executions>

   <configuration>

       <failFast>false</failFast>

   </configuration>

</plugin>

This snippet enables the Camel package plugin.

Next, add the camel-core dependency:

<dependency>

   <groupId>org.apache.camel</groupId>

   <artifactId>camel-core</artifactId>

   <version>${camel.version}</version>

</dependency>

The ${camel.version} is the version of the Camel core. I tested this solution with Camel 3.2 but it should work with any 3.x version.

Add other dependencies 

Finally, add any dependencies that your custom expression language may require. In my case, it included dependencies for the DataSonnet mapper as well as some additional helper libraries.

Create resources

Now it’s time to create some resources.
Create the src/main/resources/META-INF/services/org/apache/camel/language folder and create the file named datasonnet there. The content of this file is:

class=com.modus.camel.datasonnet.language.DatasonnetLanguage

Next, create the file language.properties in the src/main/resources/META-INF/services/org/apache/camel folder:

languages=datasonnet

groupId=com.modus.camel.datasonnet

artifactId=camel-datasonnet

version=1.0.2-SNAPSHOT

projectName=Camel :: Datasonnet

projectDescription=Camel Datasonnet support 

And finally, create a directory:

src/main/resources/org/apache/camel/language/datasonnet and then create a file datasonnet.json:

{

 "language": {

   "kind": "language",

   "name": "datasonnet",

   "modelName": "datasonnet",

   "title": "Datasonnet",

   "description": "To use Datasonnet scripts in Camel expressions or predicates.",

   "deprecated": false,

   "deprecationNote": "language,datasonnet",

   "firstVersion": "1.0.0",

   "label": "language,datasonnet",

   "javaType": "com.modus.camel.datasonnet.language.DatasonnetLanguage",

   "modelJavaType": "com.modus.camel.datasonnet.language.model.DatasonnetExpression",

   "groupId": "1.0.2-SNAPSHOT",

   "artifactId": "camel-datasonnet",

   "version": "1.0.2-SNAPSHOT"

 },

 "properties": {

   "expression": { "kind": "value", "displayName": "Expression", "required": true, "type": "string", "javaType": "java.lang.String", "deprecated": false, "secret": false, "description": "The expression value in your chosen language syntax" },

   "id": { "kind": "attribute", "displayName": "Id", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "secret": false, "description": "Sets the id of this node" },

   "inputMimeType": { "kind": "attribute", "displayName": "inputMimeType", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "secret": false, "description": "Sets the input mime type of the expression" },

   "outputMimeType": { "kind": "attribute", "displayName": "outputMimeType", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "secret": false, "description": "Sets the output mime type of the expression" }

 }

}
Create a Java class

First, create a class for our language:

package com.modus.camel.datasonnet.language;

import com.modus.camel.datasonnet.DatasonnetProcessor;

import org.apache.camel.spi.annotations.Language;

import org.apache.camel.support.LanguageSupport;

import org.slf4j.Logger;

import org.slf4j.LoggerFactory;


@Language("datasonnet")

public class DatasonnetLanguage extends LanguageSupport {

   private static Logger logger = LoggerFactory.getLogger(DatasonnetProcessor.class);


   @Override

   public DatasonnetExpression createPredicate(String expression) {

       return createExpression(expression);

   }


   @Override

   public DatasonnetExpression createExpression(String expression) {

    DatasonnetExpression datasonnetExpression = new DatasonnetExpression(expression);

       return datasonnetExpression;

   }

}
Create an interface

The interface is used for bean injection:

package com.modus.camel.datasonnet.language;



import org.apache.camel.support.language.LanguageAnnotation;

import java.lang.annotation.*;



@Retention(RetentionPolicy.RUNTIME)

@Documented

@Target({ElementType.FIELD, ElementType.METHOD, ElementType.PARAMETER })

@LanguageAnnotation(language = "datasonnet")

public @interface Datasonnet {

   String value();

}
Create classes

This class represents the DataSonnet expression at the runtime:

package com.modus.camel.datasonnet.language;


import org.apache.camel.spi.GeneratedPropertyConfigurer;

import org.apache.camel.support.ExpressionAdapter;

import org.apache.camel.support.component.PropertyConfigurerSupport;


public class DatasonnetExpression extends ExpressionAdapter implements GeneratedPropertyConfigurer {


...


}

Note that this implements the GeneratedPropertyConfigurer interface – this is done so that additional parameters controlling the DataSonnet behavior can be passed via exchange properties. You can see the full implementation of this class at https://github.com/modusbox/camel-datasonnet/blob/master/src/main/java/com/modus/camel/datasonnet/language/DatasonnetExpression.java

In addition to this, a model class extending the ExpressionDefinition should be created:

package com.modus.camel.datasonnet.language.model;


import org.apache.camel.CamelContext;

import org.apache.camel.Expression;

import org.apache.camel.model.language.ExpressionDefinition;

import org.apache.camel.spi.Metadata;


import javax.xml.bind.annotation.XmlAccessType;

import javax.xml.bind.annotation.XmlAccessorType;

import javax.xml.bind.annotation.XmlAttribute;

import javax.xml.bind.annotation.XmlRootElement;


/**

* To use Datasonnet scripts in Camel expressions or predicates.

*/

@Metadata(firstVersion = "1.0.0", label = "language,datasonnet", title = "Datasonnet")

@XmlRootElement(name = "datasonnet")

@XmlAccessorType(XmlAccessType.FIELD)

public class DatasonnetExpression extends ExpressionDefinition {

...

}

I also wanted to provide convenient datasonnet() functions to be used in route definitions, so I created an abstract DatasonnetRouteBuilder class:

package com.modus.camel.datasonnet;


import com.modus.camel.datasonnet.language.model.DatasonnetExpression;

import org.apache.camel.Expression;

import org.apache.camel.builder.RouteBuilder;

import org.apache.camel.builder.ValueBuilder;


public abstract class DatasonnetRouteBuilder extends RouteBuilder {

   public ValueBuilder datasonnet(String value) {

       return datasonnet(value, "application/json", "application/json");

   }

   public ValueBuilder datasonnet(Expression expression) {

       return datasonnet(expression, "application/json", "application/json");

   }

   public ValueBuilder datasonnet(String value, String inputMimeType, String outputMimeType) {

       DatasonnetExpression exp = new DatasonnetExpression(value);

       exp.setInputMimeType(inputMimeType);

       exp.setOutputMimeType(outputMimeType);

       return new ValueBuilder(exp);

   }

   public ValueBuilder datasonnet(Expression expression, String inputMimeType, String outputMimeType) {

       DatasonnetExpression exp = new DatasonnetExpression(expression);

       exp.setInputMimeType(inputMimeType);

       exp.setOutputMimeType(outputMimeType);

       return new ValueBuilder(exp);

   }

}

Please visit the camel-datasonnet GitHub repository for complete code and examples of usage. 

Why DataSonnet?

When my coworker originally asked if this was possible, I simultaneously thought to myself, “Why are they asking?” Camel already has many expression languages supported out of the box, including its own Simple language, Groovy, MVEL, and many others. So, why DataSonnet?

For us, the answer is consistency when handling data of different types. 

Normally, one would likely use JsonPath to access the JSON formatted data, XPath for XML, and custom Java beans for CSV. And, if you use Javascript, Groovy, or another scripting language, the code gets cluttered with handling the details of the data format. With DataSonnet, you can use the same language when handling all of these formats and create your own plugins if you need to support other formats.

Usage Example: Dynamic queries

Shortly after finishing the first iteration of this project, I was working on an API that returns a number of results from a database based on the request query parameters. All parameters are optional; for example, a consumer should be able to search by email address, first name only, first and last name, or last name only. So the SQL query must be created dynamically. With Datasonnet expression support, this task was simplified. Here’s how I implemented it:

<setBody>

  <language language=”datasonnet”>

    local headerNames = ["firstName", "lastName", "email"];

    local filteredHeaders = std.foldl(

      function(aggregate, x)

        if (std.objectHas(headers, x) > 0) then

          aggregate + [ x + "' = '" + headers[x] + "'"]

        else aggregate + [],

      headerNames,

      []);


    "SELECT * from members WHERE " + std.join(" AND ", filteredHeaders)

  </language>

</setBody>

<to uri="jdbc:MembersDB"/>

 

Please check out the DataSonnet project, provide feedback, join us in evolving the tool, and participate with our team in the community.

Learn more about DataSonnet:

DataSonnet website

Quick Start Tutorial

Cookbook

Let us know what you think about it in the comments below!


Comments (2)
  • Hi

    I wonder if you would consider contributing your Camel Datasonnet Language to Apache Camel so it can be included out of the box?

    And btw there are some ways to have Camel Maven plugin auto generate those meta files so you dont need to create them by hand.

    Claus Ibsen
    Apache Camel committer

    • Hi Claus,
      Yes, we’re excited to contribute the DataSonnet component, both as an expression language and Data Transformation tool to Apache Camel! We’ll reach out soon to learn about the process.

Leave a comment