How to have custom mapper keys in your documents with Jongo

Posted: décembre 31st, 2013 | Author: | Filed under: Dev, Java, Jongo, MongoDB, NoSQL, Tutorial | No Comments »

I’m giving a try to Jongo to replace spring-data-mongodb in a project but I have to manage documents with ‘not really java compatible keys’.
For example, a key with a first upper case letter and some ‘-’ inside … (well ok that’s ugly but I can’t change that now).

In spring-data, I’m using MappingMongoConverter, it works but Jongo seems to be easier to use and deploy than spring data.

And yes, it’s very simple to customize the way Jongo manages your beans !
The reason is that Jongo uses Jackson Mapper (by default but you could use your own or favorite one).
So, you just have to create a PropertyNamingStrategy !!!

1 – Create a NamingStrategy

With this naming strategy, we just search in a map if there’s a replacement value for the provided key.

package com.ezakus.data.jongo.customs;

import static com.fasterxml.jackson.databind.PropertyNamingStrategy.PropertyNamingStrategyBase;

import java.util.Map;

public class MapTransformNamingStrategy extends PropertyNamingStrategyBase{

    private static final long serialVersionUID = 1L;

    private Map<String, String> mapping;

    public MapTransformNamingStrategy(Map<String, String> mapping) {
        this.mapping = mapping;
    }

    @Override
    public String translate(String property) {
        if (mapping.containsKey(property)) {
            return mapping.get(property);
        }

        return property;
    }
}

2 – Use it with Jongo

As you could see in the Jongo documentation :

Jongo comes with a custom Jackson configuration that can be extended.

So you just have to use addModifier and a MapperModifier implementation to set your naming strategy to the mapper.

package com.ezakus.data.jongo.customs;

import com.fasterxml.jackson.databind.ObjectMapper;
import org.jongo.marshall.jackson.configuration.MapperModifier;

import java.util.HashMap;
import java.util.Map;

public class CustomMapperModifier implements MapperModifier{
    Map<String, String> mapping = new HashMap<String, String>();

    public CustomMapperModifier(){
        mapping.put("myProperty", "custom-key-for-my-property");
    }

    @Override
    public void modify(ObjectMapper objectMapper) {
        objectMapper.setPropertyNamingStrategy(new MapTransformNamingStrategy(mapping));
    }
}
Jongo jongo = new Jongo(db, new JacksonMapper.Builder().addModifier(new CustomMapperModifier()).build());

3 – Full example

Here is a DummyObject class to test :

package com.ezakus.data.jongo.document;

import org.bson.types.ObjectId;

public class DummyObject {

    private ObjectId id;
    private String myProperty;

    public ObjectId getId() {
        return id;
    }

    public void setId(ObjectId id) {
        this.id = id;
    }

    public String getMyProperty() {
        return myProperty;
    }

    public void setMyProperty(String myProperty) {
        this.myProperty = myProperty;
    }
}

And a unit test (I’m using an embedded mongo for the test)

package com.ezakus.data.jongo.document;

import com.ezakus.data.jongo.customs.CustomMapperModifier;
import com.mongodb.DB;
import com.mongodb.MongoClient;
import de.flapdoodle.embed.mongo.MongodExecutable;
import de.flapdoodle.embed.mongo.MongodProcess;
import de.flapdoodle.embed.mongo.MongodStarter;
import de.flapdoodle.embed.mongo.config.MongodConfig;
import de.flapdoodle.embed.mongo.distribution.Version;
import de.flapdoodle.embed.process.runtime.Network;
import org.jongo.Jongo;
import org.jongo.MongoCollection;
import org.jongo.marshall.jackson.JacksonMapper;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;

import java.net.UnknownHostException;

import static org.fest.assertions.Assertions.assertThat;

public class NamingStrategyTest {

    public static MongodExecutable mongodExe;
    public static MongodProcess    mongod;

    @BeforeClass
    public static void runBeforeAllTests() throws Exception {
        MongodStarter runtime = MongodStarter.getDefaultInstance();
        NamingStrategyTest.mongodExe = runtime.prepare(new MongodConfig(Version.Main.V2_0, 12345, Network.localhostIsIPv6()));
        NamingStrategyTest.mongod = NamingStrategyTest.mongodExe.start();
    }

    @AfterClass
    public static void runAfterAllTests() {
        NamingStrategyTest.mongod.stop();
        NamingStrategyTest.mongodExe.cleanup();
    }

    @Test
    public void testNaming(){
        DB db = null;
        try {
            db = new MongoClient("localhost", 12345).getDB("myDB");

            Jongo jongo = new Jongo(db, new JacksonMapper.Builder().addModifier(new CustomMapperModifier()).build());

            MongoCollection dum = jongo.getCollection("dummies");
            assertThat(dum.count()).isEqualTo(0);

            DummyObject d = new DummyObject();
            String prop = "where is brian ?";
            d.setMyProperty(prop);

            dum.save(d);
            assertThat(dum.count()).isEqualTo(1);

            DummyObject check = dum.findOne("{custom-key-for-my-property: '"+prop+"'}").as(DummyObject.class);
            assertThat(check.getId()).isEqualTo(d.getId());
            assertThat(check.getMyProperty()).isEqualTo(prop);

        } catch (UnknownHostException e) {
            e.printStackTrace();
        }
    }

}

Compiling and installing Hyperdex from sources with Java and YCSB bindings on Debian

Posted: août 2nd, 2012 | Author: | Filed under: BigData, Dev, Java, Linux, NoSQL, Tutorial | No Comments »

After a lot of hours I finally succeed to build Hyperdex on a Debian with Java and YCSB bindings.

1 – Apt sources

Edit /etc/apt/sources.list and add backports packages

deb http://backports.debian.org/debian-backports squeeze-backports main

If you don’t have contrib and non-free for the main line, add them :

deb http://debian.mirrors.ovh.net/debian/ squeeze main contrib non-free

Add the Hyperdex repository

wget -O - http://debian.hyperdex.org/hyperdex.gpg.key \
  | apt-key add -

wget -O /etc/apt/sources.list.d/hyperdex.list \

http://debian.hyperdex.org/hyperdex.list

Update

apt-get update

2 – Install dependencies

apt-get install git openjdk-6-jdk build-essential libcityhash-dev libgoogle-glog-dev \
libpopt-dev libpo6-dev libe-dev libbusybee-dev python python-dev python-pyparsing \
python-setuptools python-argparse cython swig2.0 autoconf automake autoconf-archive libtool pkg-config gperf
easy_install sphinx
wget https://github.com/downloads/brianfrankcooper/YCSB/ycsb-0.1.4.tar.gz
tar zxf ycsb-0.1.4.tar.gz
export CLASSPATH=`pwd`/ycsb-0.1.4/core/lib/core-0.1.4.jar
git clone git://git.hyperdex.org/po6.git
git clone git://git.hyperdex.org/e.git

cd po6
autoreconf -i; ./configure; make && make install
cd ..

cd e
autoreconf -i; ./configure; make && make install
cd ..

3 – Install Hyperdex

git clone git://git.hyperdex.org/HyperDex.git

cd Hyperdex
autoreconf -i; ./configure --enable-java-bindings --enable-ycsb; make && make install

CORS-Compliant REST API with Jersey and ContainerResponseFilter

Posted: février 27th, 2012 | Author: | Filed under: Dev, Java, Java EE, JAX-RS, Jersey, Tools, Tutorial | Tags: , , , , , , , | 24 Comments »

I recently had to consume a REST API with a cross-domain call and i’ve found a solution in this excellent blog post from Kdecherf.

The problem with this implementation is that you need to call a method for each of your api’s methods.

So with the ContainerResponseFilter interface, you could do this for ALL of your methods … transparently.

1 – Configure web.xml

You just have to add an init-param to your Jersey Servlet

<init-param>
	<param-name>com.sun.jersey.spi.container.ContainerResponseFilters</param-name>
	<param-value>com.ezakus.api.web.security.ResponseCorsFilter</param-value>
</init-param>

2 – ResponseCorsFilter

package com.ezakus.api.web.security;

import javax.ws.rs.core.Response;
import javax.ws.rs.core.Response.ResponseBuilder;

import com.sun.jersey.spi.container.ContainerRequest;
import com.sun.jersey.spi.container.ContainerResponse;
import com.sun.jersey.spi.container.ContainerResponseFilter;

public class ResponseCorsFilter implements ContainerResponseFilter {

	@Override
	public ContainerResponse filter(ContainerRequest req, ContainerResponse contResp) {

		ResponseBuilder resp = Response.fromResponse(contResp.getResponse());
		resp.header("Access-Control-Allow-Origin", "*")
	      		.header("Access-Control-Allow-Methods", "GET, POST, OPTIONS");

		String reqHead = req.getHeaderValue("Access-Control-Request-Headers");

		if(null != reqHead && !reqHead.equals("")){
			resp.header("Access-Control-Allow-Headers", reqHead);
		}

		contResp.setResponse(resp.build());
        	return contResp;
	}

}

And that’s all !

Thanks again to Kdecherf for his war upon JSONP and for sharing those headers :D


Using Jsp in a Jersey JAX-RS RESTful application

Posted: décembre 29th, 2011 | Author: | Filed under: Dev, Glassfish, Java, Java EE, JAX-RS, Jersey, Tools, Tutorial | Tags: , , , , , , , , , , , | 30 Comments »

So, ok we could easily produce some RESTful applications with Jersey.
But sometimes, the output could be very big to put in a method and a template could be useful.

Jersey provides MVC support for JSP pages.
There is a JSP template processor that resolves absolute template references to processable template references that are JSP.


1 – Configure web.xml

<filter>
	<filter-name>jersey</filter-name>
	<filter-class>com.sun.jersey.spi.container.servlet.ServletContainer</filter-class>
	<init-param>
		<param-name>com.sun.jersey.config.property.packages</param-name>
		<param-value>com.ezakus.web</param-value>
	</init-param>
	<init-param>
		<param-name>com.sun.jersey.config.property.JSPTemplatesBasePath</param-name>
		<param-value>/WEB-INF/jsp</param-value>
	</init-param>
	<init-param>
		<param-name>com.sun.jersey.config.property.WebPageContentRegex</param-name>
		<param-value>/(resources|(WEB-INF/jsp))/.*</param-value>
	</init-param>
</filter>
<filter-mapping>
	<filter-name>jersey</filter-name>
	<url-pattern>/*</url-pattern>
</filter-mapping>

Instead of a servlet mapping you need to have a filter.

With the JSPTemplatesBasePath param, you choose your jsp folder

	<param-name>com.sun.jersey.config.property.JSPTemplatesBasePath</param-name>
	<param-value>/WEB-INF/jsp</param-value>

And with the WebPageContentRegex you are able to serve static resources.
In the previous example, static resources are on the /resources/ or /WEB-INF/jsp/ path but you can put what you want :

	<param-name>com.sun.jersey.config.property.WebPageContentRegex</param-name>
	<param-value>/(resources|js|css|images)/.*</param-value>

2 – Return Viewable or Response

Now you can use the Viewable class with your jsp path

package com.ezakus.web;

import javax.ejb.Stateless;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;

import com.sun.jersey.api.view.Viewable;

@Stateless
@Path("/")
public class MyController {

    @GET
    @Produces("text/html")
    public Viewable index() {
    	return new Viewable("/index");
    }

}

Note : /index assume that you have a /WEB-INF/jsp/index.jsp on your path

You can also use the Response class

package com.ezakus.web;

import javax.ejb.Stateless;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.Response;

import com.sun.jersey.api.view.Viewable;

@Stateless
@Path("/")
public class MyController {

    @GET
    @Produces("text/html")
    public Response index() {
    	return Response.ok(new Viewable("/index")).build();
    }

}

3 – Using Viewable’s model

The Viewable object could be created with a model :

package com.ezakus.web;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import javax.ejb.Stateless;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.Response;

import com.sun.jersey.api.view.Viewable;

@Stateless
@Path("/")
public class MyController {

    @GET
    @Produces("text/html")
    public Response index() {
    	Map<String, Object> map = new HashMap<String, Object>();
        map.put("user", "usul");
        List<String> l = new ArrayList<String>();
        l.add("light saber");
        l.add("fremen clothes");
        map.put("items", l);
    	return Response.ok(new Viewable("/cart", map)).build();
    }

}

Note : Jersey will assign the model instance to the attribute « it » in the jsp. (Yes, life is hard)

<%@page contentType="text/html"%>
<%@page pageEncoding="UTF-8"%>
<%@taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c"%>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
   "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
  <title>Welcome!</title>
</head>
<body>
  <h1>Welcome ${it.user}!</h1>
  <p>
  	items in your cart :<br />
    <c:forEach var="item" items="${it.items}">
    	${item}<br />
    </c:forEach>
  </p>
</body>
</html>

Java EE 6 Glassfish 3.1.1 Cluster with Load-Balancer Installation

Posted: décembre 21st, 2011 | Author: | Filed under: Dev, Java, Java EE, Tutorial | Tags: , , , , , , , , , , , , | 12 Comments »

In this tutorial we’ll see how to install a Glassfish cluster with this architecture

 
architecture
 

Assumptions : there’s nothing on servers (either java, xauth or whatever you want)

 

Prepare Node 2 and Node 1


 

Add the glassfish user (with the same password on each) and choose the installation path

adduser glassfish mkdir /opt/glassfish3
chown glassfish:glassfish
/opt/glassfish3

Add non-free in sources.list for sun jdk installation

# vi /etc/apt/sources.list
deb http://mirror.ovh.net/debian/ squeeze main non-free
deb-src http://mirror.ovh.net/debian/ squeeze main non-free

Install sun jdk

apt-get update apt-get install sun-java6-jdk

 

Prepare LB


 

Add the glassfish user (still with the same password)

adduser glassfish

 

On Node 1


  • Install xauth

The simpliest way is to install glassfish in graphical mode so we need xauth for x forwarding with ssh.

apt-get install xauth

Logout and Log in with -X option.

  • Glassfish install

Download the GlassFish Server 3.1.1 Open Source Edition Full Platform

wget http://download.java.net/glassfish/3.1.1/release/glassfish-3.1.1-unix.sh
chmod +x glassfish-3.1.1-unix.sh ./glassfish-3.1.1-unix.sh

Choose custom, install and configure, choose the installation path (/opt/glassfish3 for us)

chown -R glassfish:glassfish 

Add the path to <installation path>/bin to glassfish user’s PATH (.bashrc .profile …)
With this, glassfish user could use asadmin command line tool

PATH="/opt/glassfish3/glassfish/bin:$PATH"
  • Glassfish config

To know what we could do with asadmin

asadmin list-commands

So let’s go

asadmin login
asadmin enable-secure-admin asadmin restart-domain
#If you forget the domain name : <strong>asadmin list-domains
</strong>
  • Installing Node 2 from Node 1

asadmin setup-ssh --generatekey=true node-2 lb-1
asadmin install-node node-2
  •  Create cluster

First, create the cluster config (based on the existing one),  all cluster nodes will share the same config.

asadmin copy-config default-config cluster-config

Now we could create the nodes

asadmin create-node-ssh --nodehost localhost node-1-ssh
asadmin create-node-ssh --sshuser glassfish --sshkeyfile /home/glassfish/.ssh/id_rsa --nodehost node-2 node-2-ssh

And finally the cluster and the instances

asadmin create-cluster --config cluster-config usul-cluster
asadmin create-instance --cluster usul-cluster --node node-1-ssh gf-1
asadmin create-instance --cluster usul-cluster --node node-2-ssh gf-2
# Verifying
asadmin list-instances --long=true
# Start the cluster
asadmin start-cluster usul-cluster

Load-Balancer setup


  • Install xauth because lb installer needs GUI

Edit : As John said in the comments : The Load Balancer installer requires a GUI on first install, but you can save the LB configuration to a file and subsequent installs can be done in a headless manner.

yum install xauth

We also need the JDK, so download java 6 sdk bin installer on java.sun.com (the bin one, not rpm.bin)

wget http://download.oracle.com/otn-pub/java/jdk/6u30-b12/jdk-6u30-linux-x64.bin

chmod +x jdk-6u30-linux-x64.bin
./jdk-6u30-linux-x64.bin
mv jdk1.6.0_30 /usr/local && cd /usr/local && ln -s jdk1.6.0_30 jdk
# Install in the system
# 1 because there's no java on the server, put 2 or 3 if you already have another java version)
# and launch alternatives --config
java alternatives --install /usr/bin/java java /usr/local/jdk/bin/java 1

# Verifying
java -version
  • Install and prepare Apache

yum install httpd mod_ssl openssl

For cert generation, be careful to put your server name in Common Name

# Generate private key
openssl genrsa -out ca.key 1024
# Generate CSR
openssl req -new -key ca.key -out ca.csr
# Generate Self Signed Key
openssl x509 -req -days 365 -in ca.csr -signkey ca.key -out ca.crt
# Copy the files to the correct locations
cp ca.crt /etc/pki/tls/certs
cp ca.key /etc/pki/tls/private/ca.key
cp ca.csr /etc/pki/tls/private/ca.csr

Edit /httpd/conf/ssl.conf and set the correct paths

SSLCertificateFile /etc/pki/tls/certs/ca.crt
SSLCertificateKeyFile /etc/pki/tls/private/ca.key

Now we need to tweak this installation because the load balancer installer needs some files and paths

# Folder for the glassfish DAS certificate
mkdir /etc/httpd/conf/ssl.crt
# Installer will look for files in $apache_home/conf/extra
ln-s /etc/httpd/conf.d /etc/httpd/conf/extra
touch /etc/httpd/conf/extra/httpd-ssl.conf
touch /etc/httpd/conf/extra/httpd-vhosts.conf
touch/etc/httpd/conf/extra/httpd-mpm.conf
# Installer will look for bin/apachectl and bin/envvars
mkdir /etc/httpd/bin
touch /etc/httpd/bin/envvars
touch /etc/httpd/bin/apachectl

Edit Mpm config file

PidFile "logs/httpd.pid"
LockFile "logs/accept.lock"

<IfModule mpm_prefork_module>
 StartServers 5
 MinSpareServers 5
 MaxSpareServers 10
 MaxClients 150
 MaxRequestsPerChild 0
</IfModule>

<IfModule mpm_worker_module>
 StartServers 2
 MaxClients 150
 MinSpareThreads 25
 MaxSpareThreads 75
 ThreadsPerChild 25
 MaxRequestsPerChild 0
</IfModule>

Edit apachectl

#!/bin/bash
/usr/sbin/apachectl -v

 

chmod 755 /etc/httpd/bin/apachectl
  • Export the DAS certificate

The load balancer needs an xml file to be updated : loadbalancer.xml By putting the certificate on the LB, we could update it from the DAS with apply-http-lb-changes

#Export crt
keytool \
 -export \
 -rfc \
 -alias s1as \
 -keystore <installPath>/glassfish/domains/usul/config/keystore.jks \
 -file ./glassfish.crt \
 -storepass changeit

# Send it on the LB
scp glassfish.crt root@lb-1:/etc/httpd/conf/

# The installer will put the crt into ssl.crt later
  • Install the load-balancer

Download loadbalancer : http://www.oracle.com/technetwork/middleware/glassfish/downloads/index.html

# Don't forget to ssh -X because of GUI intall
java -jar glassfish-lbconfigurator-3_1_1.jar

Choose apache and put the installation dir (/etc/httpd)

Choose the DAS cert in /etc/httpd/conf/

When finished, edit /etc/init.d/httpd and add

 if [ -f /etc/httpd/bin/envvars]; then
 . /etc/httpd/bin/envvars
 fi
  • Configure the ssl access

keytool -printcert -file /etc/httpd/conf/ssl.cert/glassfish.crt

Copy Serial number (UPPER CASE !!!) and Organization (O) and Organization Unit (OU)
Edit /etc/httpd/conf/extra/httpd-ssl.conf and edit both SSLRequire at the end of file with O OU and Serial (in Upper Case)
You will have something like this

 SSLVerifyClient require
 SSLVerifyDepth 1
 SSLRequireSSL
 SSLCACertificateFile /etc/httpd/conf/ssl.crt/glassfish.crt
 SSLRequire ( %{SSL_CIPHER} !~ m/^(EXP|NULL)-/ \
 and %{SSL_CLIENT_S_DN_O} eq "Oracle Corporation" \
 and %{SSL_CLIENT_S_DN_OU} eq "GlassFish" \
 and %{SSL_CLIENT_M_SERIAL} eq "4EEB6172" )

 SSLVerifyClient require
 SSLVerifyDepth 1
 SSLRequireSSL
 SSLCACertificateFile /etc/httpd/conf/ssl.crt/glassfish.crt
 SSLRequire ( %{SSL_CIPHER} !~ m/^(EXP|NULL)-/ \
 and %{SSL_CLIENT_S_DN_O} eq "Oracle Corporation" \
 and %{SSL_CLIENT_S_DN_OU} eq "GlassFish" \
 and %{SSL_CLIENT_M_SERIAL} eq "4EEB6172" )

Edit /etc/httpd/conf/extra/httpd-vhosts.conf

#Be sure to have :
NameVirtualHost *:80
<VirtualHost *:80>
 ServerName lb-1
 DocumentRoot "/var/www"
 #  The document root is not important, it must exist to avoid warnings or errors in logs but this is not used
</VirtualHost>

Raise the MaxClients in /etc/httpd/conf/extra/httpd-mpm.conf because installer put 1 that is not enough

chown -R apache:apache /etc/httpd
# just check before that apache is the user for apache in httpd.conf
/etc/init.d/httpd restart

Create the load-balancer in the DAS

asadmin create-http-lb --devicehost lb-1 --deviceport 443 --target usul-cluster lb-1

asadmin apply-http-lb-changes lb-1