Tuesday, September 23, 2014

ORM Tips

First Off, ORM is tougher than writing declarative SQL, make sure you have experienced developers, who are willing to "really understand" the data model, rather than developers who want to just "code-break-fix"


When you look at any persistent entity attributes think of them as following 3 categories:

  • "flat" attributes, that are stored in the table that the parent entity is mapped to
  • "referred" attributes, these merely refer to other entities, whose life cycle is independent of parent entity. Typically entities shown in dropdowns, lookups, master data entities etc
  • "cascade" attributes, refer to entities that may have CRUD cascades w r t the parent entity

The more the number of cascade attributes, the more the complexity in persisting the current entity
(this can be used for estimations)

Inserts are easy, updates are tricky. Allocate enough time for testing entity update, with different combinations of "cascade" attributes. (null, existing, new entities)

Complex cross entity (readonly) queries, need not go through the ORM. (caution: there are risks of stale/out-of-sync data, if not done properly)

The service code is not tested, till all lazy initialization has been exercised. Developers release the code as working too soon, but only when the services layer is invoked from the controllers, where-in the json serializers try to access the entire depth of the object graph, the code starts throwing data access exceptions. Working with shallow hibernate proxies often lulls us into false sense of security temporarily.

To translate entity objects into VO / TO objects and back, use object oriented code, rather than procedural, so let each VO / TO take care of its own mapping to corresponding entity object.


To test REST services, you can use simple HTML with jquery, but better is JUnit tests which use Spring's RestTemplate as REST Client and make REST service calls. This kind of test is very authentic (exercises controllers realistically)
and more importantly...the test code is strongly typed, so in case your VO changes, the compiler will break your test and catch your attention, this makes refactoring smooth.

Monday, September 8, 2014

Database multi-tenancy using Hibernate 4

With many business applications being written, with intent to be deployed on cloud, there is increasingly a need to design multi-tenancy into your application.


For persistence layer multi-tenant design offers a few options:
http://msdn.microsoft.com/en-us/library/aa479086.aspx


  1. separate database per tenant
  2. separate schema, same database per tenant
  3. same schema, same database for all tenant with discriminator column for tenant rows in same table


In this article, I am going to talk more about implementing the options 1 and 2 using the support in hibernate 4, for multi-tenancy.

Also for the database connections, we have choice of having a common database connection pool, from which connections are allocated to each tenant request, or separate connection pools for each tenant.
For the common connection pool approach, each connection before being used for data access, needs to be "primed" for usage against a tenant database or schema, using statements like "use <tenant_schema>" or "use <tenant_database>"


Multi-tenancy in Hibernate


For the "separate connection pool per tenant approach"

First we need to implement a custom connection provider as follows

import java.sql.Connection;
import java.sql.SQLException;

import org.apache.commons.dbcp2.BasicDataSource;
import org.hibernate.engine.jdbc.connections.spi.ConnectionProvider;

public class ConnectionProviderImpl implements ConnectionProvider {
 
 private final BasicDataSource basicDataSource = new BasicDataSource();
 
 public ConnectionProviderImpl(String database){
                //this should be read from properties file
  basicDataSource.setDriverClassName("com.mysql.jdbc.Driver");
  basicDataSource.setUrl("jdbc:mysql://localhost:3306/"+database);
  basicDataSource.setUsername("myuser");
  basicDataSource.setPassword("mypassword");
  basicDataSource.setInitialSize(2);
  basicDataSource.setMaxTotal(10);
 }

 @Override
 public boolean isUnwrappableAs(Class arg0) {
  return false;
 }

 @Override
 public  T unwrap(Class arg0) {
  return null;
 }

 @Override
 public void closeConnection(Connection arg0) throws SQLException {
  arg0.close();
 }

 @Override
 public Connection getConnection() throws SQLException {
  return basicDataSource.getConnection();
 }

 @Override
 public boolean supportsAggressiveRelease() {
  return false;
 }

}




Next we need to implement AbstractMultiTenantConnectionProvider of hibernate, as follows
Here I am maintaining a map of database identifiers against, connection providers.
When hibernate invokes the selectConnectionProvider( ) method with the tenant identifier, we use the tenant identifier to return the "appropriate" connection provider from the map.
It is necessary to implement a method like getAnyConnectionProvider( ), which should return, a sensible - default connection provider.

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;

import org.hibernate.engine.jdbc.connections.spi.AbstractMultiTenantConnectionProvider;
import org.hibernate.engine.jdbc.connections.spi.ConnectionProvider;

public class MultiTenantConnectionProvider extends AbstractMultiTenantConnectionProvider {
 
 private HashMap connProviderMap = new HashMap();
 
 public MultiTenantConnectionProvider(){
  

  List providerNames = new ArrayList();
  providerNames.add("default_db");
  providerNames.add("db1");
  providerNames.add("db2");
  //need to get above from properties file
  
    for (String providerName : providerNames) {
       connProviderMap.put(providerName, new ConnectionProviderImpl(providerName));
    }
  
 }
 
 @Override
 protected ConnectionProvider getAnyConnectionProvider() {
  System.out.println("inside MultiTenantConnectionProvider::getAnyConnectionProvider");
  return connProviderMap.get("default_db");
 }

 @Override
 protected ConnectionProvider selectConnectionProvider(String tenantId) {
  ConnectionProvider connectionProvider = connProviderMap.get(tenantId);
  if(connectionProvider == null)
   connectionProvider =  new ConnectionProviderImpl("default_db");
  
  return connectionProvider;
 } 

}



For common connection pool for all tenants.

We need to implement MultiTenantConnectionProvider of hibernate, as follows
Since we are using the same connection pool, in getConnection(), we need to prime the connection using a SQL statement like 'use <database>', so that the connection's further use will be in database as per tenant_id.
Also in releaseConnection( ), we have a 'use <default_db>' as a fallback.



import java.sql.Connection;
import java.sql.SQLException;

import org.hibernate.HibernateException;
import org.hibernate.engine.jdbc.connections.spi.ConnectionProvider;
import org.hibernate.engine.jdbc.connections.spi.MultiTenantConnectionProvider;

public class MultiTenantConnectionProviderWithSingleDBPool implements
  MultiTenantConnectionProvider {
 
 private final ConnectionProvider connectionProvider = new ConnectionProviderImpl(CurrentTenantIdentifierResolver.DEFAULT_TENANT_ID); 
 

 @Override
 public boolean isUnwrappableAs(Class arg0) {
  return false;
 }

 @Override
 public  T unwrap(Class arg0) {
  return null;
 }

 @Override
 public Connection getAnyConnection() throws SQLException {
  System.out.println("inside MultiTenantConnectionProvider::getAnyConnection");
  return connectionProvider.getConnection();
 }

 @Override
 public void releaseAnyConnection(Connection connection) throws SQLException {
  connectionProvider.closeConnection( connection );
 }

 @Override
 public Connection getConnection(String tenantIdentifier) throws SQLException {
  final Connection connection = getAnyConnection();
  try {
     connection.createStatement().execute( "USE " + tenantIdentifier );
  }
  catch ( SQLException e ) {
     throw new HibernateException(
       "MultiTenantConnectionProvider::Could not alter JDBC connection to specified schema [" +tenantIdentifier + "]",e);
  }
  return connection;
 }

 @Override
 public void releaseConnection(String tenantIdentifier, Connection connection) throws SQLException {
  try {
   connection.createStatement().execute( "USE default_db" );
  }
  catch ( SQLException e ) {
     throw new HibernateException(
     "Could not alter JDBC connection to specified schema [" +
   tenantIdentifier + "]",e);
  }
  connectionProvider.closeConnection( connection );
 }

 @Override
 public boolean supportsAggressiveRelease() {
  return false;
 }

}





refer hibernate docs
http://docs.jboss.org/hibernate/orm/4.1/devguide/en-US/html/ch16.html



Now, to persuade hibernate to use tenant identifier before any database access, we can
  1. manually set the tenant identifier on hibernate session factory
  2. we can implement hibernate tenant identifier resolver


Manually set the tenant identifier on hibernate session factory
This can be done, in a web application in any suitable interceptor where hibernate session is available
Session session = sessionFactory.withOptions()
        .tenantIdentifier( yourTenantIdentifier )
        ...
        .openSession();


Hibernate tenant identifier resolver can be implemented as follows.
Here I am using a ThreadLocal to get the tenant identifier. The threadlocal itself can be set early on in the http request thread in say, a servlet filter.


import org.hibernate.context.spi.CurrentTenantIdentifierResolver;

public class CurrentTenantIdentifierResolver implements
  CurrentTenantIdentifierResolver {
 
 public static ThreadLocal _tenantIdentifier = new ThreadLocal();
 public static String DEFAULT_TENANT_ID = "default_db";

 @Override
 public String resolveCurrentTenantIdentifier() {
  System.out.println("from inside resolveCurrentTenantIdentifier....");
  String tenantId = _tenantIdentifier.get();
  if(tenantId == null)
   tenantId = DEFAULT_TENANT_ID;
  
  System.out.println("threadlocal tenant id ="+tenantId);
  return tenantId;
 }

 @Override
 public boolean validateExistingCurrentSessions() {
  return true;
 }

}



Finally, the hibernate configuration for specifying that we are using hibernate mutli-tenancy, in hibernate.cfg.xml





That's it people,
Happy multi-tenanting :-)

Cheers!

Thursday, April 17, 2014

Using free openshift java PAAS, to run a web application on an online version of tomcat

Many times you have a war file, that you would like to deploy such that the application becomes available online.A classical use case is using a REST service as stub for your mobile app or thin client web app.

Openshift online PAAS is an easy solution for such free hotsitng of a java web application


Create a free account at open shift online 
Download their rhc command line tool
Fire following command to create 1 of 3 free apps, where name of new hosted web app is for example say, restgrails

rhc create-app restgrails tomcat-7

Go to the open shift web console and note down the ssh url of the git repository
setup ssh keys for the open shift git repo using command

rhc setup

If required, place the keys generated in <user-dir>\.ssh into <git-install-dir>\.ssh

fire following command on your local machine

git clone <git_url> restgrails

on your local machine the above git command will create a new directory called restgrails and checkout default web app from openshift git repo to your local machine

cd restgrails

Fire following commands to get rid of the default web app's src and pom.xml, since we will deploy a ready made war file onto the openshift PAAS

git rm -rf src/ pom.xml

git commit -am "deleted default source code"

Now prepare the ready made war file for deploying onto open shift

copy war file to restgrails/webapps/
rename war file to ROOT.war  --this is important if you want to map your webapp to the openshift online app URL

git add .
git commit -am "added new root.war to webapps"
git push --force

Above git push command, will trigger a deploy of the war file into the remote tomcat on open shift and soon the web app will become available online as say

http://restgrails-ideafountain.rhcloud.com/
ie
http://<web-app-name>-<domain-name-reg-with-openshift-online>.rhcloud.com/

you can tail the tomcat server logs with the following command
rhc tail restgrails

For databases, openshift online does support databases like mongodb, mysql and postgresql, so if your app uses hibernate, it can easily be configured to use online database like say mysql or postgresql, while running it online

Wednesday, March 26, 2014

Why is enterprise application development different from just programming?

Any monkey can write code that works!
Writing code that fulfills a functionality is a very important first step in the process of developing an enterprise application, but it is only the beginning, the enterprise application is far from done...

So what else is involved in developing an enterprise application?
As it turns out there are several other aspects of development that have to be considered:

  • Is the code unit tested with sufficient test coverage?
  • Is the code thread safe, in the face of multiple users accessing it?
  • Is the code transactional?
  • Is the code secure?
  • Is the code easy to debug? Is it instrumented with logging at appropriate levels?
  • Does the code handle exceptions and errors?
  • Is the code optimized for performance and for memory consumption?
  • Is the code resilient? Can it recover, from a failure?
  • Is the code easily extensible and maintainable?
  • Is the code portable across deployment environments?
  • Are all the configurable properties externalized and easily changed during deployment?
  • Is the code able to inter-operate with legacy systems to be integrated?

Answering all the above questions to the satisfaction of the non-functional requirements, is equally important. Many developers are aware of all the above aspects of development but completely ignore them, when asked about estimates when they think they can complete a particular functionality or feature of the enterprise application under development :-(

On a side note, even when we as developers decide to write our own reusable component or framework, please take into consideration that all the above aspects need to be built into the component/framework and hence many times it makes better sense to use an opensource component which is popular used or has withstood the test of time.Don't re-invent the wheel :-)


Taking an application from developers machine to production
Taking a functionally complete application from a developers machine upto production is a long walk. Lets see some of the common things that need to be done:

  • automated integration tests especially for regression testing
  • scripts for automated checkout, packaging and deployment
  • automated configuration of application  as per environment eg. integration, qc, uat and prod
  • scripts for application life cycle management - start, restart, stop, application
  • scripts/tools for application database cleanup and archival
  • scripts/tools for application log files cleanup and archival
  • scripts/tools for monitoring application health and sending alerts
  • scripts/tools for scrapping application logs for exceptions, etc and sending out alerts
  • scripts/tools for performance and memory monitoring of application
  • migrating existing database to new application database + testing app against migrated data
  • scripts for scheduling house keeping work around the application


As can be easily seen, there is a very substantial and important component of "operations" which needs to be undertaken, for successful implementation of any application. All these activities can hardly be classified as typical coding. These need to be factored into time estimates as well as workforce planning