Monday, September 14, 2015

OSGi - Update Java Source code programmatically


This time its OSGi.

For my new assignment only I was and is the team; this time requirement was

Many times naive developers code in Java and leaves 
    • code un-formatted
    • does not remove unnecessary imports
    • does not remove unused variables
    • etc....
 Can we correct such code up to certain extent pro-grammatically?  

As I gradually started looking into Sonar, SonarQube etc I find that all these tools gives problem with our code but not the solution to those problems.  Then I searched for such APIs/Frameworks which could give me both - Problems in my code and Solution to them.

I finally landed at Eclipse JDT world.  On further digging I find that I need to make one headless OSGi plugin/application to achieve what I want.

After making headless eclipse plugin and certain RnD, I was not getting workbench object which is only available once Eclipse IDE opens.  

  • So this is the case where IDE needs to be opened to run code/application.  
  • Once IDE is opened then project is loaded which needs to be examined that means you need eclipse project.  
  • Then warnings/errors that we can configure in Eclipse gives proposals for problems (errors and warnings).  These proposals contains priority & changes based on context which can be applied to certain problem as a solution.

Ex.  If there are two or more imports which are not used by your program and when program is parsed by Eclipse it will check for first unused import and gives two proposals which are - (1) To remove that import only (2) Organize import which will remove all such unused imports.

Approach 1
So what I did programmatically as my initial approach (Yes there is one more approach which I am using now):
  • Used APIs to open project first
  • From Eclipse Project convert it to Java project
  • Collect names of specific files (called Documents in Eclipse APIs)
  • Open file (it will load it in IDE)
  • Parse each file for problems
  • For each problem get JavaProposals
  • Pick best suited proposals or if you do not want to address certain problem skip it
  • Apply proposal
  • Save Document
  • Close Document (So that too many documents does not get open)
  • Close Project
  • This is how you can go for each file.

Approach 2
After some more RnD I changed my approach which is more performant:
  • In Eclipse APIs certain Action classes are available to execute certain menu actions such as FormatAllAction
  • There is one menu item in Eclipse which is Java -> Code Style -> Clean Up
  • This Clean Up can be used for 
    • Format a file (formatting options can be configured)
    • Organize imports
    • Remove unused variables
    • Add serialize UID 
    • and few more....
  • So, I created my own profile and applied required settings
  • Run Action class for CleanUp
  • And I am done.

In this approach you can only resolve problems which are available in Code Clean Up option while in #1 you can do anything which is available as proposal.

This way we can correct code up to certain extent.  There are more action classes which are available and can be used.  If I made any further changes then I will certainly update here.

Although now I am on my way of giving a thought if I can do same without opening IDE :)


Wednesday, December 3, 2014

Private cloud for elasticity scale and zero downtime


To be very honest, this was a POC and I personally enjoyed it very much.

Although, I am not allowed to disclose the concept as is, so sharing an abstract idea of building a cloud environment that could provide automated production ready environment.  All the deployments, configuration changes, nodes creation for traffic sharing will be dynamic in nature.

We had a team of 6 brilliant engineers for building up this POC. I was fortunate enough being a part of it.

Here we go...

Some background: This solution was mingled/cooked with more than 10 technologies. 

Problems targeted were hot deployment, zero downtime configuration changes, application switching contexts at run time, environment keeping track of machine health, creation of new virtual machines dynamically, new machines becoming part of load balancer dynamically and so on; wow, isn’t exciting?

This is what expected. Click here for enlarged image

This is something fascinating. Enjoy the list J.

  • vmWare ESXi Host Client / Server
  • Hyperic
  • Splunk
  • Zookeeper
  • Puppet
  • C-Tier
  • NoSQL,
  • Queues
  • Spring
  • SVN
  • Jenkins-Hudson
  • Nexus


So what we did, huhh!!

How it works – Whenever a developer make any changes in the code base and commit to the SVN, Hudson first runs few plugins including test cases on the build.  Once the green is received from all plugins and test cases run successfully, a fresh build is prepared and copied to the Nexus repository.
C-Tier copies that build and copies it over all the instances of Application Server running over different hosting machines. Being dynamic in nature Application Server will deploy the new builds on the fly.  If there are any configuration changes then C-Ties pushes those changes to ZooKeeper server.  The moment ZooKeeper receives configuration changes it notifies to all of its clients programs running.

Now this is application responsibility to track down ZooKeeper client’s configuration changes and apply it to the code. 

After successful deployment of a build on application server, two hawks hyperic (system monitoring tool) and splunk (log monitoring tool) will keep eye on that instance of application server and will report to the base in case of any alarming situation.  Right Sir!! (In a military style J)

In case traffic is increasing on the application, hyperic will raise an alarm based on certain configured rules then C-Tier will initiate a puppet agent.

A puppet agent will copy a deployment ready vmWare image and creates new virtual machine.  New agents for splunk, hyperic and c-tier will be installed additionally on it.  And finally application server and the application will become part of that new virtual machine.  At last new virtual machine will be registered in to load balancer for traffic sharing. 

Other tasks c-tier performs: If application server goes down at any moment, then hyperic agent; based on configured rules; take action (in case run a script) to start the application server again.

That’s it.
Please let me know if you need any other help or more information.



Tuesday, July 23, 2013

Using ElasticSearch to cache more than 4.6 million records and integrating with jQuery auto-complete highlighting


The topic is really interesting, right :)
Buzz words in the topic “Using ElasticSearch to cache more than 4.6 million records and integrating with jQuery auto-complete highlighting” is ElasticSearch.  Let's go through it.

Some background: In one of my project there is requirement to show auto completion box to display company names.  These company names were stored in one of the oracle table and in count total companies records are more than 4.6 million.

It was expected to
-          Fetch the records from database; populate the elastic-search index using these records. 
-          Integrate the calls for elastic-search from jQuery using ajax.
-          Server hits as the user types, fetching data from elastic-search and populate in jQuery auto-complete box.
-          Highlight the data matching in auto-complete box.


This is what expected.

  •           ElasticSearch
  •          jQuery

In the approach, there were multiple tasks that were completed.
1.       ElasticSearch
Elastic search is a “flexible and powerful open source, distributed real-time search and analytics engine for the cloud”.

I installed the elastic search 0.90.2 version.   

By default elastic search does not provide any way to populate its indices via any data source.  There is a plugin called jdbc-river (  River is basically a terminology used to populate indices with specific key.  There are other types of rivers available using which you can populate indices via CSV, Oracle, MySQL etc.

After downloading elastic search I set it up and then install the river-jdbc plugin (instructions are available on site).  Now using the river I executed below command to fetch the records from database and build indices.

PUT Body:
    "type" : "jdbc",
    "jdbc" : {
        "driver" : "oracle.jdbc.driver.OracleDriver",
        "url" : "jdbc:oracle:thin:@//[IP]:1521/[DB]",
        "user" : "user",
        "password" : "pwd",
        "sql" : "select col1, col2, col3 from table"
    "index" : {
        "index" : "index_name",
        "type" : "type_name"

You can put any string as index and type.  You can use any REST client from FF or Chrome to fire above query.  Just after successful execution, elastic search will start fetching the data and building indices.  In my case elastic search fetch 4.6 million records.  I specifically fetched 2 columns and third column was concatenation of first two.

Now elastic search is ready and populated.  I just need to hit the REST services of elastic search to get desired data. 

You can test your elastic search using below command
     Url: http://localhost:9200/index_name/type_name/_search
     POST Body
     {"from":0,"size":10, "query":{"field":{"COL1":"*760*" } } }

2.       jQuery
As a second part, jQuery need to be integrated.  I created one html page that has code to call elastic search REST services after user has typed 3 characters in one of the text box.  I call the records in the chunk of 10. 
After getting 10 matching records, process it to make a java-script array and pass it to auto-complete control. 
Then there is one hack or you can say MonkeyPatch (a terminology to override thirdparty’s function for custom behavior.) as a java script function to match the words with spaces in search result and highlight their all occurrences.

<html lang="en">
  <meta charset="utf-8" />
  <title>jQuery UI Autocomplete - Default functionality</title>
  <link rel="stylesheet" href="jquery-ui.css" />
  <script src="jquery-1.9.1.js"></script>
  <script src="jquery-ui.js"></script>


function monkeyPatchAutocomplete() {

          // Don't really need to save the old fn,
          // but I could chain if I wanted to
          var oldFn = jQuery.ui.autocomplete.prototype._renderItem;

          jQuery.ui.autocomplete.prototype._renderItem = function( ul, item) {
              //var re = new RegExp("^" + this.term, "i") ;
              var searchStr = this.term;
              if (searchStr.indexOf (" ")!=-1)
                  var re = new RegExp((replaceAll (" ", searchStr, "|")), "ig") ;
                  t = item.label.replace(re,"<span style='font-weight:bold;color:Blue;'>" + "$&" +  "</span>");
              } else {
                  var re = new RegExp(this.term, "i") ;
                  var t = item.label.replace(re,"<span style='font-weight:bold;color:Blue;'>" + "$&" +  "</span>");

              return jQuery( "<li></li>" )
                  .data( "item.autocomplete", item )
                  .append( "<a>" + t + "</a>" )
                  .appendTo( ul );

function replaceAll(oldStrPattern, str, newStrPattern)
  var temp="";
  if(str!=null && oldStrPattern!=null)
     var idx = str.indexOf(oldStrPattern);
     while (idx > -1)
         temp += str.substr (0, idx);
         temp += newStrPattern;
         str=str.substr (idx+oldStrPattern.length,str.length);
         idx = str.indexOf(oldStrPattern);
     temp += str;
   return temp;

  jQuery(function() { = true;

    jQuery( "#tags" ).autocomplete({

      source: function( request, response ) {
        var isDuns = false;
        var val = jQuery("#tags").val();
        var arr = new Array ();
        try {
            var i = parseInt (val);
            isDuns = !isNaN(i);
        } catch (e) {
            isDuns = false;

        if (isDuns){
            val = "*"+val+"*";
        } else {
            val = "+"+val+"*";

        if (AUTOCOMPLETE_VIEW==2)
            var dt = {"from":0,"size":10, "query":{ "field" : { "COL2" : val } } };
            if (val.indexOf(" ")!=-1)
                val = replaceAll (" ", val, "* +");
                dt = {"from":0,"size":10, "query":{ "field" : { "COL2" : val } } };
        } else {
            //var dt = {"from":0,"size":10,"query":{"wildcard" : { "COL1" : val+"*" }},"sort":[{"COL1":{"order":"asc"}}]};
            var dt = {"from":0,"size":10, "query":{ "field" : { "COL1" : val } } };
            if (isDuns){
                dt = {"from":0,"size":10, "query":{ "field" : { "COL3" : val } } }
            if (val.indexOf(" ")!=-1)
                val = replaceAll (" ", val, "* +");
                dt = {"from":0,"size":10, "query":{ "field" : { "COL1" : val } } };
                isDuns = false;

        if (val != null && val.length > 2) {

                type: "POST",
                url: "",
                dataType: "json",
                async: false,
                data: JSON.stringify(dt),
                contentType: "application/json; charset=utf-8",
                success: function(data) {
                    var result=data;
                    var idx = 0;
                    var hitsJson = (data.hits.hits);                  
                    for (var key in hitsJson) {
                      if (hitsJson.hasOwnProperty(key)) {
                          if (AUTOCOMPLETE_VIEW==2) {
                            arr.push ((hitsJson[key]._source.COL2));
                          } else {
                            if (!isDuns){
                                arr.push ((hitsJson[key]._source.COL1));
                            } else {
                                arr.push ((hitsJson[key]._source.COL3));

                , error: function (xhr) {
                    alert ("err");
                    alert (JSON.stringify(xhr));
    }, minLength: 3, delay: 300


<div class="ui-widget">
  <label for="tags">Tags: </label>
  <input id="tags" />


I used jQuery word instead of $ as in project multiple JS frameworks were used and conflicts were detected.

That’s it.
Please let me know if you need any other help or more information.



Monday, July 1, 2013

Spring security @PreAuthorize example and generic exception


In my recent project I have working on securing spring methods exposed by controllers via @RequestMappings. 

I have to cross check the permissions/authorities assigned to the user who has logged-in to the system.  If the authority mentioned above the method matches then execution goes successfully else exception is throws.

·         Spring
·         Hibernate
·         Tomcat
·         Maven
·         jQuery

I made the class that checks the user who is logged-in at present (I made the UserInfo object to be flowed in each request object after successful authentication.  For details please refer my blog

Ingredients to cook above code are:
Firstly I made my custom class extending as BasePermissionEvaluator.  This class stores the object of UserInfo (refer blog mentioned above for details) and has below structure.

public class BasePermissionEvaluator implements PermissionEvaluator
    private UserInfo userInfo;
     * @return the userInfo
    public UserInfo getUserInfo()
        return userInfo;

     * @param userInfo the userInfo to set
    public void setUserInfo(UserInfo userInfo)
        this.userInfo = userInfo;

    public boolean hasPermission(Authentication authentication, Object targetDomainObject, Object permission)
        boolean hasPermission = false;

        if (authentication != null && permission instanceof String)
            hasPermission = userInfo.hasPermission((String)permission);
        } else
            hasPermission = false;
        return hasPermission;

    public boolean hasPermission(Authentication authentication, Serializable targetId, String targetType, Object permission)
        throw new RuntimeException("Id and Class permissions are not supperted by this application");

Corresponding applicationContext.xml entries
<security:global-method-security pre-post-annotations="enabled">
    <security:expression-handler ref="expressionHandler" />
<bean id="webExpressionHandler" class="" />

    <bean id="expressionHandler"
        <property name="permissionEvaluator" ref="permissionEvaluator" />

    <bean id="permissionEvaluator"
        class=" BasePermissionEvaluator" />

If the exception is thrown by any of the method then handles it using below entries
        <property name="exceptionMappings">
                <prop key="java.lang.Exception">/pages/common/error</prop>
Here you are supposed to create on error.jsp inside pages/common folder of you context root folder.

Below is one of the Controller’s methods secured via @PreAuthorize annotation.
    @RequestMapping(value = "service/user/", method = RequestMethod.GET)
    public @ResponseBody
    List<User> list()
        return userService.list();

<%@ page isErrorPage="true" %>
<!doctype html>
     <meta charset="utf-8">
     <title>vCare - for you</title>
     <link rel="stylesheet" type="text/css" href="<%=ctx%>/content/css/bootstrap.css" />
     <link rel="stylesheet" type="text/css" href="<%=ctx%>/content/css/bootstrap-responsive.css" />
     <link rel="stylesheet" type="text/css" href="<%=ctx%>/content/css/default.css" />
     <script type="text/javascript" src="<%=ctx%>/content/scripts/jquery-1.8.2.js"></script>
     <script type="text/javascript" src="<%=ctx%>/content/scripts/bootstrap.min.js"></script>
     <div class="header">
           <div class="wrapper row-fluid">
                <div class="span12">
                     <div class="logo">
                           <a href="#">
                                <img src="<%=ctx%>/content/images/logo-150-60.png" border="0" />
     <div class="main-content">
           <div class="wrapper">
                <div class="error-wrap">
                     <h3>Sorry, an error occurred.</h3>
                     <hr />
                     <p>Below are the details: </p>
                     <div class="alert alert-error">
                     <hr />
                     <a class="btn btn-success" href="<%=request.getContextPath ()%>/">Go Back</a>

Please let me know if you need any other help or more information.