IBM Developer SuperStar 2012 Contest

http://ibm.com/developerworks/superstar

Developer Contest - Delhi

Contest Link

The contest will test your skills on some of the technologies covered at the upcoming IndicThreads Conference On Software Development to be held on 13-14 July 2012 in Delhi, India. 

One of the major benefits of the Java language in comparison with other object oriented languages (like C++) is that programmers do not have to handle memory allocation during execution of the program. It is totally delegated to the garbage collector (GC) which is in charge of removing unused objects to release memory.

JVM memory organisation

The memory of the JVM is first slitted into 2 parts :

1. the heap memory
2. the non heap memory

The heap memory

The heap is devided into two generations: The Young Generation and the Tenured Generation.

The heap contains all Objects created during execution, this space should be frequently visited by the GC in order to clean up unused objects (=objects not referenced anymore). It is also slitted into different section according to the lifetime of the objects. Initially all objects are stored into the Eden section. Most of this objects will then be destroyed at the next GC visit because they are not used anymore. But some of them need to be kept because they have a longer lifetime and they will be used in the future. Therefore they are moved into the Survivor bucket. In the Survivor bucket GC calls are less frequent than in the Eden bucket. This two buckets represent the Young generation and contains all the “newly created” objects.

If objects stored in Survivor buckets, survive to other GC visits they are then moved into the Tenured generation (or Old generation) bucket until they are destroyed by the Garbage Collector.

The non heap memory

It comprises of ‘Method Area’ and other memory required for internal processing.

1. Method Area
• The method area is responsible for storing class information. The Class-Loader will load the bytecode of a class and will pass it to the JVM. The JVM will generate an internal class representation of the bytecode and store it in the method area. The internal representation of a class will have the following data areas:
• Runtime Constant Pool -  Numeric constants of the class of types int, long, float or double, String-constants and symbolic references to all methods, attributes and types of this class.
• Method Code - The implementation (code) of all methods of this class including constructors etc.
• Attributes -  A list of all named attributes of this class.
• FieldsValues of all fields of this class as references to the Runtime Constant Pool
2. Java Stacks or Frames
• Every time a method is invoked a stack frame is created and pushed onto the Java Stack. When the method terminates, this frame is popped off the stack. Thus, the top-most frame of the Java stack always belongs to the method that is currently being executed by the JVM.

In addition to the heap and method area, that are available for all threads of a JVM, every thread also has exclusive access to memory that is created for each thread:

• PC Register The Program Counter register. The register points to the current JVM instruction of the method the thread is executing, if the method is not a native method. If it is a native method the content of the PC register is not defined.
• Java Virtual Machine Stack Each thread gets its own stack on which so called Frames are pushed for each method the thread currently executed. This means that there can be many frames on the stack for nested method calls – but there is only one frame active at the same time for one thread. The frame contains the local variables of the method, a reference to the Runtime Constant Pool of the method’s class and an operand stack for the execution of JVM operations. (The JVM is a stack machine!)
• Native Methode Stack Native methods get its own stack, the so called „C-Stack“.

Structure of JVM

http://docs.oracle.com/javase/specs/jvms/se5.0/html/Overview.doc.html

Consequences

Bad tuning of the JVM can lead to several issues. The most obvious is that you set memory parameters too low to run your application so it cannot even start. But there are others.

Errors

If your memory parameters are too low you will probably have to tackle OutOfMemoryError since your application requires more space than what you allocated.

Non optimal performance

If your memory parameters are too low you will decrease the performance of the application. Indeed, the GC has to run very often in order to maintain the available space, which requires some time and also needs to stop the application.

Excessive footprint

On the other hand, allocating a huge memory to the JVM is also a bad practice because you will use (consume) much more than what you need and also because other resources won’t be able to use this unused space : OS, database, …. Therefore this may lead to non optimal performance as well.

Reference http://batterywalam.wordpress.com/category/j2ee-diagnostics/jvm-tuning/jvm-memory-organization

XA and non XA Transactions

Came across this wonderful explanation by Mike regarding XA transactions here

An XA transaction, in the most general terms, is a "global transaction" that may span multiple resources. A non-XA transaction always involves just one resource. An XA transaction involves a coordinating transaction manager, with one or more databases (or other resources, like JMS) all involved in a single global transaction. Non-XA transactions have no transaction coordinator, and a single resource is doing all its transaction work itself (this is sometimes called local transactions).

XA transactions come from the X/Open group specification on distributed, global transactions. JTA includes the X/Open XA spec, in modified form. Most stuff in the world is non-XA - a Servlet or EJB or plain old JDBC in a Java application talking to a single database. XA gets involved when you want to work with multiple resources - 2 or more databases, a database and a JMS connection, all of those plus maybe a JCA resource - all in a single transaction. In this scenario, you'll have an app server like Websphere or Weblogic or JBoss acting as the Transaction Manager, and your various resources (Oracle, Sybase, IBM MQ JMS, SAP, whatever) acting as transaction resources. Your code can then update/delete/publish/whatever across the many resources. When you say "commit", the results are commited across all of the resources. When you say "rollback", _everything_ is rolled back across all resources.

The Transaction Manager coordinates all of this through a protocol called Two Phase Commit (2PC). This protocol also has to be supported by the individual resources. In terms of datasources, an XA datasource is a data source that can participate in an XA global transaction. A non-XA datasource generally can't participate in a global transaction (sort of - some people implement what's called a "last participant" optimization that can let you do this for exactly one non-XA item). 

Memory usage of Java Strings

If you're used to using a language such as C and are not used to dealing with Unicode, you may expect a string to essentially take up one byte per character plus a single byte terminator. But in a language such as Java, gone are those days:

• every object has at least 8 bytes of housekeeping data, and arrays 12 bytes, and will be padded to a multiple of 16 bytes (in 32-bit versions of Hotspot);
• a Java String actually consists of more than one object;
• a Java char takes up two bytes, even if you're using them to store boring old ASCII values that would fit into a single byte;
• a Java String contains some extra variables that you might not have considered.

How to calculate String memory usage

For reasons we'll explore below, the minimum memory usage of a Java String is generally as follows:

Minimum String memory usage (bytes) = 8 * (int) ((((no chars) * 2) + 38) / 8)

Or put another way:

• multiply the number of characters of the String by two;
• add 38;
• if the result is not a multiple of 8, round up to the next multiple of 8;
• the result is generally the minimum number of bytes taken up on the heap by the String.

Understanding String memory usage

To understand the above calculation, we need to start by looking at the fields on a String object. A String contains the following:

• a char array— thus a separate object— containing the actual characters;
• an integer offset into the array at which the string starts;
• the length of the string;
• another int for the cached calculation of the hash code.

This means even if the string contains no characters, it will require 4 bytes for the char array reference, plus 3*4=12 bytes for the three int fields, plus 8 bytes of object header. This gives 24 bytes (which is a multiple of 8 so no "padding" bytes are needed so far). Then, the (empty) char array will require a further 12 bytes (arrays have an extra 4 bytes to store their length), plus in this case 4 bytes of padding to bring the memory used by the char array object up to a multiple of 16. So in total, an empty string uses 40 bytes.

If the string contains, say, 17 characters, then the String object itself still requires 24 bytes. But now the char array requires 12 bytes of header plus 17*2=34 bytes for the seventeen chars. Since 12+34=46 isn't a multiple of 8, we also need to round up to the next multiple of 8 (48). So overall, our 17-character String will use up 48+24 = 72 bytes. As you can see, that's quite a long way off the 18 bytes that you might have expected if you were used to C programming in the "good old days"¹.

Memory usage of Java objects

General formula for calculating memory usage

In general, the heap memory used by a Java object in Hotspot consists of:

• an object header, consisting of a few bytes of "housekeeping" information;
• memory for primitive fields, according to their size ;
• memory for reference fields (4 bytes each);
• padding: potentially a few "wasted" unused bytes after the object data, to make every object start at an address that is a convenient multiple of bytes and reduce the number of bits required to represent a pointer to an object.
  
  

  Java type	Bytes required
  boolean	1
  byte
  char	2
  short
  int	4
  float
  long	8
  double

  
  
  • a normal object requires 8 bytes of "housekeeping" space;
  • arrays require 12 bytes (the same as a normal object, plus 4 bytes for the array length).

Object size granularity

In Hotspot, every object occupies a number of bytes that is a multiple of 8. If the number of bytes required by an object for its header and fields is not a multiple 8, then you round up to the next multiple of 8.

This means, for example, that:

• a bare Object takes up 8 bytes;
• an instance of a class with a single boolean field takes up 16 bytes: 8 bytes of header, 1 byte for the boolean and 7 bytes of "padding" to make the size up to a multiple of 8;
• an instance with eight boolean fields will also take up 16 bytes: 8 for the header, 8 for the booleans; since this is already a multiple of 8, no padding is needed;
• an object with a two long fields, three int fields and a boolean will take up:
  • 8 bytes for the header;
  • 16 bytes for the 2 longs (8 each);
  • 12 bytes for the 3 ints (4 each);
  • 1 byte for the boolean;
  • a further 3 bytes of padding, to round the total up from 37 to 40, a multiple of 8.

Java Frameworks

Struts

When Java servlets were first invented, many programmers quickly realized that they were a Good Thing. They were faster and more powerful than standard CGI, portable, and infinitely extensible. But writing HTML to send to the browser in endless println() statements was tiresome and problematic. The answer to that was JavaServer Pages, which turned Servlet writing inside-out. Now developers could easily mix HTML with Java code, and have all the advantages of servlets. The sky was the limit!

The Apache Struts Project was launched in May 2000 by Craig R. McClanahan to provide a standard MVC framework to the Java community.

• Model: System State and Business Logic JavaBeans
• View: JSP Pages and Presentation Components
• Controller: ActionServlet and ActionMapping

Spring

Spring is the most popular application development framework for enterprise Java™. Millions of developers use Spring to create high performing, easily testable, reusable code without any lock-in.

JSF 2.0 - Oracle Mojarra JavaServer Faces

JavaServer(TM) Faces technology simplifies building user interfaces for JavaServer applications. Developers of various skill levels can quickly build web applications by: assembling reusable UI components in a page; connecting these components to an application data source; and wiring client-generated events to server-side event handlers.

DOJO

Dojo is a collection of JavaScript utilities providing solutions for a vast array of problems faced by the professional JavaScript developer.

Native Modifier

The native Modifier : In java applications, sometimes we will want to use a method that exists outside the JVM. In this scenario, the native modifier can help, native modifier can only apply to a method. Just as the abstract keyword, the native keyword indicates that the implementation of the method exists elsewhere. In case of abstract, the implementation of a method may exist in a subclass of the class in which the abstract method is declared. 

But in the case of native, the implementation of the method exists in a library outside the JVM. The native method is usually implemented in a non-Java language such as C or C++. Before a native method can be invoked, a library that contains the method must be loaded and that library is loaded by making the following system call:

System.loadLibrary("libraryName");
To declare a native method, precede the method with the native modifier, but do not define any body for the method. For example:

public native  void Nativemethod() ;

After you declare a native method, you must write the native method and follow a complex series of steps to link it with your Java code. For example, the following code fragment presents an example of loading a library named NativeMethodDef which contains a method named NativeMethod():

Java Tutorialslass NativeModifierExample
 {
       native void NativeMethod();
       static 
     {
            System.loadLibrary("NativeMethodDef");
           }
     }

Note that the library is loaded in a static code block which suggest that the library is loaded at the class load time, so it is there when a call to the native method is made. One can use the native method in the same way as we use a non-native method. For example, the following two lines of code would invoke the native method,

NativeModifierExample obj = new NativeModifierExample();
obj.NativeMethod();

Marker Interface

Marker interface is used as a tag to inform a message to the java compiler so that it can add special behavior to the class implementing it. Java marker interface has no members in it.

Lets take the java.io.Serializable marker interface. It doesn’t has any members defined it it. When a java class is to be serialized, you should intimate the java compiler in some way that there is a possibility of serializing this java class. In this scenario, marker interfaces are used. The java class which may be serialized has to implement the java.io.Serializable marker interface. In such way, we are intimating the java compiler.

From java 1.5, the need for marker interface is eliminated by the introduction of the java annotation feature. So, it is wise to use java annotations than the marker interface. It has more feature and advantages than the java marker interface.

We cannot create marker interfaces, as you cannot instruct JVM to add special behavior to all classes implementing (directly) that special interface.

Java Marker Interface Examples

• java.lang.Cloneable
• java.io.Serializable
• java.util.EventListener

JEHA (Java Exception HAndler)

Exception handling is one of the most important features of Java language. Codifying try and catch blocks, however, is not a pleasant task. Sometimes it is necessary to repeat catch logic in many places of the program and be alert in choosing the correct exception class to use.

How about eliminate all code in catch blocks and make everything became annotations? This is the main idea of Jeha: Annotate your class. Jeha do the rest.

http://jeha.sourceforge.net/

Reverse Line Reader

We would need a class to read the log files in reverse order to display the latest message.

   1: package com.whitelotus.util;

   2:  

   3: import java.io.ByteArrayOutputStream;

   4: import java.io.File;

   5: import java.io.IOException;

   6: import java.io.RandomAccessFile;

   7: import java.io.UnsupportedEncodingException;

   8: import java.nio.ByteBuffer;

   9: import java.nio.channels.FileChannel;

  10:  

  11:  

  12: public class ReverseLineReader {

  13:     private static final int BUFFER_SIZE = 8192;

  14:  

  15:     private final FileChannel channel;

  16:     private final String encoding;

  17:     private long filePos;

  18:     private ByteBuffer buf;

  19:     private int bufPos;

  20:     private byte lastLineBreak = '\n';

  21:     private ByteArrayOutputStream baos = new ByteArrayOutputStream();

  22:  

  23:     public ReverseLineReader(File file, String encoding) throws IOException {

  24:         RandomAccessFile raf = new RandomAccessFile(file, "r");

  25:         channel = raf.getChannel();

  26:         filePos = raf.length();

  27:         this.encoding = encoding;

  28:     }

  29:  

  30:     public String readLine() throws IOException {

  31:         while (true) {

  32:             if (bufPos < 0) {

  33:                 if (filePos == 0) {

  34:                     if (baos == null) {

  35:                         return null;

  36:                     }

  37:                     String line = bufToString();

  38:                     baos = null;

  39:                     return line;

  40:                 }

  41:  

  42:                 long start = Math.max(filePos - BUFFER_SIZE, 0);

  43:                 long end = filePos;

  44:                 long len = end - start;

  45:  

  46:                 buf = channel.map(FileChannel.MapMode.READ_ONLY, start,

  47:                         len);

  48:                 bufPos = (int) len;

  49:                 filePos = start;

  50:             }

  51:  

  52:             while (bufPos-- > 0) {

  53:                 byte c = buf.get(bufPos);

  54:                 if (c == '\r' || c == '\n') {

  55:                     if (c != lastLineBreak) {

  56:                         lastLineBreak = c;

  57:                         continue;

  58:                     }

  59:                     lastLineBreak = c;

  60:                     return bufToString();

  61:                 }

  62:                 baos.write(c);

  63:             }

  64:         }

  65:     }

  66:  

  67:     private String bufToString() throws UnsupportedEncodingException {

  68:         if (baos.size() == 0) {

  69:             return "";

  70:         }

  71:  

  72:         byte[] bytes = baos.toByteArray();

  73:         for (int i = 0; i < bytes.length / 2; i++) {

  74:             byte t = bytes[i];

  75:             bytes[i] = bytes[bytes.length - i - 1];

  76:             bytes[bytes.length - i - 1] = t;

  77:         }

  78:  

  79:         baos.reset();

  80:  

  81:         return new String(bytes, encoding);

  82:     }

  83: }

Using ReverseLineReader

   1: public static void main(String[] args) throws IOException {

   2:       File file = new File("tps.log");

   3:       RandomAccessFile raf = new RandomAccessFile(file, "rw");

   4:       for(int i=0;i<=10;i++) {

   5:           raf.writeBytes("HELLO " + i + "\n");

   6:       }

   7:       raf.close();

   8:       ReverseLineReader reader = new ReverseLineReader(file, "UTF-8");

   9:  

  10:       String line;

  11:       while ((line = reader.readLine()) != null) {

  12:           System.out.println(line);

  13:       }

  14:   }

  15:  

  16:  

Java retry strategy

http://fahdshariff.blogspot.in/2009/08/retrying-operations-in-java.html

There are many cases in which you may wish to retry an operation a certain number of times. Examples are database failures, network communication failures or file IO problems.
Approach 1
This is the traditional approach and involves a counter and a loop.

   1: final int numberOfRetries = 5 ;

   2: final long timeToWait = 1000 ;

   3:  

   4: for (int i=0; i<numberOfRetries; i++) {

   5:  //perform the operation

   6:  try {

   7:   Naming.lookup("rmi://localhost:2106/MyApp");

   8:   break;

   9:  }

  10:  catch (Exception e) {

  11:   logger.error("Retrying...",e);

  12:   try {

  13:    Thread.sleep(timeToWait);

  14:   }

  15:   catch (InterruptedException i) {

  16:   }

  17:  }

  18: }

Approach 2
In this approach, we hide the retry counter in a separate class called RetryStrategy and call it like this:  

   1: public class RetryStrategy

   2: {

   3:  public static final int DEFAULT_NUMBER_OF_RETRIES = 5;

   4:  public static final long DEFAULT_WAIT_TIME = 1000;

   5:  

   6:  private int numberOfRetries; //total number of tries

   7:  private int numberOfTriesLeft; //number left

   8:  private long timeToWait; //wait interval

   9:  

  10:  public RetryStrategy()

  11:  {

  12:   this(DEFAULT_NUMBER_OF_RETRIES, DEFAULT_WAIT_TIME);

  13:  }

  14:  

  15:  public RetryStrategy(int numberOfRetries, long timeToWait)

  16:  {

  17:   this.numberOfRetries = numberOfRetries;

  18:   numberOfTriesLeft = numberOfRetries;

  19:   this.timeToWait = timeToWait;

  20:  }

  21:  

  22:  /**

  23:   * @return true if there are tries left

  24:   */

  25:  public boolean shouldRetry()

  26:  {

  27:   return numberOfTriesLeft > 0;

  28:  }

  29:  

  30:  /**

  31:   * This method should be called if a try fails.

  32:   *

  33:   * @throws RetryException if there are no more tries left

  34:   */

  35:  public void errorOccured() throws RetryException

  36:  {

  37:   numberOfTriesLeft --;

  38:   if (!shouldRetry())

  39:   {

  40:    throw new RetryException(numberOfRetries +

  41:      " attempts to retry failed at " + getTimeToWait() +

  42:      "ms interval");

  43:   }

  44:   waitUntilNextTry();

  45:  }

  46:  

  47:  /**

  48:   * @return time period between retries

  49:   */

  50:  public long getTimeToWait()

  51:  {

  52:   return timeToWait ;

  53:  }

  54:  

  55:  /**

  56:   * Sleeps for the duration of the defined interval

  57:   */

  58:  private void waitUntilNextTry()

  59:  {

  60:   try

  61:   {

  62:    Thread.sleep(getTimeToWait());

  63:   }

  64:   catch (InterruptedException ignored) {}

  65:  }

  66:  

  67:  public static void main(String[] args) {

  68:   RetryStrategy retry = new RetryStrategy();

  69:   while (retry.shouldRetry()) {

  70:    try {

  71:     Naming.lookup("rmi://localhost:2106/MyApp");

  72:     break;

  73:    }

  74:    catch (Exception e) {

  75:     try {

  76:      retry.errorOccured();

  77:     }

  78:     catch (RetryException e1) {

  79:      e.printStackTrace();

  80:     }

  81:    }

  82:   }

  83:  }

  84: }

Approach 3
Approach 2, although cleaner, hasn't really reduced the number of lines of code we have to write. In the next approach, we hide the retry loop and all logic in a separate class called RetriableTask. We make the operation that we are going to retry Callable and wrap it in a RetriableTask which then handles all the retrying for us, behind-the-scenes:

   1: public class RetriableTask<T> implements Callable<T> {

   2:  

   3:  private Callable<T> task;

   4:  public static final int DEFAULT_NUMBER_OF_RETRIES = 5;

   5:  public static final long DEFAULT_WAIT_TIME = 1000;

   6:  

   7:  private int numberOfRetries; // total number of tries

   8:  private int numberOfTriesLeft; // number left

   9:  private long timeToWait; // wait interval

  10:  

  11:  public RetriableTask(Callable<T> task) {

  12:   this(DEFAULT_NUMBER_OF_RETRIES, DEFAULT_WAIT_TIME, task);

  13:  }

  14:  

  15:  public RetriableTask(int numberOfRetries, long timeToWait,

  16:                       Callable<T> task) {

  17:   this.numberOfRetries = numberOfRetries;

  18:   numberOfTriesLeft = numberOfRetries;

  19:   this.timeToWait = timeToWait;

  20:   this.task = task;

  21:  }

  22:  

  23:  public T call() throws Exception {

  24:   while (true) {

  25:    try {

  26:     return task.call();

  27:    }

  28:    catch (InterruptedException e) {

  29:     throw e;

  30:    }

  31:    catch (CancellationException e) {

  32:     throw e;

  33:    }

  34:    catch (Exception e) {

  35:     numberOfTriesLeft--;

  36:     if (numberOfTriesLeft == 0) {

  37:      throw new RetryException(numberOfRetries +

  38:      " attempts to retry failed at " + timeToWait +

  39:      "ms interval", e);

  40:     }

  41:     Thread.sleep(timeToWait);

  42:    }

  43:   }

  44:  }

  45:  

  46:  public static void main(String[] args) {

  47:   Callable<Remote> task = new Callable<Remote>() {

  48:    public Remote call() throws Exception {

  49:     String url="rmi://localhost:2106/MyApp";

  50:     return (Remote) Naming.lookup(url);

  51:    }

  52:   };

  53:  

  54:   RetriableTask<Remote> r = new RetriableTask<Remote>(task);

  55:   try {

  56:    r.call();

  57:   }

  58:   catch (Exception e) {

  59:    e.printStackTrace();

  60:   }

  61:  }

  62: }

//creates a task which will retry 3 times with an interval of 5 seconds
RetriableTask r = new RetriableTask(3, 5000, new Callable(){
    public Object call() throws Exception{
        //put your code here
    }
});
r.call();