Amazon

Friday, August 4, 2017

static vs volatile in java

 A very common interview question asked from mid level experience java developer and the question is: What is the difference between static and volatile variable in java?

So lets try to divide it in two parts:

 What is a Static variable in java:

Declaring a static variable in Java, means that there will be only one copy, no matter how many objects of the class are created. The variable will be accessible even with no Objects created at all. However, threads may have locally cached values of it.


What is volatile in java:

When a variable is volatile and not static, there will be one variable for each Object. So, on the surface it seems there is no difference from a normal variable but totally different from static. However, even with Object fields, a thread may cache a variable value locally.


Static and volatile are introduced for special purposes in java. Volatile keyword mainly used in multi threading environment. How?

Declaring a variable as volatile (be it static or not) states that the variable will be accessed frequently by multiple threads. In Java, this boils down to instructing threads that they can not cache the variable's value, but will have to write back immediately after mutating so that other threads see the change. (Threads in Java are free to cache variables by default). 

 

An additional interesting question would be: Is there a difference between a static and a static volatile variable?

static variable is stored once per class. A static volatile variable is stored once per class and will be accessed frequently by multiple threads, i.e. reads cannot be cached.


Even if you access a static value through multiple threads, each thread can have its local cached copy! To avoid this you can declare the variable as static volatile and this will force the thread to read each time the global value.
However, volatile is not a substitute for proper synchronization!


These are the basic differences between static and volatile as I can think off.

Please do add a comment if you found some other difference, I will add them in this post.

Thanks for reading.

Tuesday, July 4, 2017

PermGen Vs Metaspace

As you saw in Java 8 PermGen space has been decommission. In JDK 8, classes metadata is now stored in the native heap and this space is called Metaspace.

I recommend that you read the PermGen removal summary and comments from Jon on this subject. 

In summary:

PermGen space situation
  • This memory space is completely removed.
  • The PermSize and MaxPermSize JVM arguments are ignored and a warning is issued if present at start-up.
Metaspace memory allocation model
  • Most allocations for the class metadata are now allocated out of native memory.
  • The klasses that were used to describe class metadata have been removed.
Metaspace capacity
  • By default class metadata allocation is limited by the amount of available native memory (capacity will of course depend if you use a 32-bit JVM vs. 64-bit along with OS virtual memory availability).
  • A new flag is available (MaxMetaspaceSize), allowing you to limit the amount of native memory used for class metadata. If you don’t specify this flag, the Metaspace will dynamically re-size depending of the application demand at runtime.
Metaspace garbage collection
  • Garbage collection of the dead classes and classloaders is triggered once the class metadata usage reaches the “MaxMetaspaceSize”.
  • Proper monitoring & tuning of the Metaspace will obviously be required in order to limit the frequency or delay of such garbage collections. Excessive Metaspace garbage collections may be a symptom of classes, classloaders memory leak or inadequate sizing for your application.
Java heap space impact
  • Some miscellaneous data has been moved to the Java heap space. This means you may observe an increase of the Java heap space following a future JDK 8 upgrade.
Metaspace monitoring
  • Metaspace usage is available from the HotSpot 1.8 verbose GC log output.
  • Jstat & JVisualVM have not been updated at this point based on our testing with b75 and the old PermGen space references are still present.

How to set the Metaspace in Java 8

There are some new flags added for Metaspace in JDK 8:

  • -XX:MetaspaceSize=
where is the initial amount of space(the initial high-water-mark) allocated for class metadata (in bytes) that may induce a garbage collection to unload classes. The amount is approximate. After the high-water-mark is first reached, the next high-water-mark is managed by the garbage collector
  • -XX:MaxMetaspaceSize=
where is the maximum amount of space to be allocated for class metadata (in bytes). This flag can be used to limit the amount of space allocated for class metadata. This value is approximate. By default there is no limit set.
  • -XX:MinMetaspaceFreeRatio=
where is the minimum percentage of class metadata capacity free after a GC to avoid an increase in the amount of space (high-water-mark) allocated for class metadata that will induce a garbage collection.
  • -XX:MaxMetaspaceFreeRatio=
where is the maximum percentage of class metadata capacity free after a GC to avoid a reduction in the amount of space (high-water-mark) allocated for class metadata that will induce a garbage collection.

Sunday, July 2, 2017

permgen vs metaspace in java 8

As you may be aware in Java 8, one of its feature is removal of Permanent Generation (PermGen).
In this post, we will see how and for what the PermGen was being used till java 7 and its successor in java 8

A Java memory problem such as java.lang.OutOfMemoryError: PermGen space is one of the most frequent and complex problems a Java EE application support person can face with a production system. This article is one of the post that will focus on a particular OOM flavour: PermGen space depletion of a Java HotSpot VM.

Parameter used: -XXPermSize=1024m
before we see more details about it lets see a memory model in java:
 

Primary objective is to revisit the fundamentals of the permanent generation space and to teach you how to identify a particular pattern of PermGen space problem and possible causes of PermGen memory leak.
 
java.lang.OutOfMemoryError: PermGen space patterns
Find below some of the most common patterns of OutOfMemoryError due to the depletion of the PermGen space.
Pattern
Symptoms
Possible root cause scenarios
Resolution
OOM observed during or after a migration of a Java EE server to newer version
- OOM may be observed on server start-up at deployment time
- OOM may be observed very shortly after server start-up and after 1 or 2+ hours of production traffic
- Higher PermGen capacity is often required due to increased Java EE server vendor code and libraries
- Increase your PermGen space capacity via
-XX:MaxPermSize
OOM observed after a certain period of time
- OOM observed after a longer but consistent period of time (days)
- PermGen space monitoring will show hourly or daily increase during your application business hours
- There are many possible causes of PermGen space memory leak. The most common is a class loader leak: increasing number of Class objects overtime
- Improper JVM arguments like usage of the Xnoclassgc flag (turn OFF Class garbage collection)
- Review your JVM HotSpot start-up arguments for any obvious problem like Xnoclassgc flag
- Analyse the JVM HotSpot Heap Dump as it can provides some hints on the source of a class loader leak
- Investigate any third party API you are using for any potential class loader leak defect
- Investigate your application code for any improper use of Reflection API and / or dynamic class loading
OOM observed following a redeploy of your application code (EAR, WAR files...)
- OOM may be observed during or shortly after your application redeploy process
- Unloading and reloading of your application code can lead to PermGen leak (class loader leak) and deplete your PermGen space fairly quickly
- Open a ticket with your Java EE vendor for any known class loader leak issue
- Shutdown and restart your server (JVM) post deployment to cleanup any class loader leak
  
 
 
In next part we will see more details about metaspace introduced in java 8 as a successor of PermGen.

Logging levels in Log4j

As promised in previous blog, I have collected some information about log levels and its configurations.

There are mainly eight log level's available in log4j.
ALL, TRACE, DEBUG , INFO, WARN, ERROR, FATAL, OFF
  TRACE is used for finer-grained informational events than the DEBUG.

We can define more than log levels in our applications but one per appender. Following is a simple example of how it look like:


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
import org.apache.log4j.*;

public class LogClass {
   private static org.apache.log4j.Logger log = Logger.getLogger(LogClass.class);
   
   public static void main(String[] args) {
      log.setLevel(Level.WARN);

      log.trace("Trace Message!");
      log.debug("Debug Message!");
      log.info("Info Message!");
      log.warn("Warn Message!");
      log.error("Error Message!");
      log.fatal("Fatal Message!");
   }
}



# Define the root logger with appender file
log = /usr/home/log4j
log4j.rootLogger = WARN, FILE

# Define the file appender
log4j.appender.FILE=org.apache.log4j.FileAppender
log4j.appender.FILE.File=${log}/log.out

# Define the layout for file appender
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.FILE.layout.conversionPattern=%m%n

Now if we compile and run the above program and we will get following result in /usr/home/log4j/log.out file.


Warn Message!
Error Message!
Fatal Message!

 If someone needs more information do add a comment, I will try to provide that information as well.


Happy reading and keep adding comments. cheers..!

Saturday, June 17, 2017

Logging with Mapped Diagnostic Context(MDC)

I was trying to write something on this topic from very long time since this is less known but very useful technology in case of high performance, multi threaded distributed systems.

What is MDC or NDC or ThreadContext(log4j 2)?

We all know that log4j is a widely accepted and most powerful logging mechanism in today's world. No application can be written without log4j. But its always messy since it works on the principle of log levels i.e. ALL< DEBUG < INFO < WARN < ERROR < FATAL < OFF. We will see how log level works in a different  post, Here

It generates so many messages in the log files depending on the log level that finding the real cause becomes very tedious.

MDC is a concept or feature of Log4j logging library which can be used to group related log messages together

Log4j also provides a similar utility called as NDC, known as Nested Diagnostic Context

Both of the above can be replaced with ThreadContext class in log4j 2.

The ThreadContext class provides a Map and a Set to replace MDC and NDC. The Thread Context Map allows any number of items to be added and be identified using key/value pairs, while ThreadContextStack allows one or more items to be pushed on the Stack and then be identified by their order in the Stack or by the data itself. 


Remember MDC is managed on a per thread basis and every child thread automatically inherits a copy of mapped diagonstice context from its parent. This is achieved by InheritableThreadLocal  i which is a sub class of  ThreadLocal class.

How to use MDC to log information in log4j

 You can put any information into Mapped Diagnostic Context or MDC by calling the put() method. MDC is a static class i.e. a class with just static methods, which means you can directly call them without creating any instance of MDC.

Remember, this information is stored as thread local variable, which may cause a memory leak in a web application if used incorrectly (
see). If you are using MDC to log client or order specific information e.g. orderId in a web application using a filter than you can also remove it once done.




try{
  MDC.put("tradeId", trade.getId());
}finally{
  MDC.remove.remove("tradeId");
}

By the way from log4j2 onwards, MDC and NDC (Nested Diagnostic Context) are merged into a class called ThreadContext So, instead of using MDC, you need to use ThreadContext class to put unique context data.

try{
   ThreadContext.put("tradeId", trade.getId());
}finally {
   ThreadContext.clear();
}


Once available in MDC, this information can be printed in the log file using PatternLayout. You can use %X{tradeId) to print tradeId in log messages, remember tradeId for different trades will be different, which allows you to trace logging messages for individual trades. 

This is same as we print the logs in log4j. We will see pattern layout in details here in details.


I will share a complete example in my coming blogs. Stay connected and happy reading :)


Wednesday, April 19, 2017

From LocalDate to LocalDateTime

How to convert LocalDate to LocalDateTime?

First we need to understand what we have and what we want to get.

From LocalDate we have: Instant + System default time zone.
What we want to have is: Instant + System default time zone + Local Time

LocalDateTime.of() methods gives us so many options to create LocalDateTime. On of the option is as given below

LocalDate localDate = LocalDate.now()

LocalDateTime ldt= LocalDateTime.of(localDate, LocalTime.MIN);

Here LocalTime.Min will give us the Day's start time i.e. 00:00:00
To get LocalDateTime with current time we can use following:

LocalDateTime localDateTimeNow = LocalDateTime.of(localDate, LocalTime.now());

Example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
import java.time.Instant;
import java.time.LocalDate;
import java.time.LocalDateTime;
import java.time.LocalTime;
import java.time.ZoneId;
import java.util.Date;

public class LocalDateToLocalDateTime {

 public static void main(String[] args) {

  // Asia/Kuala_Lumpur +8
  ZoneId defaultZoneId = ZoneId.systemDefault();
  System.out.println("System Default TimeZone : " + defaultZoneId);

  Date date = new Date();
  System.out.println("date : " + date);

  // 1. Convert Date -&gt; Instant
  Instant instant = date.toInstant();
  System.out.println("instant : " + instant); // Zone : UTC+0

  // 2. Instant + system default time zone + toLocalDate() = LocalDate
  LocalDate localDate = instant.atZone(defaultZoneId).toLocalDate();
  System.out.println("localDate : " + localDate);

  // 3. Instant + system default time zone + toLocalDateTime() =
  // LocalDateTime
  LocalDateTime localDateTime = instant.atZone(defaultZoneId).toLocalDateTime();
  System.out.println("localDateTime : " + localDateTime);

  LocalDateTime ldt = LocalDateTime.of(localDate, LocalTime.now());
  System.out.println("localDateTime : " + ldt);
 }
}


Friday, November 18, 2016

Using Lambda Expressions to sort a collection in Java 8

Below is an example to show how to use Lambda expression to sort a collection.

Prior to Java 8, it was necessary to implement the java.util.Comparator interface with an anonymous (or named) class when sorting a collection:
Java SE 1.2
Collections.sort(
    personList,
    new Comparator<Person>() {
        public int compare(Person p1, Person p2){
            return p1.getFirstName().compareTo(p2.getFirstName());
        }
    }
);
Starting with Java 8, the anonymous class can be replaced with a lambda expression. Note that the types for the parameters p1 and p2 can be left out, as the compiler will infer them automatically:
Collections.sort(
    personList, 
    (p1, p2) -> p1.getFirstName().compareTo(p2.getFirstName())
);
The example can be simplified by using Comparator.comparing and method references expressed using the :: (double colon) symbol.
Collections.sort(
    personList,
    Comparator.comparing(Person::getFirstName)
);
A static import allows us to express this more concisely, but it is debatable whether this improves overall readability:
import static java.util.Collections.sort;
import static java.util.Comparator.comparing;
//...
sort(personList, comparing(Person::getFirstName));
Comparators built this way can also be chained together. For example, after comparing people by their first name, if there are people with the same first name, the thenComparing method with also compare by last name:
sort(personList, comparing(Person::getFirstName).thenComparing(Person::getLastName));

static vs volatile in java

  A very common interview question asked from mid level experience java developer and the question is: What is the difference between stat...