Friday, September 19, 2014

Java Memory leak tips

Last couple of days I had to track an Out of mermory error on a production application. During this tracking I searched and use different tools for differents problems.
So this post is a reminder.

the error context environment is : a java application launch by maven (use exec:java ; benefits from transitive dependencies) launch from eclipse configuration for external tool (this tool is an eclipse one, benefits from log redirection through console)

Use java Debug Architecture

in this context the debugging could be done with java debug architecture :
So for JVM (TI) side : launch maven exec command with JVM args :
-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=9009
(activate debug, load jdwp, transport mode = socket, target application listen for a debugger to attach, target vm will be suspendended until debugger apps connection)

Or the new one (not use in my case)
-agentlib:jdwp=transport=dt_socket,address=localhost:9009,server=y,suspend=y

For JDI side one can use eclipse with "Remote Java Application"

Run the VM then the debugger.

Use java -verbose

this java option could takes args :
-verbose [class | gc | jni]

In our case the gc arg is usefull, it allows log of Garbage Collection event.
[GC 4416K->1055K(15872K), 0.0029002 secs]
[GC 5471K->1640K(15872K), 0.0030503 secs]
[GC 6056K->2100K(15872K), 0.0034942 secs]
[GC 6516K->2633K(15872K), 0.0027937 secs]
[GC 7049K->2846K(15872K), 0.0028010 secs]

You could also use to have more details or output to file:
-XX:+PrintGCTimeStamps
-XX:+PrintGCDetails
-Xloggc:<file>

You could then have :
 0.319: [GC 0.319: [DefNew: 4927K->512K(4928K), 0.0024456 secs]
5922K->1924K(15872K), 0.0024653 secs] [Times: user=0.02 sys=0.00, real=0.00 secs] 
 0.398: [GC 0.398: [DefNew: 4928K->512K(4928K), 0.0021861 secs] 
6340K->2307K(15872K), 0.0022057 secs] [Times: user=0.01 sys=0.00, real=0.00 secs] 
 0.465: [GC 0.465: [DefNew: 4928K->512K(4928K), 0.0025962 secs] 
6723K->2509K(15872K), 0.0026180 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
 0.545: [GC 0.545: [DefNew: 4928K->511K(4928K), 0.0022087 secs]
6925K->3017K(15872K), 0.0022284 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
 0.620: [GC 0.620: [DefNew: 4927K->511K(4928K), 0.0019218 secs] 
7433K->3584K(15872K), 0.0019427 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
 0.684: [GC 0.684: [DefNew: 4927K->395K(4928K), 0.0021938 secs] 
8000K->3580K(15872K), 0.0022122 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
Heap
 def new generation   total 4928K, used 3683K [0x24c40000, 0x25190000, 0x2a190000)
  eden space 4416K,  74% used [0x24c40000, 0x24f760d0, 0x25090000)
  from space 512K,  77% used [0x25090000, 0x250f2d70, 0x25110000)
  to   space 512K,   0% used [0x25110000, 0x25110000, 0x25190000)
 tenured generation   total 10944K, used 3185K [0x2a190000, 0x2ac40000, 0x34c40000)
   the space 10944K,  29% used [0x2a190000, 0x2a4ac688, 0x2a4ac800, 0x2ac40000)
 compacting perm gen  total 12288K, used 4456K [0x34c40000, 0x35840000, 0x38c40000)
   the space 12288K,  36% used [0x34c40000, 0x3509a390, 0x3509a400, 0x35840000)
    ro space 10240K,  45% used [0x38c40000, 0x390c7988, 0x390c7a00, 0x39640000)
    rw space 12288K,  54% used [0x39640000, 0x39ccb5d8, 0x39ccb600, 0x3a240000)

Use the dump

The get a rapid diagnosis, you could use a JVM option that allow to output a dump when Out of memory error

-XX:+HeapDumpOnOutOfMemoryError
Or use with ctrl break
-XX:+HeapDumpOnCtrlBreak

You can then use the produce dump (hprof file) in Visual VM (See below).

Use Visual VM

This tool located in ${JAVA_INSTALL_DIR}/bin/jvisualvm.exe (on windows), allow you this view many things on the VM (mainly):
  • see threads, memory... behavior 
  • trigger dump 


Object size Measure (Reflexion)

Using this jar (as maven deps)
<dependency>
<groupid>net.sf.ehcache</groupid>
<artifactid>ehcache</artifactid>
<version>2.8.3</version>
</dependency>

You could now use a method relying on reflexion for object size calculation :

try {
    ReflectionSizeOf reflectionSizeOf = new ReflectionSizeOf();
    Size deepSizeOf = reflectionSizeOf.deepSizeOf(1000, false, model);
    logger.info("Memory in use Model deepSize | F3 |  {}", deepSizeOf.getCalculated());
    deepSizeOf = reflectionSizeOf.deepSizeOf(1000, false, context);
    logger.info("Memory in use Context deepSize | F4 |  {}", deepSizeOf.getCalculated());
} catch (Exception e) {
    e.printStackTrace();
}

Object size Measure (Using Java Agent)

Agent could instrument java through JVM addition (transformation) of byte-codes to method.
An instrument is a class. This class should provides methods :
public static void premain(String args, Instrumentation inst) throws Exception
public static void agentmain(String args, Instrumentation inst) throws Exception

declared in META-INF/MANIFEST.MF
Agent-Class: my.package.JavaAgent
Can-Redefine-Classes: true
Can-Retransform-Classes: true
Premain-Class: my.package.JavaAgent
There is two ways that provides instrument access
  • launch JVM with instrumentation flag (instrument premain method is used)
  • use instrumentation dynamically (agentmain method use), implies tools.jar usage
An instrument provide access to a method :
  long     getObjectSize(Object objectToSize)

WARNING this method provide only object size without object instance references

If you encounter problem while loading instrument you could use this one
org.apache.openjpa.enhance.InstrumentationFactory

Comming from
<dependency>
 <groupId>org.apache.openjpa</groupId>
 <artifactId>openjpa-kernel</artifactId>
 <version>2.3.0</version>
</dependency>

Much more usefull information about agent here

Use TPTP

See here

Java memory map

To finish this post, I tried to build a graphic memory figure to remind me how it is approximately organized


Friday, June 27, 2014

Eclipse contribute to "Show view" menu for an existing perspective

How to contribute to "Show view" menu for an existing perspective

In this post I show how to add a shortcut to your own perspective to Window -> Show View menu in a particular perspective.

In fact you just have to define in your plugin.xml (see  Eclipse Doc. PerspectiveExtensions)
  • a perspectiveExtensions point
  • specify the perspective tagrget id
  • add a shortcut to your view.
It gives for a shortcut in Java perspective :

<extension point="org.eclipse.ui.perspectiveExtensions">
    <perspectiveExtension targetID = "org.eclipse.jdt.ui.JavaPerspective">
        <viewShortcut id="my.view.id"/>
    </perspectiveExtension>
</extension>

If you want to do this in your own perspective you can use this too or define in the createInitialLayout method of your perspective (implementing IPerspectiveFactory, see Eclipse doc IperspectiveFactory)

 public void createInitialLayout(IPageLayout pageLayout)
  { 
  pageLayout.addShowViewShortcut("my.view.id");
  }




Tuesday, June 3, 2014

Playing with kineticjs



Explanations

This post present a small Kineticjs experiment. And a JS integration in Google blog experiment. The small js above allow:
  • to zoom in and out the rectangle
  • to move the rectangle by selecting it

Encountered problems to be investigate and Todos (may be...)

  • scale function does not work
  • usage of JQuery $() function (to be removed)
  • refactor duplicated code
  • Use kineticJs drag start event

Ref

Notes using CDN here

  • The url for kineticjs could be found at https://github.com/ericdrowell/KineticJS/wiki/Version-Archive. Page extract : All of the versions below are hosted on the Amazon CloudFront CDN (Content Delivery Network), which means that you can hotlink to the URLs and enjoy fast response times no matter where you are in the world.
  • for JQuery the url could be found at http://jquery.com/download/

Tuesday, April 29, 2014

Usefull Maven compilation command line

Usefull Maven compilation command line Reminder


mvn install -pl project.name.artefact -am -T 2.0C

Explanation 

from (http://florianlr.wordpress.com/2012/09/19/17/)
-pl, --projects
Comma-delimited list of specified reactor projects to build instead of all projects. A project can be specified by [groupId]:artifactId or by its relative path.

-am,--also-make
If project list is specified, also build projects required by the list.

-T,--threads
Thread count, for instance 2.0C where C is core multiplied

Wednesday, February 26, 2014

Adding a list editor in eclipse preference. Extend ListEditor

Adding a list editor in Eclipse preferences implies to extend ListEditor.
This  post concern the build of a ListEditor dealing with filter definition relying on regular expression.


The beginning,  create a FilterDialog

This dialog aims to get a filter entry. This dialog should return a String containing a regular expression. (I don't verify that the content typed is OK but I will)


the FilterDialog source code :

import org.eclipse.jface.dialogs.TitleAreaDialog;
import org.eclipse.swt.SWT;
import org.eclipse.swt.layout.GridData;
import org.eclipse.swt.layout.GridLayout;
import org.eclipse.swt.widgets.Composite;
import org.eclipse.swt.widgets.Control;
import org.eclipse.swt.widgets.Label;
import org.eclipse.swt.widgets.Shell;
import org.eclipse.swt.widgets.Text;

public class FilterDialog extends TitleAreaDialog {

  private Text filterValue;

  private String filter;

  public FilterDialog(Shell parentShell) {
    super(parentShell);
  }

  @Override
  public void create() {
    super.create();
    setTitle("Add filter to keep tests");
  }

  @Override
  protected Control createDialogArea(Composite parent) {
    Composite area = (Composite) super.createDialogArea(parent);
    Composite container = new Composite(area, SWT.NONE);
    container.setLayoutData(new GridData(GridData.FILL_BOTH));
    GridLayout layout = new GridLayout(2, false);
    container.setLayoutData(new GridData(SWT.FILL, SWT.FILL, true, true));
    container.setLayout(layout);
    createFilter(container);
    return area;
  }

  private void createFilter(Composite container) {
    Label lbtFirstName = new Label(container, SWT.NONE);
    lbtFirstName.setText("Filter as regular expression");

    GridData dataFirstName = new GridData();
    dataFirstName.grabExcessHorizontalSpace = true;
    dataFirstName.horizontalAlignment = GridData.FILL;

    filterValue = new Text(container, SWT.BORDER);
    filterValue.setLayoutData(dataFirstName);
  }

  @Override
  protected boolean isResizable() {
    return true;
  }
 
  private void saveInput() {
    filter = filterValue.getText();
  }

  @Override
  protected void okPressed() {
    saveInput();
    super.okPressed();
  }

  public String getFilter() {
    return filter;
  }
 
} 

Extend ListEditor


the source code :
import java.io.File;

import org.eclipse.jface.preference.ListEditor;
import org.eclipse.jface.window.Window;
import org.eclipse.swt.widgets.Composite;


public class FilterTestEditor extends ListEditor {

    protected FilterTestEditor() {
    }

    public FilterTestEditor(String name, String labelText, Composite parent) {
        init(name, labelText);
        createControl(parent);
    }

    protected String createList(String[] items) {
        StringBuffer path = new StringBuffer("");//$NON-NLS-1$
        for (int i = 0; i < items.length; i++) {
            path.append(items[i]);
            path.append(File.pathSeparator);
        }
        return path.toString();
    }


    protected String getNewInputObject() {
     String filter=null;
        FilterDialog dialog = new FilterDialog(getShell());
        dialog.create();
       
        if (dialog.open() == Window.OK) {
           System.out.println(dialog.getFilter());
           filter = dialog.getFilter();
         } 
        return filter;
    }
    
    protected String[] parseString(String stringList) {
  StringTokenizer sto = new StringTokenizer(stringList, File.pathSeparator
                + "\n\r");
        ArrayList<String> v = new ArrayList<String>();
        while (st.hasMoreElements()) {
            v.add((String) sto.nextElement());
        }
        return (String[]) v.toArray(new String[v.size()]);
    }
}

and finally in the préférence page add

addField(new FilterTestEditor(KEY_FILTER_TEST, "Test filtering", getFieldEditorParent()));

Running external tools programmatically with eclipse

Small tutorial for running external tools programmatically with eclipse

Last couple of day I had to refactor code for an eclipse plugin.
The source code to modify rely on java runtime exec to launch a temporary maven project. This code works fine but the process launched does not log in eclipse console.
So at this point I look for a way to launch this maven build, external to the workspace as the eclipse menu "launch external tool does". After a few search on the web I found a way to be able to do this:
  • create a launch configuration
  • save it in a file
  • build the same setup in source code
  • and finally launch the configuration

First thing to do : create an external tool configuration...
saved in a file (see Shared file field)

Run this configuration.

You should find a file in the root folder project named : New_configuration.launch.
Now when editing this file I have all the parameters needed to write my code :
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<launchConfiguration type="org.eclipse.ui.externaltools.ProgramLaunchConfigurationType">
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_LOCATION" 
 value="${system_path:mvn.bat}"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_TOOL_ARGUMENTS" 
 value="-Dlogback.configurationFile=customlogback.xml -U clean install exec:java"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_WORKING_DIRECTORY" 
 value="D:\DONNEES\FDS_DATA\genDoc"/>
</launchConfiguration>

With this content and this address I could now write the source code that allow me to launch programmatically my temporary maven project.
A nice point here, you could rely on eclipse variable (like ${system_path:mvn.bat})  
//to be able to listen the end
manager.addLaunchListener(this);
ILaunchConfigurationType type = manager
  .getLaunchConfigurationType("org.eclipse.ui.externaltools.ProgramLaunchConfigurationType");
ILaunchConfiguration[] configurations = manager
  .getLaunchConfigurations(type);
for (int i = 0; i < configurations.length; i++) {
 ILaunchConfiguration configuration = configurations[i];
 if (configuration.getName().equals("Gen doc")) {
  configuration.delete();
  break;
 }
}
ILaunchConfigurationWorkingCopy workingCopy = type.newInstance(
  null, "Gen doc");
String exec = "mvn.sh";
if (nameOS.toLowerCase().contains("win")) {
 exec = "mvn.bat";
}
workingCopy.setAttribute(
  "org.eclipse.ui.externaltools.ATTR_LOCATION",
  "${system_path:" + exec + "}");
workingCopy.setAttribute(
  "org.eclipse.ui.externaltools.ATTR_TOOL_ARGUMENTS",
  "-Dlogback.configurationFile=genDoclogback.xml -U clean install exec:java");
workingCopy.setAttribute(
  "org.eclipse.ui.externaltools.ATTR_WORKING_DIRECTORY",
  tmpGenDir.getCanonicalPath());
ILaunch launch = workingCopy.launch(ILaunchManager.RUN_MODE,
  new NullProgressMonitor());
//to retreive the launched project at the end
launchProject.put(launch, project);

and to be able to listen the end of your launch configuration implement "ILaunchesListener2" and add

@Override
public void launchesTerminated(ILaunch[] launches) {
 for (ILaunch launch : launches) {
  Project project = launchProject.get(launch);
  if (project != null) {
   launchProject.remove(launch);
   //if needed...
   Display.getDefault().asyncExec(new Runnable() {
    @Override
    public void run() {
     
    }
   });
  }
 }
}


Saturday, January 18, 2014

Raspbery-pi, Gertboard : building an oscilloscope


Playing with raspberry pi and gertboard (part one)

Last few years I follow the raspberry-pi initiative and I really much appreciate it. This year I decide to use this micro computer associated to a gertboard to acheive an old project : building a stand alone oscilloscope.

what I want to acheive :


Step one : where is my  hdmi screen ????

Using my Windows8 PC for raspberry export display

I don't want to use or buy a new HDMI display for this Raspberry pi experiment.
So I will use my own laptop. For the very first RP start, You need an HDMI to setup your RP.
  • Connect you RP  (Raspberry pi) 
  • With the setup screen activate SSH.
  • You could now power off RP and disconnect your HDMI screen.
  • Power on RP
  • Goes on your windows station and you had to install an XServer and ssh client  : you could use mobaxterm, cygwin or a Virtual box with a Linux ISO installed in it (network setup with bridge adapter).
  • With this client, open a terminal and connect your RP with ssh :
xhost +
ssh -X pi@ip_addr_of_remote_machine
# the password is rasberry

Use man xhost and man ssh for more information
You sould have something like that (with mobaxterm)


At this point you could access to your RP.

just use :
lxsession

After a few second a window should be open with the RP desktop.

How to access my ATmega on my gertboard?

Just have a look at https://projects.drogon.net/raspberry-pi/gertboard/arduino-ide-installation-isp/ this is the "Gordons Projects" blog wich is very well done. You will find out there all that you need for your gertboard.
I just need to add an apt-get update, and all works as expected.

 (to be continued)