Wednesday, March 21, 2012

CSV to Chart Utility

In my last project we had a requirement where we need to build the good report from the csv data. One way to do that is to use excel and create chart from excel. 
But to combine all chart in single report(If there are multiple charts), do data processing if any (We can use micro if person is good in excel) is tedious job.  I am attaching the mevenize project with some screenshot which will help you to understand the Utility.

This tool is helpful to create good charts/reports from CSV. You can pass the set of CSV, tool will do the data processing(data conversion)
if defined and create a pdf/xls/doc based charts on user choice. Tool will create a chart and tabular report as configuration and 
render that in a page. For all valid CSV’s, it will create a chart report in a new page.

Underneath it is using Dynamic Reporting for charting/reporting and Spring DI for report configuration. All report configuration specified below can be configured using XML file. 
1.       Name of PDF (refer attachment Report Config.png)
2.       Report header (refer attachment ReportConfig.png)
3.       Report footer
4.       Chart name (refer attachment Component-Config.png)
5.       Chart X and Y axis (refer attachment Component-Config.png)
6.       Chart Label (refer attachment Component-Config.png)
7.       Chart Type (refer attachment Component-Config.png) - Currently It support Bar, Line,XYLine chart.
8.       Blank page (refer attachment BlankPageConfig.png)
9.       Data Processor – Data processor is used to convert Raw data into some specified format which should be shown on report like change milliseconds to second, hours to min, Byte to MegaByte, etc (refer attachment DataProcessor.png)

Some configuration for Report Component
Report Configuration
       <bean id="performanceReport" class="com.sapient.dynamicreports.configuration.Report">
              <property name="name" value="PerformanceReports" /> - output report file name
              <property name="displayName" value=" Performance Test Report  " /> - output report header
              <property name="outputType" value="PDF" /> - output report type
              <property name="outputPath" value="c:\\output" /> - output path
              <property name="components">
                     <list>
              List of components ---
                           <!-- <ref bean="blankPage" /> -->
                           <ref bean="cpuUtilization1" />
                           <ref bean="memoryUtilization1" />
                           <ref bean="threadUtilization1" />

                           <ref bean="cpuUtilization2" />
                           <ref bean="memoryUtilization2" />
                           <ref bean="threadUtilization2" />

                           <ref bean="cpuUtilization3" />
                           <ref bean="memoryUtilization3" />
                           <ref bean="threadUtilization3" />

                           <ref bean="responseTimePerTransaction" />

                     </list>
              </property>

 Component Configuration

       <bean id="activeThreadOverTime" class="com.sapient.dynamicreports.configuration.Component">
              <property name="name" value="ActiveThreadOverTime" />
-    Name of the component
              <property name="displayName" value="Active Users Over Time" />
-    Display name , which will come in the output report
              <property name="dataSource" value="CSV" />
-    Data source type – in our case it will be CSV
              <property name="dataSourcePath"
                     value="c:\\csvfileset\\input\\ActiveThreadsOverTime.csv" />
-    Data source – CSV file name
              <property name="dataSourceFileName" value="ActiveThreadsOverTime.csv" />
              <property name="chartConfiguration" ref="atotChartConfiguration" />
-    Define the chart Configuration
              <property name="pageType" value="A4" />
-    Page type of chart
              <property name="pageOrientation" value="PORTRAIT" />
-    Page orientation of chart
              <property name="dataProcessor" ref="customDataProcessor" />
-    Data processor if any
              <property name="tableType" value="NO" />
-    Table type could be NO, AGGREGATED, SIMPLE
       </bean>


       </bean>





Some of the screenshot (ReportPage1.png, ReportPage2.png) will give an idea how the report will look like. also fine high level class diagram for code in Image1 and Image2.  You can download code from here

Monday, March 12, 2012

JQuant - Library for Financial/Mathematical Projects.


While Surfing over the internet comes accross the utility called JQuant,
 Another thought strikes in my mind is 'lets share it with everone using Blog'.
Its a very useful utility for Finanacial Project and Supports with lots of mathematical function. Basically JQuant is derived from the QuantLib  which is written in C++.


JQuantLib provides a free, open-source and comprehensive framework for quantitative finance. 
It's a 100% Java translation of QuantLib, which is written in C++. JQuantLib provides pricing valuation of a wide range of asset classes,
methods and models.


The Basic feature which JQuant supports is 


Features

  •  Support to a wide range of financial instruments, including but not limited to European Options, American Options, Bermudan Options, Asian Options, Bonds, Swaps, FRA, Repo, Cap/Floors, etc
  •  Several pricing engines: Black-Scholes, Barone-Adesi-Whaley, Bjerksund-Stensland, Ju Quadratic, Integral, Binomial Cox-Ross-Rubinstein, Binomial Jarrow-Rudd, Additive EquiProbabilities, Binomial Trigeorgis, Binomial Tian, Binomial Leisen-Reimer, Binomial Joshi, Finite Differences
  • Implemented as Java Library with minimum external dependencies
  • Coded with performance in mind. GC imposes minimalist performance penalty.



Capability of QuantLib
The Wiki Page gives the complete descrition of QuantLib and what all can bedone with the supported library.

I will keep you posted on other tool/utility which can help us in our day to day projects.

Monday, March 5, 2012

Application Monitoring

For few days i was looking to develop something which can monitor tomcat and tomcat activities.Monitor the tomcat application at client side is something can help us. Using Jconsole/jvisualvm using JMX/Mbean is one way to monitor the application but that has limited feature with limited UI. Something which is rich in UI and easy to use can really helpful.

I have checked and found few thing 


This link tell us different way to monitor the web application. What i like here is  Psi-probe.


Its simple to use to start with. Just put the probe.war in tomcat directory and configured the tomcat user in tomcat-users.xml and you are ready to go. 

Unlike many other server monitoring tools, PSI Probe does not require any changes to your existing apps. It provides all of its features through a web-accessible interface that becomes available simply by deploying it to your server. These features include:
  • Requests: Monitor traffic in real-time, even on a per-application basis.
  • Sessions: Browse/search attributes, view last IP, expire, estimate size.
  • JSP: Browse, view source, compile.
  • Data Sources: View pool usage, execute queries.
  • Logs: View contents, download, change levels at runtime.
  • Threads: View execution stack, kill.
  • Connectors: Status, usage charts.
  • Cluster: Status, usage charts.
  • JVM: Memory usage charts, advise GC
  • Java Service Wrapper: Restart JVM.
  • System: CPU usage, memory usage, swap file usage. 

 I will keep posted to other application which can help in monitor.





Tuesday, July 26, 2011

Example Java

Here is the good links to understand and learn java in good way

http://www.exampledepot.com/egs/index.html

http://www.java2s.com

Tuesday, January 25, 2011

CSV Data Loader

Here I am sharing quick CSV Data Loader using scanner.
import java.io.File;
import java.io.FileNotFoundException;
import java.util.Scanner;

public class DataLoader {

public void laodData(String fileName) {
Scanner scanner = null;
try {
scanner = new Scanner(new File(fileName));
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

while (scanner.hasNextLine()) {
String line = scanner.nextLine();
System.out.println("CSV Line : " + line);
Scanner scanner2 = new Scanner(line);
scanner2.useDelimiter(", *");

// Read and sum numbers.
while (scanner2.hasNext()) {
System.out.println(scanner2.next());
}
System.out.println("________________ LINE END ____________________");

}
}

public static void main(String[] args) {
new DataLoader().laodData("c:\\test3.csv");
}
}


Thursday, November 4, 2010

Apache Netty

While working on Server Client Communication Framework i came accross the Apache Netty which is pretty much the same I am plannign to develop.
Here is basic example of using these API provided by apache
http://bruno.factor45.org/blag/2010/07/15/handshaking-tutorial-with-netty/

We also need to see Apache MINA.

Here is a pretty good comparision between these 2

http://stackoverflow.com/questions/1637752/netty-vs-apache-mina


I am also exploring these 2 and will post some more details soon.

Monday, October 4, 2010

Thread Helper

Thread Helper will help you to call your class method in threaded environment. It will take care all the low level complexity and provide you a simple interface to call a method in threaded environment.
To use this one should extend the class called ThreadCommand and implement the execute method of that. Whatever code will be written in execute method will be called in threaded manner.

ThreadHelper has 2 methods in it
1. Call
2. Get

Call will notify the thread helper to call the execute method of Command implementation and store the result in a map and when get method will be called, result will returned to the user and it will be removed from the map.


After call method, user can do other stuff and whenever user required the output from thread just get that from get Method. This will simulate the post evaluation mechanism. In this user first call and continue with other operation and whenever required the output just call the get method.

ThreadHelper.java


import java.util.HashMap;

import java.util.Map;

import java.util.concurrent.ExecutionException;

import java.util.concurrent.ExecutorService;

import java.util.concurrent.Executors;

import java.util.concurrent.Future;



public class ThreadHelper

{

private boolean callWhenFetch = false;

private ExecutorService executorService = null;

private Map> futurs = null;

private boolean started = false;



/**

*

* @param callWhenFetch

*/

public ThreadHelper(boolean callWhenFetch)

{

this.callWhenFetch = callWhenFetch;

}



/**

*

*/

public ThreadHelper()

{

}



public void start()

{

executorService = Executors.newCachedThreadPool();

futurs = new HashMap>();

started = true;

}



/**

* Check if the thread Helper is Started

*

* @return

*/

public boolean isStart()

{

return started;

}



/**

* Initiates an orderly shutdown in which previously submitted commands are

* executed, but no new commands will be accepted.

*/

public void stop()

{

if (futurs != null)

{

futurs = null;

}



if (executorService != null)

{

executorService.shutdown();

}

}



/**

* Attempts to stop all actively executing commands, halts the processing of

* waiting commands

*/

public void stopNow()

{

if (futurs != null)

{

futurs = null;

}



if (executorService != null)

{

executorService.shutdownNow();

}

}



/**

* This method takes ICommand as input and submit to ThreadPool.

*

* @param command

* @throws Exception

*/

public synchronized void call(ThreadCommand command) throws Exception

{

checkIfStarted();

Future submitted = executorService.submit(command);

futurs.put(command.getId(), submitted);

}



/**

* This method give the output to previously called command. If

* callWhenFetch parameter is set to true then If you haven't call call

* method before get then it will first call call method and then call get

* of that call. If flag is set to false and you tried to call get before

* call method it will throws exception

*

* @param command

* @return

* @throws NotNotifiedException

* @throws InterruptedException

* @throws ExecutionException

*/

public T get(ThreadCommand command)

throws NotNotifiedException,

InterruptedException,

ExecutionException,

Exception

{

checkIfStarted();

Future submitted = null;

try

{

synchronized (futurs)

{



if (!futurs.containsKey(command.getId()))

{

if (!callWhenFetch)

{

throw new NotNotifiedException(

"Object is not called before get.");

}

submitted = executorService.submit(command);

futurs.put(command.getId(), submitted);

}

T object = futurs.get(command.getId()).get();

submitted = futurs.remove(command.getId());



return object;

}

}

finally

{

if (submitted != null)

{

submitted.cancel(false);

submitted = null;

}

}

}



private void checkIfStarted() throws Exception

{

if (!isStart())

{

throw new Exception(

"Please start the thread helper before using it.");

}

}



public int waitingCallCount()

{

return futurs.size();



}

}







ThreadCommand.java



import java.util.concurrent.Callable;



public abstract class ThreadCommand implements Callable

{



private String id;



public ThreadCommand(String id)

{

this.id = id;

}



public T call() throws Exception

{

return execute();

}



public String getId() {

return id;

}



public abstract T execute();



@Override

public String toString()

{

return "ThreadCommand [id=" + id + "]";

}





}




ThreadHelperTest.java



import java.util.ArrayList;

import java.util.List;



import junit.framework.TestCase;






public class ThreadHelperTest extends TestCase

{



public void testSingleThreadImmediateResponseBefoereStart()

{

try

{

ThreadCommand command = new ThreadCommand("Test")

{



@Override

public String execute()

{

// TODO Auto-generated method stub

return "I am just checking how execute is working.";

}

};



ThreadHelper helper = new ThreadHelper();

helper.call(command);

helper.get(command);

fail();

}

catch (Exception e)

{

assertEquals("Please start the thread helper before using it.", e

.getMessage());



}

}



public void testGetBeforeCallWhenCallWhenFetchIsFalse()

{

try

{

ThreadCommand command = new ThreadCommand("Test")

{



@Override

public String execute()

{

// TODO Auto-generated method stub

return "I am just checking how execute is working.";

}

};



ThreadHelper helper = new ThreadHelper(true);

//helper.call(command);

helper.get(command);

fail();

}

catch (Exception e)

{

e.printStackTrace();

assertEquals("Please start the thread helper before using it.", e

.getMessage());



}

}



public void testSingleThreadImmediateResponse()

{

try

{

ThreadCommand command = new ThreadCommand("Test")

{

public String execute()

{

return "I am just checking how execute is working.";

}

};



ThreadHelper helper = new ThreadHelper();

helper.start();

helper.call(command);

String str = helper.get(command);

assertEquals("I am just checking how execute is working.", str);



}

catch (Exception e)

{

fail();



}

}



public void testMutipleThreadImmediateResponse()

{

try

{

ThreadCommand command1 = new ThreadCommand("Test1")

{

public String execute()

{

return "[Test 1]I am just checking how execute is working.";

}

};



ThreadCommand command2 = new ThreadCommand("Test2")

{

public String execute()

{

return "[Test 2]I am just checking how execute is working.";

}

};



ThreadHelper helper = new ThreadHelper();

helper.start();

helper.call(command1);

helper.call(command2);

String str = helper.get(command2);

assertEquals(

"[Test 2]I am just checking how execute is working.",

str);

str = helper.get(command1);

assertEquals(

"[Test 1]I am just checking how execute is working.",

str);



}

catch (Exception e)

{

fail();



}

}



public void testMutipleThreadImmediateResponseForSameCommand()

{

try

{

ThreadCommand command1 = new ThreadCommand("Test1")

{

public String execute()

{

return "[Test 1]I am just checking how execute is working.";

}

};



ThreadHelper helper = new ThreadHelper();

helper.start();

helper.call(command1);

String str = helper.get(command1);

assertEquals(

"[Test 1]I am just checking how execute is working.",

str);

str = helper.get(command1);

fail();



}

catch (Exception e)

{

assertEquals("Object is not called before get.", e.getMessage());



}

}



public void testThreadHelperStatusAfterAllGet()

{

try

{

List> commands = new ArrayList>();

for (int i = 0; i < k =" i;"> command1 = new ThreadCommand(

"Test" + k)

{

public String execute()

{

return "[Test "

+ k

+ "]I am just checking how execute is working.";

}

};

commands.add(command1);

}



ThreadHelper helper = new ThreadHelper();

helper.start();

for (ThreadCommand command : commands)

{

helper.call(command);

}



assertEquals(100, helper.waitingCallCount());



// get all odd number command

for (int i = 1; i < i =" i" i =" 0;"> command : commands)

{

if(i == 100) {

break;

}

assertEquals("[Test "

+ i

+ "]I am just checking how execute is working.", helper

.get(commands.get(i)));

i = i + 2;



}

// get rest of the commands

assertEquals(0, helper.waitingCallCount());



}

catch (Exception e)

{

e.printStackTrace();

fail();



}

}

}