Wednesday, July 17, 2013

ORA-01000: maximum open cursors exceeded


Possible Fix :

1)Ensure Oracle 8.1.7 JDBC driver or a later version should be used
2)Java code can be checked for closure of ResultSet & Statement after the using this specific stored procedure
3) Increase the value of OPEN_CURSORS parameter in Oracle

Steps to fix:



1)Do static code check and all of the cursors/connections are closed properly in the java code
2) Ask DBA Team to check the value of OPEN_CURSORS parameter in Oracle and see whether the value is too less or not.
3)Further analysis is possible only if we are having details of all open cursors at the time of exception (DBA Team can provide the details of opened cursors)


EPS file does not open correctly in photoshop-("Could not open or parse a DSC plate file because the file cannot be found"):


I have an issue from Customer saying that .eps file is not opened properly in photoshop application and throws the error "Could not open or parse a DSC plate file because the file cannot be found".

1)Downloaded open source viewer application such as InfraView and installed.The .eps file opened propely but in blurred form, so that I fgiured out some information is missing to reconstruct the file.
2)Tried to open in photoshop and got the error.I converted psd file to eps file which worked fine.At that time I came across options 'multiple file version/single file version' before saving the psd file to eps.
3)read more about EPS file format in internet and clarified about 'multiple file version/single file version'
4)Opened the error .eps file in Notepad++ and checked and found out .C,.M,.Y and .K files are missing and that was originally created as multiple file version.


Resolution:

EPS file was created as multiple file version and some of the files are missed out( Logo.C,Logo.M, Logo.Y,Logo.K ).
create single file version of eps file and upload the image to Application

Unable to install the trial version of a product with IBM Installation Manager:

I  downloaded trial version of rational application developer (RAD) from IBM website and tried to install the product. Normally all of the IBM products will be installed via IBM installation manager. The configuration file packed inside installation file (exe) will have information to obtain IBM product repository, in out case this is http link https://www-912.ibm.com/software/repositorymanager/action/getInitData

I clicked on the installation file which extracted the files to temporary location and started screens to install the installation manager and received popup with below error

CRIMC1085E: Resumable download failed for: https://www.ibm.com/software/repositorymanager/action/getInitData.    CRIMC1083E: Failed to retrieve 'https://www.ibm.com/software/repositorymanager/action/getInitData'.      CRIMC1084E: An exception is thrown while retrieving the file 'https://www.ibm.com/software/repositorymanager/action/getInitData' to 'C:\Users\ADMINI~1\AppData\Local\Temp\cicdip_Administrator\1374108030737\nf\cicURLLgetInitData7337285020417225312.tmp'.      This issue can occur for different reasons. You might have an issue with your network connection. If you retrieve the file from a machine or network in your environment, the file system might be mounted incorrectly. On AIX, when you install from a file system that is mounted with the 'cio' option, the installation fails with the exception: A system call received a parameter that is not valid.      Verify your network connection. If you retrieve the files from your environment, copy the source files to a different file system and install from that file system.      java.net.ConnectException: Connection timed out: connect

I ignored the error and installed "IBM installation manager" and I restarted the installation manager and agian failed with same error, I ignored the error and the installation manager didn't have any repositories to install RAD. So I searched in Google and found IBM support page http://www-01.ibm.com/support/docview.wss?uid=swg21457139 which says "Turn Off firewall". But firewall is already disabled in my system. 

My computer is connected to internet via proxy server and based on the error message I suspected that proxy server needs to be configured to obtain the repository information and I'm right.From the installation manager I clicked "Add Repositories" link and expanded "Internet" option and configured proxy server details.

 How do we get repository information to install RAD ? 


Accessed the link https://www.ibm.com/software/repositorymanager/action/getInitData obtained from error message which returns RAD repository data in XML tags. Specified the repository information in Installation manager which listed RAD 9.0 under installation packages.

Note:You need to provide credentials for proxy server and IBM site credentials when repository information is validated in installation manager.RAD repository location: https://www.ibm.com/software/repositorymanager/com.ibm.rational.application.developer.v9

 

 

Monday, November 12, 2012

Websphere EJB Optimization: Convert pass by value calls to pass by reference without Local Interface

I've encountered a specific situation in the project where EJB calls should be optimized to reduce response time and memory footprint without modifying code.

Every J2EE programmer know about EJB local interfaces that enables EJB calls like local method calls as long as client and EJB resides in same physical machine. This option reduces overhead of data serialization and transfer between client and EJB layer and also reduces memory foot print since the data should be copied from one layer to another layer.

How do we enable the same feature for any existing project missing Local Interfaces ?

IBM websphere provides the following options to optimize EJB calls without adding local interfaces.

1) Set "Pass by reference" property  in ORB service to true. Navigate to Application servers --> server_name --> Container services --> ORB service. Find the property "pass by reference" and set the value to TRUE.

2)Specify noLocalCopies in servant.jvm.options (-Dcom.ibm.CORBA.iiop.noLocalCopies=1).

Caution: We must ensure that the application is not negatively impacted by using pass-by-reference semantics. For example, if one bean passes another bean an object, expecting that the second bean can modify the object without affecting the first bean's object.

Wednesday, October 24, 2012

Unable to upload the TIF image of large dimensions 1920x1080 to Java Application


I got an incident from customer saying that we are unable to upload TIF images to application and how ever they are able to upload when it was opened in Photoshop and saved.

I have raised this issue with third party and we were not having exact application logs to look into

Reason:

Application could not load this because it was missing the dimension information (width x height). This information is required in order for STEP to be able to load a TIFF image.
The dimension information is metadata on the file that is normally provided by whatever program the file was created in. Unfortunately, I cannot tell what program created this file, as it did not leave its name in the metadata either. Re-saving this image in any Photoshop application will supply the required metadata and then you should be able to import it into STEP. Just be aware that Photoshop will default to 72 dpi unless a different resolution is set.






Workaround:

  1. Select all the images into Microsoft Picture Manager.
  2. Click on Edit Pictures -- >Re-size and then select the required dimensions and press OK.
  3. Click on File -- >Save All to save all the re-sized images
Please note that original images will be re-sized and make sure that back up original images in case if required

Java DNS Cache issues

Java 1.5 DNS Cache issues:

Recently I encountered an issue with the application and things went pretty ugly. One of the database server is migrated from one hardware to another (copy DB to new HW) and DNS change was part of the implementation. The permissions on old database tables were revoked, but we are able to connect to that database.

Once all migration activities are completed I started testing our application (J2EE) connectivity to new database server (after couple of hours) I encountered permission issue on selecting database tables. 

DBA's were telling that I'm still connecting to old servers and requested to point JDBC connection to new servers, but the fact is we have the new host name already in our JDBC string which is used in other part of the application.

Finally we figured it out the issue would have happened because of DNS cache and I found out that DNS queries to our new database server has TTL value of 3600 seconds, so Linux DNS cache is not an issue since I started testing couple of hours after DNS change.

use dig +ttlid <host-name> command in Linux to find out TTL value returned by DNS servers.

;; ANSWER SECTION:
google.com.             300     IN      A       74.125.225.39
google.com.             300     IN      A       74.125.225.34
The value of 300 is TTL for google.com and DNS entries for google.com will be flushed out from DNS cache after 300 seconds.

So where is the issue ?

Root Cause:

Java (1.4 & 1.5) has internal DNS cache whose entries never expires ( TTL =-1) which caused our application to connect to old servers.

Resolution:

Change the TTL value of Java DNS cache:

The SDK provides the following properties to control network address caching:
networkaddress.cache.ttl (default: -1)

You can use the following four properties to override the default behavior.

Workarounds:

  • Upgrade to JDK 1.6 / JDK 1.7 which introduced changes to default DNS caching behavior. The TTL values are set to 30 seconds without a security manager. 
  • JVM restart after DNS changes.

References:

  1. http://www-01.ibm.com/support/docview.wss?uid=swg21207534
  2. http://www-01.ibm.com/support/docview.wss?uid=swg21439843


Wednesday, October 17, 2012

Websphere connection pool errors:ORA-02396: exceeded maximum idle time, please connect again

Today one of our application experienced Oracle time out errors “ORA-02396: exceeded maximum idle time, please connect again” when ever user submits long running operations and was able to re produce the same issue in my local environment with the help of debugger .

Oracle profile for the app user has idle time out of 45 minutes and when ever application took more than 45 minutes for processing the request and tries to update the results to DB, the application is throwing this error. Oracle discards idle connections after 45 minutes and for some reason websphere is not discarding this stale connections. Application is getting connection from Connection pool via JNDI lookup and closing out resources during method exits and not keeping the connection open.


The data source has the default values for connection pool properties.
Reap time: 180 seconds
Unused timeout: 1800 seconds
Aged timeout: 0 seconds
Global Transactions :No
EJB Transaction Attribute:Default (required) -CMT
Transaction started : Not in Web Layer/EJB Layer. DAO classes are opening connections.
Pattern: open/use/close pattern for DB connections.

I’ve changed “Aged timeout parameter” to 1200 seconds in my local environment which didn’t help. Not sure why Connection pool is keeping stale connections.

Finally I decided to do more research why stale connections are not discarded from Websphere connection pool even with appropriate settings.

The application specifies “required” transaction attribute for all container managed transactions and resource sharing scope is sharable for all local transactions (LTC).

According to websphere documentation when ever new JDBC connection is allocated under LTC (Local transaction containment) it is marked as having an affinity to the current LTC and when application closes the connection it is not returned to the pool until transaction ends and connection remains used by this transaction so connection pool mechanism can’t discard this connection even though it becomes stale connection.

Please refer http://www.ibm.com/developerworks/websphere/library/techarticles/0506_johnsen/0506_johnsen.html for more details.

I have verified this in my local websphere and when ever LTC has started the same logical connection is returned every time application tries to acquire db connection.


Here are some resolutions for this issue

1) Change the resource sharing scope of JDBC connections to “UnSharable” which creates more physical connection to database instead of sharing single physical connection using multiple logical connections and connection will be released to connection pool after application closes the connection.

2Add exception handler to specific part of the code and obtain new connection gracefully

3)Optimize application processing or long running queries

I did more research on the possible implementation of option (1) and it seems that option (1) is not feasible to implement with our application DB access pattern.

Resource sharing property can be specified either in web.xml or ejb-jar.xml along with resource reference (Datasource) and application code (servlet or EJB) should lookup the data source using this reference container will utilize sharable property (res-sharing-scope).How ever our app code is not using resource references to obtain database connection and instead obtain connections in DAO class.
Since the sharable property in CMConfigData (Connection Manager config Data) object in websphere object and it is not exposed outside (no setters) , the possibility of using IBM helper classes to set the sharable property in the opened database connection is blocked


We can implement Option 2 and 3 together and IBM also suggests handling StaleConnection exceptions in the code and more information is available at http://www.redbooks.ibm.com/redbooks/SG246688/axa.htm





We typically change the classloader from the default of PARENT_FIRST (i.e. get WebSphere to find/load the classes) to PARENT_LAST (i.e the application is responsible for finding/loading classes). However, when we did it for this application, we still got the same error: "java.lang.NoSuchMethodError". So, what we needed was some classloading debugging to find out what was going on...
Setting up debugging of classloaders in Websphere
If you have a dev instance of WebSphere 5.1/6 then the following process should work and allow you to get debugging going. (Note, sorry all you Firefox lovers, the WebSphere admin console needs IE to work for this as it will not allow you to set the right debugging modules.) To setup debugging, go to the Application Server settings (for MDF, this will be "server1")
1. Go to PROCESS DEFINITION > JVM and tick the "Verbose JNI" and "Verbose Classloading" and click on OK.
2. Click at the bottom, CUSTOM PROPERTIES and a new custom property called "ws.ext.debug" set to "true" and again click on OK
3. You have now set up the debugging flags for the bootstrap and system classloaders. You now need to do the same thing for the app servers three classloaders. From the Application server settings (again, "server" for MDF instances) click on LOGGING AND TRACING > DIAGNOSTIC TRACE and click on MODIFY button.
4. A window will pop up (this is the bit that does not work on Firefox) and you need to find (at the bottom) the "Websphere ClassLoader" and click and set to ENABLE ALL. When you click on OK, the text in the Trace Specfication will change to "WebSphere ClassLoader=all=enabled". Now change the log size (default is 20Mb, which is way to small) to around 200Mb. Click on OK. Now SAVE and Syncronise your changes and restart your app.
5. When you restart WAS, you will get a new log file called "trace.log" which will contain all the classloading debugging info, with the boostrap and system classloading going into the native_stdout/err logs.
You can see below the boostrap classloading extract - its a very big file as it outlines all classes WebSphere boots and loads.

...

|Opened /opt/websphere/asvr/java/jre/lib/jce.jar|

|Opened /opt/websphere/asvr/java/jre/lib/charsets.jar|

|Loaded java.lang.Object from /opt/websphere/asvr/java/jre/lib/rt.jar|

|Loaded java.io.Serializable from /opt/websphere/asvr/java/jre/lib/rt.jar|

...
The following sample shows the output in the trace.log (a very very big file, so you are going to need to use text manipulation tools like grep, sed or awk realistically)

...

6/21/07 12:44:47:994 BST]  1e88c7f CompoundClass > findClass name=org.w3c.dom.Node

this=com.ibm.ws.classloader.CompoundClassLoader@42d227

6/21/07 12:44:47:996 BST]  1e88c7f CompoundClass d class org.w3c.dom.Node found in

SinglePathClassProvider : com.ibm.ws.classloader.SinglePathClassProvider@ccd65d classpath =

/u01/websphere/asvr/installedApps/cell/app_war.ear/app.war/WEB-INF/lib/NCSO-6.jar

6/21/07 12:44:47:997 BST]  1e88c7f CompoundClass d loaded org.w3c.dom.Node from this=

6/21/07 12:44:47:998 BST]  1e88c7f CompoundClass d loaded org.w3c.dom.Node using classloader=

6/21/07 12:44:48:162 BST]  1e88c7f CompoundClass > loadClass name=org.w3c.dom.NodeList

this=com.ibm.ws.classloader.CompoundClassLoader@42d227

6/21/07 12:44:48:162 BST]  1e88c7f CompoundClass > findClass name=org.w3c.dom.NodeList

this=com.ibm.ws.classloader.CompoundClassLoader@42d227

...
http://www-1.ibm.com/support/docview.wss?rs=180&uid=swg21228700%3C/p%3E
http://websphere.sys-con.com/read/196105.htm%3C/p%3E
https://issues.apache.org/bugzilla/show_bug.cgi?id=38719#c2
How to change the users passwords from J2EE Application ?

There are no generic J2EE API's to manipulate J2EE application user details.How ever each Application servers exposes MBeans to perform these operations through console or via scripting tools.
Eg: While creating the users from admin console, specifc mbeans will be called behind the scenes.

I wrote an example to reset the password for particular user using weblogic MBeans from Servlet.Eventhough this is not the exhaustive example which provides the basic steps for listing users/groups and modifying the user passwords


import java.io.IOException;
import java.io.PrintWriter;

import javax.naming.Context;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import weblogic.jndi.Environment;
import weblogic.management.MBeanHome;
import weblogic.management.security.RealmMBean;
import weblogic.management.security.authentication.AuthenticationProviderMBean;
import weblogic.management.security.authentication.GroupEditorMBean;
import weblogic.management.security.authentication.GroupMemberListerMBean;
import weblogic.management.security.authentication.MemberGroupListerMBean;
import weblogic.management.security.authentication.UserEditorMBean;
import weblogic.management.security.authentication.UserPasswordEditorMBean;
import weblogic.management.security.authentication.UserReaderMBean;
import weblogic.management.security.credentials.CredentialMapperMBean;
import weblogic.management.security.credentials.UserPasswordCredentialMapEditorMBean;
import weblogic.management.security.credentials.UserPasswordCredentialMapReaderMBean;
import weblogic.management.utils.InvalidParameterException;
import weblogic.management.utils.NotFoundException;

public class PasswordServlet extends HttpServlet
{

protected transient MBeanHome home;


/**
* The doGet method of the servlet.

*
* This method is called when a form has its tag value method equals to get.
*
* @param request the request send by the client to the server
* @param response the response send by the server to the client
* @throws ServletException if an error occurred
* @throws IOException if an error occurred
*/
public void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException
{

doPost(request, response);
}

/**
* The doPost method of the servlet.

*
* This method is called when a form has its tag value method equals to post.
*
* @param request the request send by the client to the server
* @param response the response send by the server to the client
* @throws ServletException if an error occurred
* @throws IOException if an error occurred
*/
public void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException
{
response.setContentType("text/html");
findAdminHome();
PrintWriter out = response.getWriter();
RealmMBean[]realms= home.getActiveDomain().getSecurityConfiguration().findRealms();
for(int n=0;n< authProviders.length; i++) { if (authProviders[i] instanceof UserEditorMBean) { out.println("UserEditorMBean: " + authProviders[i].wls_getDisplayName()); } if (authProviders[i] instanceof GroupEditorMBean) { out.println("GroupEditorMBean: " + authProviders[i].wls_getDisplayName()); } if (authProviders[i] instanceof MemberGroupListerMBean) { out.println("MemberGroupListerMBean: " + authProviders[i].wls_getDisplayName()); } if (authProviders[i] instanceof GroupMemberListerMBean) { out.println("GroupMemberListerMBean: " + authProviders[i].wls_getDisplayName()); } if (authProviders[i] instanceof UserReaderMBean) { UserReaderMBean users=(UserReaderMBean)authProviders[i]; try { //String listUser=users.listUsers("USR*", 100); out.println("Users List: " +users.userExists("USR333")); out.println("Users List: " +users.userExists("USR330")); } catch (InvalidParameterException e) { // TODO Auto-generated catch block e.printStackTrace(); } } } for (int i = 0; i < credMappers.length; i++) { if (authProviders[i] instanceof UserPasswordCredentialMapReaderMBean) { UserPasswordCredentialMapReaderMBean user=(UserPasswordCredentialMapReaderMBean)authProviders[i]; out.println("UserPasswordCredentialMapReaderMBean: " + credMappers[i].wls_getDisplayName()); } if (authProviders[i] instanceof UserPasswordCredentialMapEditorMBean) { out.println("UserPasswordCredentialMapEditorMBean: " + credMappers[i].wls_getDisplayName()); } if (authProviders[i] instanceof UserPasswordEditorMBean) { UserPasswordEditorMBean pwd=(UserPasswordEditorMBean)authProviders[i]; try { pwd.changeUserPassword("all", "allusers", "ALLUSERS"); out.println("Password Chnaged : " + credMappers[i].wls_getDisplayName()); } catch (NotFoundException e) { e.printStackTrace(); } catch (InvalidParameterException e) { e.printStackTrace(); } } } } out.flush(); out.close(); } private void findAdminHome() { Environment env = new Environment(); env.setProviderUrl("t3://localhost:7001"); env.setSecurityPrincipal("weblogic"); env.setSecurityCredentials("weblogic"); try { Context ctx = env.getInitialContext(); home = (MBeanHome) ctx.lookup(MBeanHome.ADMIN_JNDI_NAME); } catch (javax.naming.NamingException e) { e.printStackTrace(); } } }

Connect to Websphere JDBC datasource from standalone Client:

We may hit a situation to connect to configured Websphere JDBC datasource from standalone java client and need appropriate drivers for target database we are connecting to.

There are no additional jars need to be downloaded from IBM website except database driver jar and we will be using IBM command(setupCmdLine.sh) to setup environment and connect to Oracle database.

Script to setup WAS environment and call Java Client:

# Use IBM script to initialize common WebSphere runtime settings
REPLACE_WAS_HOME=${WAS_HOME}
. $WAS_HOME/bin/setupCmdLine.sh

# Add oracle drivers to classpath
CLASSPATH="$WAS_CLASSPATH:/tmp/JDBC/ojdbc5.jar:."

# Specify JVM
JAVA="$JAVA_HOME/bin/java"

# Specify JVM command line options
JAVA_OPTS="-Djava.ext.dirs=$JAVA_HOME/jre/lib/ext:$WAS_EXT_DIRS:$WAS_HOME/plugins"
JAVA_OPTS="$JAVA_OPTS  -cp $CLASSPATH"
$JAVA $JAVA_OPTS ConnectJDBC

Java Client:

import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import java.io.Serializable;
import java.sql.Connection;
import java.util.ArrayList;
import java.util.Hashtable;
import java.util.List;
import java.io.*;

public class ConnectJDBC {

public static void connect() {
final Hashtable env = new Hashtable();
env.put(Context.INITIAL_CONTEXT_FACTORY,"com.ibm.websphere.naming.WsnInitialContextFactory");
env.put(Context.PROVIDER_URL, "corbaloc:iiop::,::,::");

final Context jndiContext;
try {
jndiContext = new InitialContext(env);
} catch (NamingException e) {
System.err.println("Could not create JNDI API context: "+ e.toString());
System.exit(1);
return;
}

try {
javax.sql.DataSource ds =  ((javax.sql.DataSource)jndiContext.lookup(""));
Connection conn =ds.getConnection(,);
String SQL = "{call . }";
} catch (Exception e) {
System.err.println("Could not perfrom JDBC Procedure Call: "+ e.toString());
System.exit(2);
}
}

public static void main(String[] args) {
connect();
}
}

Note: Please make sure the script #setupCmdLine.sh has appropriate permissions to avoid any failures. Note that standalone client will not be able use component-managed authentication for database and it's responsibility of client to provide authentication details during database connection..

Dependency Jars:

The standalone client need the following jars to connect to JDBC datasource configured in websphere when we are unable to use #setupCmdLine.sh script to setup the environment.

com.ibm.ws.runtime_6.1.0.jar
ojdbc14.jar
rsadbutils.jar
rsahelpers.jar
sibc.jndi.jar
sibc.jms.jar
sibc_install-o0902.06.jar
com.ibm.jaxws.thinclient_6.1.0.jar

Note: I performed all these tests using eclipse editor and Websphere 6.1 server.

Post JMS messages to WebSphere queues from stand alone Client:

We may encounter a situation to send JMS messages to queues/topics configured in Websphere from standalone java program and unit/functional testing would be one good scenario for our case.

We need additional client libraries to lookup resources from JNDI and connect to websphere SIB and queues which can be downloaded from here.

Client Program:

import javax.jms.*;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.Hashtable;
import java.util.List;
import java.io.*;


public class PostJMSMessage {

public static void publish() {
final Hashtable env = new Hashtable();
env.put(Context.INITIAL_CONTEXT_FACTORY,"com.ibm.websphere.naming.WsnInitialContextFactory");
env.put(Context.PROVIDER_URL, "iiop://:");

final Context jndiContext;
try {
jndiContext = new InitialContext(env);
} catch (NamingException e) {
System.err.println("Could not create JNDI API context: "+ e.toString());
System.exit(1);
return;
}

try {
ConnectionFactory connectionFactory = (ConnectionFactory) jndiContext.lookup("");
Connection qConn = connectionFactory.createConnection();
Session qSession = qConn.createSession(false,Session.AUTO_ACKNOWLEDGE);
Queue q = (Queue) jndiContext.lookup("");

MessageProducer producer = qSession.createProducer(q);
ObjectMessage message = qSession.createObjectMessage();
message.setObject((Serializable) );
producer.send(message);

producer.close();
qSession.close();
qConn.close();
in.close();
} catch (Exception e) {
System.err.println("Could not perfrom JMS operation: "+ e.toString());
System.exit(2);
}
}

public static void main(String[] args) {
publish();
}
}


Jars required in classpath:

The following jars are required in classpath along with any application specific jars to compile and run the program

sibc.jms.jar
sibc.jndi.jar
sibc_install-o0902.06.jar

Compile & Run:

Navigate to the directory where the client program and jars are copied and find out IBM jdk path.

/opt/IBM/WebSphere/AppServer/java/bin/javac -cp :sibc.jms.jar:sibc.jndi.jar:. PostJMSMessage.java

/opt/IBM/WebSphere/AppServer/java/bin/java -cp :sibc.jms.jar:sibc.jndi.jar:. PostJMSMessage

Monday, July 25, 2011

How to extract exception stack trace from javaEE application log file ?

I had the requirement to extract complete list of exception stack trace from java EE application log files for past 6 months and analyze the root cause for all exceptions.

I used the following sed commands to get the list which has all duplicate list of exceptions.

sed -ne '/ERROR/,/WebContainer :/p' *.log.2011* >/tmp/exception_list.txt
sed -i '/WARN/d' /tmp/exception_list.txt
sed -i '/INFO/d' /tmp/exception_list.txt
sed '/com.ibm.ws/d' -e '/org.apache/d' /tmp/exception_list.txt
sed -e '/com.ibm.ws/d' -e '/org.apache/d' /tmp/exception_list.txt


Remove duplicate exception block from the log.

1) Remove time/date information from log entry cut -f 2 -d']'
exception_list.txt
2)Remove additional spaces if any sed 's/ ERROR/ERROR/'
/tmp/exception_list.txt

uniq
command will not work unless is sorted (sort), so use below command to remove duplicates with out changing line order

awk ' !x[$0]++'
/tmp/exception_list.txt >/tmp/exception_list_final.txt


Wednesday, July 20, 2011

How to delete duplicate MS communicator conversation copied to outlook History folder ?

Recently I faced below problem and couldn't find any solution/tools in internet. I decided to solve the problem myself.

I have enabled the option to save MS communicator conversation in outlook and the conversation will be saved to outlook at particular intervals. In case the chat session go beyond this interval, multiple copies of session will be saved with the information at the moment of interval timeout. i.e . The recent copy of the conversation includes all information from previous copy and all older copies can be deleted from outlook folder.

The tools to delete duplicate items didn't help here so I wrote VB script to handle this task myself. Here is the logic
1)Open Conversation History folder
2)Sort items by subject then Received in ascending order to group the items by subject and get the oldest item first among the subject
3) In first step mark the first item as pointer and named as X
4) In subsequent steps compared the subject of current item and previous pointer item (X) and if matches then verified body of item X contained in body of current item
5) If so then delete the item X
6) Outside the loop make current item as X and continue the loop

VB Script:

Set objOutlook = CreateObject("Outlook.Application")
Set objNamespace = objOutlook.GetNamespace("MAPI")
Set objFldr = objNamespace.Folders.Item("Mailbox - Ramasamy, Yuvaraj").Folders.Item("Conversation History")
Set colItems = objFldr.Items
colItems.Sort "[Subject][Received]", false
item_comp = Null
WScript.Echo colItems.count
For Each objItem in colItems

If isNUll(item_comp ) then
set item_comp = objItem
elseif (item_comp.subject = objItem.subject) then
If left(objItem.body,len(item_comp.body)) = item_comp.body then
item_comp.Delete
end if
end if
set item_comp = objItem
Next
WScript.Echo "all done"

Friday, April 22, 2011

Change/delete locked symantec scheduled admin scan in Windows:

Most organizations should have installed some commercial Anti virus software in all IT assets (laptops/desktops etc...) and add READ ONLY configuration for daily quick scan and weekly scheduled full scan. IT administrators might have scheduled this on weekdays to ensure 100% completion of weekly scan and they don't want to abort any scan or postpone the scan schedule to number of days.

In my office laptop the scheduled weekly scan run on Wednesday's @11:30 AM and will take 5-6 hours to complete.Due to heavy disk scan and CPU contention I'm unable to use other resource hungry applications in my laptop (RAD/Firefox etc...).

Though we can't edit this settings from symantec UI, this can be exploited via registry editor.
Here are the steps to change SYMANTEC in Windowx XP:

1)Open registry editor (regedit)
2)Navigate to "HKEY_LOCAL_MACHINE\SOFTWARE\Symantec\Symantec Endpoint Protection\AV\LocalScans\{xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx}\Schedule" and find out the key based on the name of the scheduled scan.
Note: Otherwise search entire registry for the schedule name (weekly scan wed@11:30) using Find(F4) option
3)Open the DWORD "DayOfWeek" and edit the value as appropriate (6 -for scan on Saturdays)


The schedule can be even deleted from the registry at your own risk which will completely remove the scheduled scan.

Under "LocalScans" you should see many nodes named "ClientScheduledScan_" or something similar. The "ClientScheduledScan_" is the node in the registry tree that registers my forced Administrator system scan with Symantec AntiVirus. Yours could be different, so you'll need to look around under LocalScans to find the correct one.

Here's a snapshot:




Disclaimer
: Disabling admin scan on a PC by your employer is probably a violation of their IT-security policy and could get you into trouble.


Enter your Comments