Publishing WSO2 product logs to a Topic in Apache ActiveMQ.

Hi All,

This post will describe how the logs of a WSO2 product can be published to a topic in Apache ActiveMQ. WSO2 products use apache's log4j module for logging purposes. We will be using the "org.apache.log4j.net.JMSAppender" class to publish logs to a topic. ActiveMQ already has a guide on how this can be done [1].

log4j.properties

Add the following configuration to <WSO2_PRODUCT>/repository/conf/log4j.properties file.

log4j.appender.JMS_APPENDER=org.apache.log4j.net.JMSAppender
log4j.appender.JMS_APPENDER.InitialContextFactoryName=org.apache.activemq.jndi.ActiveMQInitialContextFactory
log4j.appender.JMS_APPENDER.Append=true
log4j.appender.JMS_APPENDER.ProviderURL=tcp://127.0.0.1:61616
log4j.appender.JMS_APPENDER.TopicBindingName=dynamicTopics/logTopic
log4j.appender.JMS_APPENDER.TopicConnectionFactoryBindingName=ConnectionFactory

We also have to add the "JMS_APPENDER" logger to the root logger in log4j.properties file. Modify the following line...

log4j.rootLogger=ERROR, CARBON_CONSOLE, CARBON_LOGFILE, CARBON_MEMORY, CARBON_SYS_LOG, ERROR_LOGFILE

To the following...

log4j.rootLogger=ERROR, CARBON_CONSOLE, CARBON_LOGFILE, CARBON_MEMORY, CARBON_SYS_LOG, ERROR_LOGFILE, JMS_APPENDER

Dependency JAR files.

Add the following jar files from <ACTIVEMQ_HOME>/lib to <WSO2_PRODUCT>/repository/components/lib folder.

• activemq-broker-5.*.jar
• activemq-client-5.*.jar
• geronimo-jms_1.1_spec-1.1.1.jar
• geronimo-j2ee-management_1.1_spec-1.0.1.jar
• hawtbuf-1.*.jar

Reading received log messages.

The logs are published as JMS ObjectMessages from the JMSAppender class. So therefore we wont be able to see the content of it directly. We would need to create a customer client by using the activemq dependency which can be downloaded or used as a pom dependency from here. The jar file is also available at the <ACTIVEMQ_HOME>/ folder. Using this dependency jar and the code below, we can extract the contents of the log messages.

import org.apache.activemq.ActiveMQConnectionFactory;
import org.apache.activemq.command.ActiveMQObjectMessage;
import org.apache.log4j.spi.LoggingEvent;

import javax.jms.Connection;
import javax.jms.JMSException;
import javax.jms.MessageConsumer;
import javax.jms.MessageListener;
import javax.jms.Session;

public class Log4jJMSAppenderExample implements MessageListener {

    public Log4jJMSAppenderExample() throws Exception {
        ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory("tcp://127.0.0.1:61616");
        final Connection conn = factory.createConnection();
        final Session session = conn.createSession(false, Session.AUTO_ACKNOWLEDGE);
        conn.start();
        final MessageConsumer consumer = session.createConsumer(session.createTopic("logTopic"));
        consumer.setMessageListener(this);

        Runtime.getRuntime().addShutdownHook(new Thread() {
            @Override
            public void run() {
                try {
                    consumer.close();
                    session.close();
                    conn.close();
                } catch (JMSException e) {
                    throw new RuntimeException(e);
                }
            }
        });
    }

    public static void main(String[] args) throws Exception {
        new Log4jJMSAppenderExample();
    }

    public void onMessage(javax.jms.Message message) {
        try {
            // receive log event in your consumer
            LoggingEvent event = (LoggingEvent) ((ActiveMQObjectMessage) message).getObject();
            System.out.println("Received log [" + event.getLevel() + "]: " + event.getMessage());
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
}

Notes

If there is a need to use the jndi.properties file, we will have to place it at the <WSO2_PRODUCT>/ folder instead of the <WSO2_PRODUCT>/repository/conf/. The jndi.properties file is not necessary as we have defined the topic in log4j.properties as "dynamicTopics/logTopic".

I have tested the above with WSO2 ESB 4.8.1 and Apache ActiveMQ 5.11.1.

References...

[1] - http://activemq.apache.org/how-do-i-use-log4j-jms-appender-with-activemq.html
[2] - https://docs.wso2.com/display/ESB481/Configure+with+ActiveMQ