Logstash Coralogix Output Plugin

This repository is deprecated and will no longer be supported by Coralogix.

If you are still using our plugin for some reason please contact support to convert to the generic HTTP output solution.

Coralogix provides a seamless integration with Logstash, so you can send your logs from anywhere and parse them according to your needs.

Table of contents

1. Prerequisites
2. Usage
3. Installation
4. Configuration

Prerequisites

Have Logstash installed, for more information on how to install: https://www.elastic.co/guide/en/logstash/current/installing-logstash.html

Usage

Private Key - A unique ID which represents your company, this Id will be sent to your mail once you register to Coralogix..

Application Name - The name of your main application, for example, a company named “SuperData” would probably insert the “SuperData” string parameter or if they want to debug their test environment they might insert the “SuperData– Test”.

SubSystem Name - Your application probably has multiple subsystems, for example: Backend servers, Middleware, Frontend servers etc. in order to help you examine the data you need, inserting the subsystem parameter is vital.

Installation

logstash-plugin install logstash-output-coralogix

If you are not sure where logstash-plugin is located, you can check here:

https://www.elastic.co/guide/en/logstash/current/dir-layout.html

Configuration

Common

Open your Logstash configuration file and add Coralogix output (You should configure input plugin depending on your needs, for more information regarding input plugins please refer to: https://www.elastic.co/guide/en/logstash/current/input-plugins.html).

output {
    coralogix {
        config_params => {
            "PRIVATE_KEY" => "YOUR_PRIVATE_KEY"
            "APP_NAME" => "APP_NAME"
            "SUB_SYSTEM" => "SUB_NAME"
        }
        log_key_name => "message"
        timestamp_key_name => "@timestamp"
        is_json => true
    }
}

The first key (config_params) is mandatory while the others are optional.

Application and subsystem name

In case your input stream is a JSON object, you can extract APP_NAME and/or SUB_SYSTEM from the JSON using the $ sign. For example, in the below JSON, $message.system will extract nginx value.

{
    "@timestamp": "2017 - 04 - 03 T18: 44: 28.591 Z",
    "@version": "1",
    "host": "test-host",
    "message": {
        "system": "nginx",
        "status": "OK",
        "msg": "Hello from Logstash"
    }
}

Record content

In case your input stream is a JSON object and you don’t want to send the entire JSON, rather just a portion of it, you can write the value of the key you want to send in the log_key_name. By default, logstash will send whole record content.

For instance, in the above example, if you write log_key_name message then only the value of message key will be sent to Coralogix.

If you want to send the entire message then you can just delete this key.

Timestamp

Coralogix automatically generates the timestamp based on the log arrival time. If you rather use your own timestamp, use the timestamp_key_name to specify your timestamp field, and it will be read from your log.

Note: We accepts only logs which are not older than 24 hours.

JSON support

In case your raw log message is a JSON object you should set is_json key to a true value, otherwise you can ignore it.

Proxy support

This plugin supports sending data via proxy. Here is the example of the configuration:

output {
    coralogix {
        ...
        # Proxy settings
        proxy => {
            host => "PROXY_ADDRESS"
            port => PROXY_PORT
            user => "PROXY_USER" # Optional
            password => "PROXY_PASSWORD" # Optional
        } 
    }
}