Skip to content

Homer 7 Stack Installation on Debian 9 Stretch

Jump to bottom
aqsyonas edited this page Aug 6, 2019 · 19 revisions

HOMER 7

Major Changes

  1. Capture Server was changed from Kamailio to Heplify-Server.
  2. PostgreSQL was used instead of MySQL due to better partitioning, native JSON support.
  3. Supports Natively different Metrics Stacks. (TICK, Prometheus, Loki, ElasticSearch...)
  4. Better APIs

Table of Contents

  1. PostgreSQL-10
  2. Homer Web App
  3. HEPlify-Server
  4. TICK STack
  5. HEPlify Capture Agent
  6. Useful Information and References

Overview of Components

There are quite a few components which make up a complete Homer 7 stack. Below is the list for this guide. These are the links to each individual component for easy reference.

PostgreSQL 10

PostgreSQL is where the HEPlify-Server stores all of the raw data that it ingests from the HEPlify Capture Agent. My suggestion is to size this appropriately to your environment. A smaller environment will probably not need as much resources as I've specified here, whereas a larger environment will probably require more. If you can back the database with fast disk that is helpful as well.

VM Specs
Minimum 4 CPUs
Minimum 16GB RAM
Minimum 1TB Storage Space

Package Requirements

apt-get install sudo

Note: Make sure your PostgreSQL Data directory is configured to point to a very large data storage space. By default PostgreSQL will put all data into the same directory as the configuration files.

Install and Start PostgreSQL-10

  1. Add the key.
    wget -q https://www.postgresql.org/media/keys/ACCC4CF8.asc -O- | sudo apt-key add -

  2. Add repository.
    echo "deb http://apt.postgresql.org/pub/repos/apt/ stretch-pgdg main" > /etc/apt/sources.list.d/postgresql.list

  3. Install postgresql-10 from the repo.
    apt-get update && apt-get install -y postgresql-10

  4. Enable the services.
    systemctl enable postgresql
    systemctl start postgresql

  5. Create the homer databases and set appropriate permissions.
    sudo su - postgres
    createuser homer --pwprompt
    createdb -O homer homer_config
    createdb -O homer homer_data
    psql
    postgres=# GRANT ALL PRIVILEGES ON DATABASE "homer_config" to homer;
    postgres=# GRANT ALL PRIVILEGES ON DATABASE "homer_data" to homer;

  6. Modify the connection file to allow inbound connections to the postgresql services(If postgresql is on different server than homer-app and heplify-server).
    vi /etc/postgresql/10/main/pg_hba.conf
    Add to the bottom of the file this line:
    host all all [IP of HEPlify-Server]/32 password
    host all all [IP of homer-app server]/32 password

  7. Modify the postgresql.conf configuration file and set the following variables to have a good running server.

  • I found this PostgreSQL tuning guide: http://linuxfinances.info/info/quickstart.html
  • File Path: /etc/postgresql/10/main/postgresql.conf
  • listen_addresses = '*' under the -Connection Settings- header.
  • shared_buffers = 1024MB
  • effective_cache_size = 12GB
    Note: This will likely be most of the servers physical memory, if postgres is installed by itself.
  • max_locks_per_transaction = 1000
  • data_directory = '[path to data directory]'
  1. Restart the postgresql-10 service to commit the changes.
    systemctl restart postgresql

Homer Web App

This is the Web application which displays the Homer WUI to the user.

System Requirements:

  • 2 vCPUs
  • 8GB RAM
  • 40GB Storage

Package Requirements

apt-get install curl git sudo software-properties-common make g++

  1. Install NVM.
    curl -sL https://deb.nodesource.com/setup_10.x | bash -
    apt-get install -y nodejs

  2. Clone the GitHub repository to /opt/homer-app
    cd /opt
    git clone https://github.com/sipcapture/homer-app

  3. Install npm dependencies.
    cd /opt/homer-app
    npm install --unsafe-perm && npm install --unsafe-perm -g knex eslint eslint-plugin-html eslint-plugin-json eslint-config-google

  4. Configure the Postgres Database connection in the config.js file.
    vi /opt/homer-app/config.js
    Set host: value to the IP address of the PostgreSQL server.
    Set user: and password: values to a user with access to the homer_data and homer_config databases.

  • This is an example of what the configuration block will look like for the PostgreSQL configuration.

      	const pgsql = {
  	  host: '[postgre_server_ip]',
  	  user: '[postgre_user]',
  	  port: 5432,
  	  password: '[postgre_password]',
  	  charset: 'utf8',
  	  timezone: 'utc',
  	  pool: {
  	    afterCreate: function(connection, callback) {
  	      connection.query('SET timezone = "UTC";', function(err) {
  	        callback(err, connection);
  	      });
  	    },
  	  },
  	};

  1. Insert default data by using below commands.
    knex migrate:latest
    knex seed:run

  2. Build the app bundle using webpack. Do this within the /opt/homer-app directory.
    npm run build

  3. Setting up the homer-app as a service.

  • Add the following service file in /etc/systemd/system/

  • Name: homer-app.service
    Put this in the file:

      	[Unit]
  	Description=Homer 7 UI
  	After=network.target
  	
  	[Service]
  	WorkingDirectory=/opt/homer-app
  	ExecStart=/usr/bin/node --max_old_space_size=2048 /opt/homer-app/bootstrap.js
  	Restart=on-failure
  	RestartSec=10s
  	Type=simple
  	
  	[Install]
  	WantedBy=multi-user.target
  1. Reload the systemctl services and start the service.
    systemctl daemon-reload
    systemctl enable homer-app
    systemctl start homer-app

You can now browse to the server IP in order to verify that it's working properly.

HEPlify-Server

The HEPlify-Server is the "traffic cop" of all of the data that comes into the Homer 7 stack. It is this component which ingests all the data, then sends it out to all the other components in the proper format.

System Requirements:

  • 8 vCPUs
  • 16GB RAM
  • 100GB Storage

Package Requirements

  • libpcap-dev
  • Go programming language

Install Golang.
cd /usr/src/
curl -O https://dl.google.com/go/go1.12.7.linux-amd64.tar.gz
tar xvf go1.12.7.linux-amd64.tar.gz
sudo chown -R root:root ./go
sudo mv go /usr/local

  • First, set Go's root value, which tells Go where to look for its files: vim ~/.profile
    export GOPATH=$HOME/work
    export PATH=$PATH:/usr/local/go/bin:$GOPATH/bin
  • Next, refresh your profile by running:
    source ~/.profile

Install the Heplify-Server Binaries

  1. cd to opt
    cd /opt

  2. Clone the github repo into the /opt directory.
    git clone https://github.com/sipcapture/heplify-server

  3. Build the services using Go.
    cd /opt/heplify-server && go build cmd/heplify-server/heplify-server.go

  4. Modify the configuration file to point to the proper Postgres services.
    cp /opt/heplify-server/example/homer7_config/heplify-server.toml /opt/heplify-server/
    vi /opt/heplify-server/heplify-server.toml

  • Set DBAddr to "[ip of postgres]:5432"
  • Set DBUser and DBPass according to what you configured when setting up the PostgreSQL-10 database user.
  • Set DBWorker equal to the number of CPUs in the system.
  • Set ESDiscovery to false if you do not plan on using Elasticsearch.
  • Set LogDbg to "hep,sql"
  • Set LogLvl to "warning"
  • Set DiscardMethod to ["OPTIONS","NOTIFY"]
  • Set PromAddr to "0.0.0.0:9096"

Note: A good guide on all of the HEPlify-Server configuration parameters can be found in the Wiki here: HEPLIFY-SERVER-Settings

  1. Setup the system service to run the heplify-server binary when the system loads.
    cp /opt/heplify-server/example/heplify-server.service /etc/systemd/system/
    systemctl daemon-reload
    systemctl enable heplify-server
    systemctl start heplify-server

  • You can verify the service is running with systemctl status heplify-server

  • You can verify that the service is able to connect to the postgre server by looking at the /opt/heplify-server/heplify-server.log file.

If everything is working properly the Heplify-Server service is now ready to ingest data from the Heplify capture agent service. You can verify this by using the tail command to look at the heplify-server.log file. If working properly you will see the packets flowing through this file.

tail -f /opt/heplify-server/heplify-server.log

You can verify that PostgreSQL is writing data into the database by using the top command. There should be postmaster commands running in the output.

TICK Stack Installation

InfluxDB is an Open Source Time Series Database Platform for storing Time Series Data, these are metrics & Events collected from different devices. It is a product of InfluxData and part of TICK Stack – which comprises:

T – Telegraf; This is an agent written in Go for collecting performance metrics from the system it’s running on and the services running on that system. The collected metrics are output to InfluxDB or other supported data stores.
I – InfluxDB
C – Chronograf; A web application written in Go and React.js that provides the tools to visualize your monitoring data and easily create alerting and automation rules.
K – Kapacitor A data processing engine. It can process both stream and batch data from InfluxDB.

System Requirements:

  • 4 vCPUs
  • 8GB RAM
  • 1TB Storage

Package Requirements

  1. Add repository for the TICK Stack
    echo "deb https://repos.influxdata.com/debian stretch stable" | sudo tee /etc/apt/sources.list.d/influxdb.list

  2. Install the TICK Stack.
    sudo apt-get update && sudo apt-get install influxdb kapacitor telegraf chronograf

  3. Start and Enable the Services.
    sudo systemctl start influxdb && sudo systemctl enable influxdb
    sudo systemctl start telegraf && sudo systemctl enable telegraf
    sudo systemctl start kapacitor && sudo systemctl enable kapacitor
    sudo systemctl start chronograf && sudo systemctl enable chronograf

Note: You need to configure Scrapper in chronograf to get stats from the Heplify-Server.

The HEPlify Capture Agent

HEPlify Capture needs to be installed on the system you would like to capture the traffic on. The HEPlify Capture captures the data, sends it to the HEPlify-Server to be ingested which sent it out to the other components of the stack.

OS Packages:

  • The Go programming Language.
  • PCAP Libraries

Installation

  1. Install your prerequisites.
    apt install git make libpcap-dev curl sudo

  2. Clone the github repo
    cd /opt/ && git clone https://github.com/sipcapture/heplify

  3. Run the make file in the cloned location with the make command.
    cd heplify && make

  4. Note: Heplify capture requires root permissions to run.

Testing
You should now be able to start the heplify capture by running the heplify executable file. The output should be sent to the screen, and the heplify.log file should show the most recent information.

Service Installation

  • Copy the example service file to the proper spot in the file system.
    cp /opt/heplify/example/heplify.service /etc/systemd/system/

  • Modify the executable path in the file to match what you want to be capturing. This is where you would modify it to specify which physical interface to listen on, as well as what server to send the captured packets to.

  • This is what the production hep capture service file looks like. Note that [interface_name] is the system name of the interface which will be listening and will be the monitor destination below.

      [Unit]
  Description=Captures packets from wire and sends them to Homer
  After=network.target
  
  [Service]
  WorkingDirectory=/opt/heplify
  ExecStart=/opt/heplify/heplify -i [interface_name] -hs [ip_of_heplify_server]:9060 -m SIPRTCP
  ExecStop=/bin/kill ${MAINPID}
  Restart=on-failure
  RestartSec=10s
  Type=simple
  
  [Install]
  WantedBy=multi-user.target

  • Enable the service.
    systemctl daemon-reload
    systemctl enable heplify
    systemctl start heplify

  • Validate that the service is running by using systemctl status heplify

  • The log is written to /opt/heplify/heplify.log

TESTING(HEPGEN)

Barebone HEP Packet Generator for SIP-less Devs and Unit Testing. Setup can be verified using Hepgen testing tool.

  1. Clone the github repo into the /usr/src/ directory.
    cd /usr/src/ && git clone https://github.com/sipcapture/hepgen.js.git

  2. Install.
    cd /usr/src/hepgen.js && npm install

  3. Send the traffic using below command. Please replace 127.0.0.1 with your heplify-server actual IP.
    node hepgen.js -s 127.0.0.1 -p 9060 -c "./config/b2bcall_rtcp.js"

Login to your Homer-UI, Adjust time ranges and you should be good to see Call with QOS tab.

Useful Information

Here is some other useful information that I have collected for everyone's reference.

KPIs

Key Performance Indicators which can be looked up here: https://tools.ietf.org/html/rfc6076

Session Establishment Ratio (SER)

This metric is used to detect the ability of a terminating UA or downstream proxy to successfully establish sessions per new session INVITE requests

(C) 2008-2023 QXIP BV

Homer Wiki


HEP/EEP Agent Examples:

HEP/EEP Agent Examples (LOGS):

HEP/EEP Proxy:

Extra Examples:

Extra Resources:

Clone this wiki locally