Configuring Components

Collector-http

The configuration file location: /opt/smartlabs/smartcare/collector-http/conf/collector-http.conf

Parameter	Description
app-name	Application name. Displayed only in the Apache Spark administrative interface (http://<ip>:4040/).
batch-duration	Batch duration (sec). Apache Spark processes data not in real time, but in batches. For example, the data is accumulated for 20 seconds (in this case — HTTP GET requests from subscriber devices), and then begin to be processed. Before the next batch (i.e. after 20 seconds) Collector should have time to process the incoming data and save the session information and other data in the database.
checkpoint-dir	Directory for the raw data just received within the batch. In case of a critical failure of Collector, after restart, the application will re-read the data from the directory and process them. In addition, the directory is used to store data for every minute slices, i.e. if the size of the batch is less than a minute, the data will still be stored in the directory for at least a minute.
control-port	Control port to change logging level (used for debugging purposes). To do this, you can use, for example, the standard Telnet utility and the command telnet localhost <control-port> xx, where xx is the logging level identifier: li — INFO, ld — DEBUG.
debug	Function to record intermediate collected / processed data as text files (used for debugging purposes).
debug.output-save	Flag to write intermediate data to the file system.
debug.parent-dir	Directory to write intermediate data.
mongodb	mongoDB database parameters to write the result of data processing.
mongodb.connection-string	URL to access the mongoDB database.
services	Settings of SmartCARE clients data processing services.
services.client	Settings of the service receiving the HTTP GET requests from subscriber devices.
services.client.enable	Flag to enable the service receiving the HTTP GET requests from subscriber devices.
services.client.port	Port to receive HTTP GET requests.
services.client.servletPattern	URL pattern to define the HTTP GET requests. For example, if the field value is “/care/*”, the following URL will be considered: http://ip-address:5236/care?<http-get-request>
services.client.baseDir	Directory for integrated Tomcat accepting the HTTP GET requests. Requires minimal space.
services.client.expireAfter	Session timeout. If the subscriber device has not sent any HTTP GET requests during this time, Collector finishes the session. If an event with the same CSI arrives again, the finished session will be reopened.
services.client.appropriateUidsPatterns	UID list to record the sessions. Events for the rest of the UIDs will be ignored. Used for test purposes and to reduce the load on the Collector server.
services.client.session	Session data recording settings.
services.client.session.short	Settings for automatic deletion of data on finished sessions with certain characteristics. All characteristics are taken into account simultaneously.
services.client.session.short.delete	Flag to enable the automatic deletion of data on finished sessions with certain characteristics.
services.client.session.short.durationThreshold	Maximum session duration for automatic session data deletion (sec). That is, if the session duration is shorter than the set value, the session data will be automatically deleted.
services.client.session.short.qualityThreshold	Minimum session quality (Algorithm 3) for automatic session data deletion. That is, if the session quality is higher than the set value, the session data will be automatically deleted.

Example Configuration

collector-http = {
app-name = “collector-http”

run-mode = {
test = false
test-master = “local[*]”
}

batch-duration = 20
checkpoint-dir = “/opt/smartlabs/smartcare/new/tmp/collector-http/checkpoint”
control-port = 4450

debug = {
output-save = false
parent-dir = “/opt/smartlabs/smartcare/new/tmp/collector-http/debug/”
}

# target MongoDB
mongodb = {
connection-string = “mongodb://smartcare-new-collector:password@127.0.0.1:27017/smartcare-new”
}

# list of services
services = {

# http messages receiver
client = {
enable = true
port = 5236
servletPattern = “/care/*”
baseDir = “/opt/smartlabs/smartcare/new/tmp/collector-http/tomcat”
expireAfter = 10
appropriateUidsPatterns = [“E4:27:71:2B:6”]

session = {

short = {
delete = true
durationThreshold = 60
qualityThreshold = 95
}
}
}
}
}

Collector-nginx

The configuration file location: /opt/smartlabs/smartcare/collector-nginx/conf/collector-nginx.conf

Parameter	Description
app-name	Application name. Displayed only in the Apache Spark administrative interface (http://<ip>:4040/).
batch-duration	Batch duration (sec). Apache Spark processes data not in real time, but in batches. For example, the data is accumulated for 20 seconds (in this case — nginx syslog data), and then begin to be processed. Before the next batch (i.e. after 20 seconds) Collector should have time to process the incoming data and save the session information and other data in the database.
checkpoint-dir	Directory for the raw data just received within the batch. In case of a critical failure of Collector, after restart, the application will re-read the data from the directory and process them. In addition, the directory is used to store data for every minute slices, i.e. if the size of the batch is less than a minute, the data will still be stored in the directory for at least a minute.
control-port	Control port to change logging level (used for debugging purposes). To do this, you can use, for example, the standard Telnet utility and the command telnet localhost <control-port> xx, where xx is the logging level identifier: li — INFO, ld — DEBUG.
debug	Function to record intermediate collected / processed data as text files (used for debugging purposes).
debug.output-save	Flag to write intermediate data to the file system.
debug.parent-dir	Directory to write intermediate data.
resources.parent-dir	Directory with Collector required resources.
resources.channel-bitrates-json-dict	Directory inside the resource directory, containing special JSON files with information about channels and their possible bitrates — if the operator uses adaptive streaming technology (multibitrate) for a number of channels.
resources.channel-bitrates-simple-dict	Directory inside the resource directory, containing special JSON files with information about channels and their possible bitrates — if the operator uses only one bitrate for a number of channels.
bgp	BGP module access settings.
bgp.enable	Flag to enable BGP moule polling.
bgp.type	Type of access to the BGP module (currently only “http” is used)
bgp.http.bgpClientSearchUrl	URL of the HTTP request to the BGP module for autonomous systems on the network section from the streaming server on which the module is installed and the subscriber device.
mongodb	mongoDB database parameters to write the result of data processing.
mongodb.connection-string	URL to access the mongoDB database.
services	Settings of SmartCARE clients data processing services.
services.nginx	Settings of the service that processes the streaming server (nginx) data.
services.nginx.enable	Flag of enable the service that processes the streaming server (nginx) data.
services.nginx.udp	Connection settings to the UDP server sending the nginx rsyslog data.
services.nginx.udp.host	UDP server address.
services.nginx.udp.port	UDP server port.
services.nginx.bitrate	Here you can specify a single bitrate for all TV channels. It can be used if all TV channels are streamed at the same bitrate.
services.nginx.bitrate.default	Flag to enable the services.nginx.bitrate function: true — the services.nginx.bitrate.defaultValue value is used; false — the bitrate data from the nginx UDP messages are used.
services.nginx.bitrate.defaultValue	Bitrate value for all channels by default.
services.nginx.saveSessionData	Flag to enable the nginx sessions data recording.
services.nginx.saveClusterData	Flag to enable the nginx sessions cluster data recording (in case of multibitrate streaming).
services.smrecord	Settings for processing data from the incoming TV streams recording server (SmartMEDIA Conveyor / Recorder).
services.smrecord.enable	Flag to enable data processing from the incoming TV streams recording server.
services.smrecord.udp	Connection settings to the UDP server sending the SmartMEDIA Conveyor / Recorder rsyslog data.
services.smrecord.udp.host	UDP server address.
services.smrecord.udp.port	UDP server port.
services.zabbix	Settings for processing data from zabbix agents on incoming TV streams recording servers and streaming servers installed.
services.zabbix.enable	Flag to enable processing data from the zabbix agent.
services.zabbix.check-period	Zabbix agents polling period (s).

Example Configuration

collector-nginx = {
app-name = “collector-nginx”
run-mode = {
test = false
test-master = “local[*]”
}

batch-duration = 20
checkpoint-dir = “/opt/smartlabs/smartcare/new/tmp/collector-nginx/checkpoint”
control-port = 4451

debug = {
output-save = false
parent-dir = “/opt/smartlabs/smartcare/new/tmp/collector-nginx/debug/”
}

resources = {
parent-dir = “/opt/smartlabs/smartcare/new/collector-nginx/resources/dict”
channel-bitrates-json-dict = “json”
channel-bitrates-simple-dict = “json-simple”
}

bgp = {
enable = false
type = “http”
http = {
bgpClientSearchUrl = “http://127.0.0.1:5060/search?ip=”
}
}

# target MongoDB
mongodb = {
connection-string = “mongodb://smartcare-new-collector:password@127.0.0.1:27017/smartcare-new”
}

# list of services
services = {

nginx = {
enable = true
udp = {
host = “localhost”
port = 10534
}

bitrate = {
default = true
defaultValue = 2000000
}

saveSessionData = true
saveClusterData = true
appropriateUidsPatterns = [“E4:27:71:2B:6”]
}

smrecord = {
enable = true
udp = {
host = “localhost”
port = 10535
}
}

zabbix = {
enable = false
check-period = 60
}
}
}

List of Zabbix Agents

In the main menu section “SETTINGS → Servers”, you can define settings for Zabbix agents installed on streaming, storage, or any other servers.

Parameter	Description
Server
Server address	IP address of the server.
Port	Connection port of the server.
Server name	User defined server name.
Enabled	Flag to enable collecting data from the agent.
Interfaces
Name	Network interface name. Initially determined by the agent.
Turned on	Sign of network interface activity on the server.
Enabled	Flag to enable collecting data from the network interface.

Configuring Components

Collector-http

Collector-nginx

List of Zabbix Agents

CONTENTS