Installing SmartMEDIA
Deployment Type
Local Deployment
For permanent loads (operators of IPTV/OTT services, TV broadcasters, etc.), it is usually expedient to purchase (or use previously purchased) equipment for content delivery tasks. Peak loads can be leveled by using CDN services and/or building an infrastructure with the required performance margin.
Cloud Deployment
SmartMEDIA can also be installed on leased resources and VPSes, so-called “cloud services” (for example, Amazon EC2) and cloud storage services. This solution has a number of advantages:
- absence of capital expenses for hardware resources and technological infrastructure;
- use of systems and architectures of reservation provided by the cloud service provider;
- rapid, flexible modification of the hardware platform to meet the system performance requirements when changing the service needs.
Cloud computing services are reasonable to use for irregular loads on the broadcasting system (for example, broadcasting of certain events — sports events, concerts, etc.) and, accordingly, low profitability when purchasing your own hardware.
Storage of nDVR/VoD content in this case is provided by using storage with access via S3 protocol.
System Requirements
The SmartMEDIA can be installed on servers with the x86_64 architecture and with the CentOS 7 or Red Hat Enterprise Linux 7 operating system.
Regardless of the way resources are allocated — purchased equipment or rental of capacity from hosting providers and cloud service providers, the equipment performance must meet the load requirements.
Exact performance requirements for a certain load (the number of sessions, the number and characteristics of recorded streams, etc.) can only be found in experimental way. General recommendations, explanations and calculations will be given below. They can be used as a starting point for the service’s implementation but should be corrected and actualised in time.
General Recommendations for the Hardware Platform
SmartMEDIA’s performance in stream recording and streaming is mostly determined by the performance of the storage system used (IOPS and bandwidth) and in most cases is limited by the speed of writing media data in the storage and data reading speed. The best performance can be achieved, for example, by increasing the number of disks in the storage system (when using HDD) or using solid-state drives (SSDs).
The main consumers of computing resources (CPU) are:
- SmartMEDIA Conveyor service;
- the operating system in terms of networking (I/O operations and memory copying between user and kernel spaces);
- the operating system in terms of storage performance (the processor is idle but in the I/O Wait state).
The SmartMEDIA Conveyor service, in turn, uses CPU time (CPU) for:
- incoming streams processing;
- streams encryption (if enabled). To increase performance, it’s recommended to use a CPU with hardware support for AES algorithms (AES-NI);
- if transcoding is used — for decoding and encoding of audio streams (video streams will be transcoded by QSV graphics accelerator, see below).
Requirements for Transcoding Servers
SmartMEDIA Transcoder allows you to perform transcoding of live input streams to multi profile adaptive bitrate, DASH or HLS, streams for OTT delivery. It runs on standard off-the-shelf hardware with Intel CPUs with QSV support or NVIDIA GPUs with NVENC support.
NVENC-based Transcoding
For this type of transcoding, you’ll need to use Nvidia GPUs with NVENC / NVDEC hardware accelerated encoding/decoding technology support. For details and the actual list of supported GPU models, please visit the Nvidia website.
Hardware Requirements:
- Of-the-shelf server or PC with supported Nvidia GPU installed
- RAM: 8GB or above
- Storage: HDD/SSD as much size as how long time you want to keep video archive of the recorded channels (SSD is preferable)
- OS: CentOS 7 or Oracle Linux 9
The following tables outline the estimated performance of NVIDIA GPUs when transcoding a 1920x1080i 29.97 source video to 1920×1080 at 4 Mbps, 1280×720 at 2 Mbps, and 640×360 at 600 Kbps output.
Nvidia Quadro P6000 (250W)
15 channels
# gpu pwr gtemp mtemp sm mem enc dec mclk pclk # Idx W C C % % % % MHz MHz 0 96 63 – 24 16 61 98 4513 1657 0 96 63 – 24 16 61 100 4513 1657 0 102 63 – 24 16 61 93 4513 1657 0 104 63 – 24 16 61 97 4513 1657 0 103 63 – 26 18 66 100 4513 1657 0 104 63 – 26 17 64 98 4513 1657 0 98 63 – 24 16 60 99 4513 1657 0 88 63 – 23 16 59 96 4513 1657 0 94 63 – 27 18 69 95 4513 1657 0 106 63 – 25 16 63 97 4513 1657 |
Nvidia RTX A2000 (70W)
13 channels
# gpu pwr gtemp mtemp sm mem enc dec mclk pclk # Idx W C C % % % % MHz MHz 0 64 65 – 24 26 83 96 5700 1217 0 64 65 – 30 29 100 100 5700 1807 0 64 65 – 30 29 100 99 5700 1800 0 63 65 – 24 22 79 65 5700 1860 0 63 65 – 21 19 66 74 5700 1807 0 65 65 – 27 25 84 92 5700 1800 0 62 65 – 20 19 65 81 5700 1807 0 61 65 – 24 23 76 100 5700 1837 0 64 65 – 25 23 79 90 5700 1837 0 66 65 – 25 26 99 52 5700 1845 0 66 66 – 28 27 100 99 5700 1830 0 68 66 – 28 27 100 100 5700 1822 |
Nvidia Tesla T4 (70W)
11 channels
# gpu pwr gtemp mtemp sm mem enc dec mclk pclk # Idx W C C % % % % MHz MHz 0 50 84 – 24 25 96 53 5000 1290 0 51 84 – 23 25 94 52 5000 1215 0 58 84 – 23 25 95 55 5000 1305 0 52 84 – 23 24 94 52 5000 1245 |
Columns description:
- pwr: The current power consumption of the GPU in watts (W).
- gtemp: The current temperature of the GPU in degrees Celsius.
- mtemp: The current temperature of the VRAM in degrees Celsius.
- sm: The percentage of the streaming multiprocessors actively engaged in computations.
- mem: The amount of memory used versus the total available memory on the GPU.
- enc/dec: The load on the video encoding and decoding hardware.
- mclk: The current clock speed of the GPU memory, in MHz.
- pclk: The current clock speed of the GPU cores, in MHz.
The exact performance depends not only on the above parameters, but also on the characteristics of the incoming stream (codec settings, progressive/interlaced scanning, etc.), applied filters, used audio codecs and the number of audio tracks, etc. |
QSV-based Transcoding
For this type of transcoding, you’ll need to use CPUs and chipsets with the Intel Quick Sync Video (QSV) hardware accelerated encoding/decoding technology support. For details and the actual list of supported CPU models, please visit the Intel website.
The performance of the transcoder (the number of simultaneous streams, resolution, frame rate) will depend primarily on the integrated GPU used. The performance of the CPU itself (model, frequency) affects the decoding/encoding of audio streams, demultiplexing of incoming streams and multiplexing of transcoded elementary streams.
The estimated performance of the Intel ® Xeon ® processor E3-1585Lv5 with the Intel ®Iris ™ Pro Graphics P580 is shown below:
Resolution |
Input/Output Codecs |
Number of realtime output streams (30 fps) |
Number of realtime output streams (60 fps) |
576p → 576p |
AVC → AVC |
28 |
14 |
1080p → 1080p |
AVC → AVC |
18 |
9 |
AVC → HEVC |
15 |
7 |
|
HEVC → HEVC |
8 |
4 |
|
4k → 4k |
AVC → HEVC |
4 |
2 |
HEVC → HEVC |
2 |
1 |
The estimated performance of the Intel ® Core ™ i3-9100 processor with the Intel ® UHD 630 graphics is shown below:
Input format |
Output format |
% RENDER |
% VIDEO |
% VIDEO_E |
FREQ |
Capacity |
Limit |
1920×1080 50i |
1920×1080 25p |
13 |
11 |
8 |
230 |
8 |
VIDEO |
1920×1080 50i |
1280×720 25p |
7 |
7 |
7 |
180 |
10 |
VIDEO_E |
1920×1080 50i |
720×576 25p |
5 |
5 |
6 |
150 |
10 |
VIDEO_E |
1920×1080 60i |
1920×1080 30p |
16 |
12 |
10 |
250 |
7 |
VIDEO |
1920×1080 60i |
1280×720 30p |
8 |
9 |
11 |
200 |
9 |
VIDEO_E |
1920×1080 60i |
720×576 30p |
7 |
5 |
11 |
200 |
9 |
VIDEO_E |
720×576 50i |
720×576 25p |
4 |
3 |
2 |
85 |
26 |
RENDER |
720×576 50i |
640×360 25p |
2 |
2 |
2 |
50 |
27 |
RENDER |
1920×1080 60p |
1920×1080 60p |
15 |
17 |
7 |
280 |
6 |
VIDEO |
1920×1080 60p |
1280×720 60p |
11 |
13 |
0 |
220 |
7 |
VIDEO |
1920×1080 60p |
720×576 60p |
8 |
9 |
0 |
170 |
10 |
VIDEO |
1280×720 60p |
1280×720 60p |
8 |
9 |
3 |
200 |
11 |
VIDEO |
1280×720 60p |
720×576 60p |
7 |
7 |
0 |
120 |
16 |
RENDER |
1920×1080 50i |
FHD + HDR + SD 25p |
14 |
13 |
10 |
240 |
5 |
RENDER |
1920×1080 60i |
FHD + HDR + SD 30p |
20 |
18 |
17 |
270 |
5 |
RENDER |
1920×1080 60p |
FHD + HDR + SD 60p |
30 |
26 |
7 |
400 |
4 |
RENDER |
Columns description:
- % RENDER — percentage of EUs usage. Execution Units (EUs): General purpose execution units. These are used for graphics rendering, but they are also suited to a wide range of media processing tasks. Execution Units are general purpose compute engines ideal for video processing uses. They are often used by encode for parts of the algorithm not run in fixed function like rate control and mode decisions.
- % VIDEO — percentage of VDBox usage. In addition to EU slices, there is an “unslice” with additional hardware engines individually schedulable for media tasks — VDBox (also known as MFX) for codec operations and VEBOX.
VDbox/MFX contains:- Bitstream decoder (BSD),
- ENC (intra prediction, motion estimation),
- PAK (quantization, entropy coding, pixel reconstruction, motion compensation). 6 th Generation Core / Skylake architecture introduced a new low power encode path entirely in VDBox called VDenc. However, it is lower quality.
- % VIDEO_E — percentage of VIDEO_ENHANCEMENT, VEbox usage. VEbox/VQE contains:
- Denoise,
- Advanced Deinterlace (ADI),
- Local Adaptive Contrast Enhancement (LACE),
- Camera processing features (skin tone enhancement, etc.).
- FREQ — Average GT frequency.
- Capacity — how many channels can be assigned to a single server.
- Limit — part of the GPU that will be overloaded if another one channel is added.
The exact performance depends not only on the above parameters, but also on the characteristics of the incoming stream (codec settings, progressive/interlaced scanning, etc.), applied filters, used audio codecs and the number of audio tracks, etc. |
Calculating Storage Requirements for nDVR Services
To calculate data storage requirements, the following values are required:
- D — number of channels to be recorded;
- Cndvr — number of concurrent nDVR viewing sessions;
- Bavg — average bitrate of the stream towards the subscriber, kbps;
- Bsum — total bitrate of one channel (the sum of all bitrates of all streams — audio, video, subtitles — of one channel), kbps;
- W — depth (window) of nDVR recording, hours.
Storage Capacity
Vstorage (Kbyte) = Bsum × W × D × 3600 ÷ 8
For example, for recording of 10 channels with a total bitrate of ~ 8000 kbps (video: 3000, 2000, 1500, 1000 kbps, audio: 4 tracks of 128 kbps) within 48 hours, the required storage capacity will be:
8000 (kbps) × 10 (channels) × 3600 (sec/hour) ÷ 8 (bit/byte) × 48 (hours) = ~ 1.65 Tbyte
When calculating the storage capacity, it is necessary to lay some stock (about 10–15%) for cases when input streams’ bitrates won’t be constant, and also due to performance degradation of some file systems/storages which occurs when a certain level of their fullness is exceeded.
Storage Performance (Write)
To calculate the number of IOPS, you need to determine the size of average disk operation, which depends on a significant number of factors, such as: the read and write block size, the ‘readahead’ settings, the data availability in OS’s PageCache, and so on. As a rated value, it is suggested to use the size of the operation 200 Kbyte (1600 Kbit).
In this case:
IOPS = Bsum × D ÷ 1600
For example, to record 10 channels of 8000 Kbit/s each will require storage performance of about 50 IOPS.
Storage Performance (Read) and Caching
Video streaming services (Unicast Live and nDVR) provide delivery of a relatively small amount of content to a large number of subscribers. At the same time, the popularity of different content items differs a lot:
- some programs are watched more often than others,
- the Live content is usually more popular than the content shown a few days ago,
- there can be some “blockbusters”, which will be watched much more often than others,
- etc.
Usually, these facts facilitate a good data caching. The cache can be implemented on several levels:
- the operating system’s level (PageCache).
- the SSD cache for slow block devices (using dmcache/lvmcache or bcache, for example)
- as a simple HTTP cache (in-memory or with SSD drives) with your favourite HTTP reverse proxy.
Using HTTP cache, as a rule, is more effective and in the further calculations this method is considered.
For the nDVR services, it is desirable to provide an HTTP cache volume corresponding to 1–2 hours of content storage, i.e. Sndvr_cache = Bavg × 3600 ÷ 8.
For Unicast Live services, the cache should be approximately 5–10 minutes of content. At the same time, for channels with adaptive streaming, all bitrates must be included in this volume:
Slive_cache = Bsum × 600 ÷ 8.
Depending on the cache size and the load profile, you can achieve different caching efficiency (HitRate) and performance.
In practice, HitRate can meet 90% or more, but for new service calculations it is better to use lower values — about 65–70% for services implementing both nDVR and Live (i.e. 65–70% of traffic will be given out from the cache, load on the storage will be 30–35%). |
Based on this, the storage read performance should be:
IOPS = Bavg × Cndvr ÷ 1600 × (100-HitRate)%,
where “1600” is the size of one disk operation in kilobits (200 Kbyte).
For example, to support 1,000 sessions with a bitrate of 3 Mbps if HitRate = 65%, you will need the storage with a performance of approximately 670 IOPS.
Using the CDN
If you’re expecting a broad geo distribution of the service, and/or want to reduce the impact of short-term peak loads on service quality, one of the possible options will be to use the CDN providers’ services or building your own CDN network.
SmartMEDIA can support all of the major CDN providers, including Akamai, Amazon CloudFront and others.
When using CDN, it is necessary to take into account that:
- all requests for the VoD content can be cached for an infinite period;
- requests for the Live/nDVR chunks can be cached for an infinite period (but it is reasonable to limit the caching period to the recording window duration);
- requests for the HLS/DASH playlists can only be cached for a period equal to the playlist update interval (usually — half of the chunk’s duration). Lifetime DASH manifests can also be cached for an indefinite period.
Preparations
Configure the SmartLabs yum repository
All installation steps require the SmartLabs YUM repository to be configured properly. To configure the repository, run the following commands on the command line (bash):
sudo rpm -iUhv http://smartlabs-public:6vTh5WVrQlgF@repo.smartlabs.tv/repo/public/centos/7/x86_64/smartlabs-smartmedia-v-repo-1.0-1.el7.centos.x86_64.rpm
Configure SELinux
SmartMEDIA installation packages do not include SELinux rules. You can configure them by yourself using permissive mode.
If you’re going to disable SELinux, run the following command on the command line:
sudo setenforce 0
In the /etc/selinux/config file, set SELINUX to disabled:
SELINUX = disabled
After the reboot, check the status of SELinux:
sudo getenforce
The command should return Disabled.
Make Basic Configuration for IPTables Firewall
In this document, we assume that IPTables is used as the firewall with the default policy set to DROP for all the chains of the Filter table.
To install IPTables instead of FirewallD on CentOS7/RHEL7, run the following commands:
sudo systemctl stop firewalld sudo systemctl disable firewalld sudo yum -y install iptables iptables-services sudo systemctl enable iptables |
After setting up IPTables, the default policy for all chains of the filter table will be set to ACCEPT. Be careful while switching it to DROP and ensure you will not loose the access to the server.
Allow traffic for already established TCP connections and traffic on the loopback interface:
# IPTables basic rules sudo iptables -I INPUT -i lo -m comment –comment “Allow Loopback traffic” -j ACCEPT sudo iptables -I INPUT -p tcp -m state -state ESTABLISHED -m comment –comment “Allow incoming traffic for ESTABLISHED TCP connections” -j ACCEPT sudo iptables -I OUTPUT -o lo -m comment –comment “Allow Loopback traffic” -j ACCEPT sudo iptables -I OUTPUT -p tcp -m state -state ESTABLISHED -m comment –comment “Allow outgoing traffic for ESTABLISHED TCP connections” -j ACCEPT |
After completing the installation and configuration of SmartMEDIA, remember to save the configured IPTables rules with:
sudo /usr/libexec/iptables/iptables.init save
Installing Packages
Quick “All-in-one” Installation
To install basic SmartMEDIA packages, run the following command:
sudo yum install smartmedia
After execution of this command:
- MongoDB will be installed;
- the following SmartMEDIA packages will be installed: smartmedia-conveyor, smartmedia-plgen, smartmedia-utils, smartmedia-nginx;
- the /video directory will be created;
- the time will be synchronized from the Internet resource clock.isc.org over the NTP protocol and the following services will be started:
- ntpd,
- mongod,
- smartmedia-nginx,
- smcvr,
- smplgen.
To install other components, see the Individual Component Installation section.
Configure the IPTables rules as follows:
- Allow incoming TCP connections to ports 80 and 443 for the Nginx service:
sudo iptables -I INPUT -p tcp –dport 80 -m state –state NEW -m comment –comment “SM: Allow HTTP requests from Media Streamers” -j ACCEPT sudo iptables -I INPUT -p tcp -dport 443 -m state -state NEW -m comment –comment “SM: Allow HTTPS requests from Media Streamers” -j ACCEPT |
- Allow incoming multicast streams for the Conveyor service:
Bash (Unix Shell) |
sudo iptables -I INPUT -d 224.0.0.0/4 -m comment –comment “SM: Allow incoming multicast” -j ACCEPT # Outgoing multicast traffic (JOIN requests) sudo iptables -I OUTPUT -d 224.0.0.0/4 -m comment –comment “SM: Allow outgoing multicast” -j ACCEPT |
Individual Component Installation
SmartMEDIA components can be installed independently on different servers or virtual machines. Also, you can partially use the existing infrastructure (e.g. HTTP servers or an existing MongoDB installation), or configure a MongoDB cluster with replication.
Stream Processing and Record Servers
Install SmartMEDIA Conveyor:
sudo yum install smartmedia-conveyor
Configure IPTables rules:
# Incoming multicast traffic sudo iptables -I INPUT -d 224.0.0.0/4 -m comment –comment “SM: Allow incoming multicast” -j ACCEPT # Outgoing multicast traffic (JOIN requests) sudo iptables -I OUTPUT -d 224.0.0.0/4 -m comment –comment “SM: Allow outgoing multicast” -j ACCEPT # Outgoing traffic to MongoDB and S3 storage sudo iptables -I OUTPUT -d <mongodb-ip-subnet> -p tcp –dport 27017 -m state –state NEW -m comment –comment “SM: Allow outgoing connections to MongoDB” -j ACCEPT sudo iptables -I OUTPUT -d <s3-ip-subnet> -p tcp –dport <s3-port> -m state –state NEW -m comment –comment “SM: Allow outgoing connections to S3 Data Storage” – j ACCEPT |
If you’re going to use POSIX storage (local disk, RAID array, iSCSI volume) to store media data, install the HTTP server too and allow incoming TCP connections to port 80:
# Install nginx http server sudo yum install smartmedia-nginx # IPTables rules for incoming connections: sudo iptables -I INPUT -s <streamers-ip-subnet> -p tcp –dport 80 -m state –state NEW -m comment –comment “-M ACCEPT |
If you are using HTTP or HLS stream sources, configure the appropriate IPTables rules for outbound connections to source servers.
Playlist Generator
Install the SmartMEDIA Playlist Generator package:
sudo yum install smartmedia-plgen
Configure IPTables rules:
# Incoming traffic sudo iptables -I INPUT -s <streamers-ip-subnet> -p tcp –dport 9000 -m state –state NEW -m comment –comment “SM: Allow FastCGI requests to plgen from Media Streamers” -j ACCEPT # Outgoing traffic sudo iptables -I OUTPUT -d <mongodb-ip-subnet> -p tcp –dport 27017 -m state –state NEW -m comment –comment “SM: Allow outgoing connections to MongoDB” -j ACCEPT |
Index Storage Servers (MongoDB)
Install the MongoDB database management software:
sudo yum install mongodb-org
To ensure fault tolerance, configure MongoDB replication (clustering). As an option you can host MongoDB nodes on the playlist generation servers. This will increase the speed of data access and reduce server-to-server traffic when generating playlists. It is undesirable to use sharding, because it excludes the ability to retrieve all data from the local replica for generating playlists. According to MongoDB recommendations, the odd number of nodes is preferred for replicated setup. For more details on MongoDB replication, see https://docs.mongodb.com/manual/replication/. |
Allow incoming requests to port 27017 from the record servers (to record indexes) and Playlist Generators (for reading indexes for playlists generation):
# Incoming traffic sudo iptables -I INPUT -s <conveyor-ip-subnet> -p tcp –dport 27017 -m state –state NEW -m comment –comment “SM: Allow incoming connections to MongoDB from conveyor servers” -j ACCEPT sudo iptables -I INPUT -s <plgen-ip-subnet> -p tcp –dport 27017 -m state –state NEW -m comment –comment “SM: Allow incoming connections to MongoDB from plgen servers” -j ACCEPT |
HLS/DASH Streamers
Install the smartmedia-nginx package on servers that will deliver content to the end users:
sudo yum install smartmedia-nginx
Configure IPTables Rules:
# IPTables rules for incoming connections from everywhere: sudo iptables -I INPUT -p tcp –dport 80 -m state –state NEW -m comment –comment “SM: Allow HTTP requests from Media Streamers” -j ACCEPT sudo iptables -I INPUT -p tcp –dport 443 -m state –state NEW -m comment –comment “SM: Allow HTTP requests from Media Streamers” -j ACCEPT |
For more information about SmartMEDIA components and their functionality, see “Services and Components.”
The complete list of packages included in the SmartMEDIA is shown below:
- smartmedia-conveyor,
- smartmedia-transcoder,
- smartmedia-plgen,
- smartmedia-utils,
- smartmedia-nginx,
- smartmedia-cets,
- smartmedia-mpicker,
- smartmedia-pvr,
- smartmedia-rtspserver,
- smartmedia-hlsredirector.
Installing Third-Party Components for Transcoding Support
The transcoding functionality is performed by the SmartMEDIA Conveyor service or by the SmartTranscoder service using the Intel QuickSync Video (QSV)/Intel Media SDK libraries. To enable this functionality, install the following packages on the transcoding servers:
- Linux Kernel 3.10.0-514 (exact version is required),
- intel-linux-media — Intel Media SDK/Driver,
- kmod-ukmd — QSV Kernel Module (kernel version 3.10.0-514.el7.x86_64 required),
- libdrm — Direct Rendering Manager runtime library,
- libva — Video Acceleration API,
- intel-gpu-tools — a set of utilities for Intel graphics processors.
You can do this by running the single command:
yum install smartmedia-trc
As a result, all necessary packages will be installed.
If current kernel version differs from the required, reboot the OS after the command execution to switch to the installed kernel version. |
Installing the License
When you start SmartMEDIA services for the first time after installation, they will start and immediately stop with an error indicating the absence of a license file. Information about the error can be found in the services log files (in the /var/log/smartmedia directory):
[error] [default] Can not read license file “/opt/smartlabs/smartmedia/license.bin”, error = No such file or directory [info] [default] SmartMEDIA license verification failed, exit |
To add the license:
- Send the system UUID to the address support@smartlabs.tv. The UUID can be found in the log of any service after its launch. For example:
[2017-06-01 13: 15: 32.687637] [0x00007f0c77d7a777] [info] [default] System UUID: 00000000-0000-0000-0000-0011101DD111 |
- Obtain a license file from the SmartLabs support service.
- Copy the resulting file to /opt/smartlabs/smartmedia/license.bin on the server.
- Repeat the procedure for each server on which SmartMEDIA is installed.
First Start
After installing the license, start the SmartMEDIA services. For example, Conveyor and Playlist Generator are run as follows (other components are the same):
sudo systemctl reset-failed sudo systemctl start smcvr smplgen |
After the server reboot, SmartMEDIA services will be started automatically. |
To get started, edit the SmartMEDIA component configuration files (see the Configuration Reference sections). In particular, specify the addresses of the channel sources to be recorded on the video server in the configuration file of the SmartMEDIA Conveyor component:
/opt/smartlabs/smartmedia/conveyor/config/channels.json
Changes in the channels.json file are applied automatically, service restart is not required.
The process of updating channel settings can be controlled using the Conveyor’s log.