Services and Components

Depending on services to be provided to subscribers, the SmartMEDIA solution may include different set of components. Below you can find a component interaction scheme, which illustrates the component location in the content processing workflow from the source TV streams and VoD files up to subscriber devices. Next you will find a short description of each component.

Conveyor

The SmartMEDIA Conveyor service (smcvr) is responsible for processing (transcoding, remultiplexing, segmenting), indexing and recording of LiveTV streams to the storage.

Technically, the nDVR and Unicast Live services do not differ. In both cases, the stream must be written to the storage, but in case of the Unicast Live, the record can be made to a temporary storage (for example, tmpfs) and deleted after a minute or two.

Content can be delivered to the SmartMEDIA Conveyor server within the MPEG2-TS container using one of the following protocols:

UDP over IP Multicast (without encapsulation in RTP),
HTTP Stream,
HLS (according to draft-pantos-http-live-streaming-07), without encryption.

SmartMEDIA Conveyor performs:

transcoding of incoming streams (if necessary),
remultiplexing,
media data encryption (if necessary),
recording to the storage,
index generation and writing indexes to the database (MongoDB).

Adaptive Streaming Support

To record several streams of the single channel which have different bit rates (a.k.a. Adaptive Streaming), SmartMEDIA Conveyor synchronizes them at the recording stage and create one index record for all of the chunks of different bitrates of the same time interval. Thus, all streams (audio, video and subtitles) must be synchronized in time (which means DTS discrepancy within the stream should be minimal) and have the same GOP structure of the video stream with synchronous keyframes. Otherwise, streams will not be synchronized and cannot be recorded.

Configuration and Management

There are two ways to manage SmartMEDIA Conveyor’s settings:

via the channel configuration file,
via the JSON-RPC API.

See “SmartMEDIA Configuration Reference” document for more details.

Playlist Generator

The SmartMEDIA Playlist Generator (smplgen) generates HLS playlists and DASH manifest files for recorded and indexed content as well as checks the availability of content for service requests from SmartMEDIA Redirector.

To retrieve playlists, the subscribers’ devices send requests to the web server (nginx web server is included in the standard package), which in turn requests a playlist/manifest from the Playlist Generator service using the FastCGI protocol. The Playlist Generator forms a playlist/manifest based on the index data stored in the MongoDB, and returns it to the device.

Functionality:

generation of m3u8 playlists for HLS protocol for the streams recorded in the MPEG2-TS container;
generation of playlists for protocols HLS (m3u8), DASH (MPD) and SmoothStreaming for the streams recorded in the MP4/ISOBMFF-fragmented container.

RTSPServer

The SmartMEDIA RTSPServer (rtspServer) service is designed to support devices which do not support the HLS or DASH protocol, but support RTSP. The main task of RTSPServer is to translate RTSP requests to HLS and to transfer media content to RTSP-compatible end-user devices.

It supports playback and forward/backward rewind (trickplay).

RTSPServer can also be used to build distributed content delivery solutions. In this case, the content delivery between different geographical sites can be done over HTTP with all it advantages, and RTSP will be used only for the “last mile”, to deliver content to the end-user devices from the EDGE nodes.

Redirector

The SmartMEDIA Redirector service (httpRedirector and rtspRedirector) allows you to define server groups (farms) and achieve uniform and flexible load balancing between servers or server groups, depending on client location and requested content. In particular, it:

provides failover to backup sources (servers/farms) when one or more sources fail;
monitors the health of servers and services;
checks the availability of requested content on servers;
provides caching of information about the availability of servers or content.

For a more detailed description of HTTPRedirector workflow, see «Load Balancing».

SmartPVR

The SmartPVR (smpvr) service generates, upon the request, a standalone record of the specified TV program from the records generated by the SmartMEDIA Conveyor service, based on the requested channel and the broadcast time (so-called Live-to-VoD). Creation and management of records are performed with JSON-RPC API.

SmartMPicker

SmartMEDIA SmartMPicker (smpicker) allows you to pick the working/best source of incoming stream and send it to a user or another service.

SmartMPicker can be used to ensure the fault tolerance of incoming traffic and automatically switch to the backup stream for the Conveyor service.

The source streams are defined in the SmartMPicker configuration file in priority order. If there are no data in the stream or it is ignored at the current time of the day, according to the configuration, SmartMPicker moves to the next stream from the list. If none of the streams match the conditions, a “stub” stream from the file can be generated.

The list of incoming streams maps to the address of the outgoing multicast stream, which will be delivered to the user or another service.

SmartCETS

The SmartCETS service encrypts MPEG2 TS streams in real time according to ISO / IEC 23001-9 (part: “Common encryption of MPEG-2 transport streams”). The encryption keys are requested over the Widevine Modular DRM / SmartLabs UDRM protocol (see the document “SmartMEDIA. Integration Guide”).

SmartMEDIA Transcoder

SmartMEDIA Transcoder service (smtrc) is a “satellite” service of the SmartMEDIA Conveyor. It receives a stream of media samples from the Conveyor over the TCP connection, transcodes the stream and sends the resulting stream (or multiple streams in case of ABR) back to the smcvr. It can be started on the same host where Conveyor runs as well as on the other, “remote” host and work over the TCP/IP network. In comparison with the ‘in-process’ transcoding it has the following advantages:

transcoders can be installed on dedicated hosts with the required hardware, such as QSV-enabled Intel CPUs or NVENC-enabled Nvidia GPUs. Conveyor service is still installed on the host, optimized for encryption and recording;
reducing the load on the svcvr service and offloading of the most CPU-intensive task leads to the more channels can be processed and recorded by the same Conveyor service, which simplifies the management;
transcoder service can be shared between multiple Conveyor services;
a single Conveyor service can use multiple network transcoders, even with different transcoding technologies and hardware platforms;
finally, transcoding process isolation leads to more reliable service and much more simple monitoring of this service.

Note, that the only tasks SmartTranscoder performs are decoding, processing (relampling, resizing etc.) and encoding with the required codec (h264/AVC, h265/HEVC). All other processing tasks, such as demultiplexing, synchronization, segmenting, encryption, indexing and multiplexing are performed by the Conveyor service.

SmartMEDIA Transcoder service uses proprietary protocol to exchange the data with smcvr and cannot receive streams directly.

SmartMEDIA Recryptor

The SmartMEDIA Recryptor (smrecrypt) service is designed to re-encrypt MP4 chunks prepared for adaptive streaming (HLS or DASH) from one encryption scheme to another “on the fly”, i.e. without saving the re-encrypted copy to the repository. This saves disk space by storing only one copy of encrypted content (for example, for Widevine DRM). At the same time, players using a different content protection system (for example, Apple FairPlay DRM) can also decrypt and play this content. Currently, reencryption from the cenc scheme (AES-CTR) to the cbcs scheme (AES-CBC with template encryption) is supported according to the standard ISO/IEC 23001-7 “Common encryption in ISO BMFF files”.

SmartMEDIA Recryptor operates as an HTTP proxy server, processing client requests for MP4 chunks. It requests chunks over HTTP from the origin server (for example, from S3 storage) or from the local file system, parses MP4, receives encryption keys from the SmartLabs UDRM server, decrypts media samples, re-encrypts with another encryption scheme, generates an output MP4 file and passes it to the client over HTTP. The re-encryption is performed at each request for each client. The service does not have internal caching of chunks, so it may be advisable to cache content using a caching proxy server, for example, NGINX. Also cryptographic algorithms used in decrypting and encrypting data can create an increased load on the CPU. So it is recommended to use processors with hardware encryption support, for example, Intel AES New Instructions (AES-NI).

The service receives encryption keys from the SmartLabs UDRM server using the Custom Extension UDRM Decryption API. Since the standard API of the cloud Widevine DRM does not support content decryption on the server side (but only encryption), the SmartMEDIA Recryptor service can only be used with the SmartLabs UDRM key server and cannot receive keys directly from the cloud Widevine DRM (unlike the SmartMEDIA Conveyor service). The received encryption keys are cached to reduce the number of requests and the load on the key server.

To re-encrypt MP4 chunks “on-the-fly”, they must be recorded appropriately by the SmartMEDIA Conveyor service. In particular, Key ID and PSSH must be written to each MP4 chunk to request keys. When re-encrypting, the structure of Subsamples (BytesOfClearData/BytesOfProtectedData) remains unchanged, so the original chunks must be encrypted with this in mind. This may require certain recording setup in the SmartMEDIA Conveyor service (for details, see the service documentation). Also, MP4 chunks recorded by a third-party media server, in some cases, cannot be re-encrypted “on-the-fly” using the SmartMEDIA Recryptor service.

HLS Playlist Manipulator

The HLS Playlist Manipulator allows OTT operators to insert / replace ad blocks in the original HLS stream containing SCTE-35 ad markers. Ad insertion decisions are made in interaction with the VAST server using the VAST 3.0 protocol.

The HLS Playlist Manipulator supports client players with the VAST protocol support and could be integrated with any OTT environment using the HLS protocol.

It also has a well-proven integration with the SmartLabs ecosystem:

VAST server — SmartTUBE ADS;
Client players — SmartTUBE Client apps for the large-screen and mobile platforms;
Content processing and streaming system — SmartMEDIA.

Features Implemented

Ads replacement is based on the pre-prepared sсhedule in CSV format.
The schedule links to the current airtime by the tag #EXT-X-PROGRAM-DATE-TIME in the origin playlist.
HTTP and HTTPS support.
Adaptive bitrate HLS support.

The deviation of the insertion moment from real time to 5-6 chunks is possible. For example, with a chunk length of 5 seconds, the deviation can be up to 25–30 seconds.

Operation Principle

OTT ad insertion architecture

The workflow of OTT ad replacement using HLS Playlist Manipulator are as follows:

The client player (e.g. SmartTUBE Client app’s player) starts to play a specific stream (Live or VOD) that has ad insertion opportunities marked in the origin playlists coming from 3rd party sources (e.g. SmartMEDIA Content Processor).
The client player sends a request URL to the HLS Playlist Manipulator to watch this stream. This stream request URL carries all the targeting parameters needed for the HLS Playlist Manipulator to use. For example, the request URL could look like this:

The HLS Playlist Manipulator checks:
1. Origin playlist from the source streaming server (or SmartMEDIA Playlist Generator) for the #EXT-X-PROGRAM-DATE-TIME tags and
2. Ad insertion schedule, uploaded to the database from CSV files.

If the origin playlist carries the #EXT-X-PROGRAM-DATE-TIME tags and there are ad insertion events in the DB, then the HLS Playlist Manipulator initiates the ad insertion workflow. If not, then nothing is done on the HLS Playlist Manipulator and it just passes the same playlist from the origin to the client player with no ad insertion.
In case there are ad availabilities, then the HLS Playlist Manipulator strips out the targeting parameters from the client request URL and uses these parameters to form a VAST request to the VAST server (e.g. SmartTUBE ADS). The VAST request is sent immediately. The VAST request URL could look like this:

The VAST server receives this request and immediately checks in the database it has for campaigns matching this targeting criteria. Once a suitable campaign and creatives are found, the server responds with a VAST response carrying information about the ad creatives to be inserted.
Upon receiving a response, the HLS Playlist Manipulator replaces the origin chunks with ad chunks from the ad creative/s and sends a “personalized” playlist to the client player for playback.

The HLS Playlist Manipulator workflow can be also represented by the following diagram:

HLS Playlist Manipulator workflow

Requirements, Limitations, and Recommendations

Chunk length of the ad creatives MUST be equal or greater than in the original stream.
Ads are inserted at the chunk boundaries. This means that if the ad start time is in the middle of the current chunk, the HLS Playlist Manipulator will insert the ad when this chunk ends and will not split the chunk.
The original playlists must contain the #EXT-X-PROGRAM-DATE-TIME tag.
Use a video player that correctly supports the #EXT-X-DISCONTINUITY tag according to the HLS specification.

CSV Schedule Format

The schedule for inserting ad blocks must meet the following conditions:

The CSV file format is only supported.
The schedule files are accessible via FTP or HTTP[S].
Each channel has a folder with a name corresponding to the channel ID on the VAST server.
Each folder contains files with the names of the following format:
TAYYMMDD.csv, where YY — year, MM — month, DD — day.
For example:
TA220801.csv
TA220802.csv
The ad insertion events are described in these files as follows:
ID;START;END

where ID — unique ID of an event, START — start time of the ad block, END — end time of the ad block.

For example:
1659368211;00:00:00;00:01:37
1659368212;00:05:00;00:06:23
1659368213;00:10:00;00:11:07

Utilities

vodconvert

The vodconvert utility is designed for converting VOD content to the required format and generating DASH, Smooth Streaming and HLS playlists, as well as content encryption.

The utility can receive several files with several tracks. Track selection and processing settings can be defined in command line arguments.

MPEG-DASH/Microsoft Smooth Streaming

vodconvert can accept TS, MP4 and MKV files. After processing:

each selected track will be saved to a separate MP4 file;
if encryption is required, the content will be pre-encrypted;
an MPD file (manifest) will be created.

HLS

vodconvert can accept TS files as a source content. After processing:

a playlist will be created for each source TS file, files won’t be fragmented;

or:

the source TS files will be divided into chunks of the specified duration, after which playlists will be generated.

scanTs

The scanTs utility analyzes MPEG2 TS files and displays a brief description of structure found: PAT, PMT, ECM, PCR, PES, PTS, IFrame, CC errors, packet format, PCR jumps.

cets

The cets utility is a simplified version of the SmartCETS service. It allows you to work with files more conveniently, but has some restrictions. Accepts all necessary parameters from the command line.

When reading from a file, stops as the end of the file is reached. The cets utility requests encryption keys only once — before starting encryption, and cannot perform keys rotation (i.e. encrypts the entire source with the single set of keys).

tsSnoop

The tsSnoop utility is intended for viewing the headers and structures of media containers. It outputs to the console the binary data of the source file in hex-format (optional) and a textual representation of the media data structures found (for example, TS packets, MP4 atoms, etc.). The following formats are supported: MP4, MKV, TS, H.262, H.264, H.265, MP3, DTS, AC3.