Convoy integration

Integration with Convoy

Currently Convoy, a product that managed the streamers and origins, is using the tool named ew-recorder for recording and uploading VoD. That tool was written in python and not so good in performance and resource usage. ew-vodingest tends to resolve these problems.

This document’s purpose is to make to integrating process between ew-vodingest and Convoy becomes clear.

For ew-vodingest-1.12.0 and older

How to install ew-vodingest in Convoy?

From Convoy version 2.30, it has already been installed along when installing Convoy.

Installing or upgrading ew-vodingest on Convoy by:

dnf install -y <path to ew-vodingest RPM>

Integrating for recording

For using ew-vodingest for recording on Convoy, ingest script for recorder must be changed to ew-vodingest

...
ccmi.webtv_recorders {
  liveingest1 {
    host = 1.2.3.4
    port = 8090
    live_buffer = 600
    storage = storage1
    ingest_script = /usr/bin/ew-vodingest
    channel_groups = { swlive }
  }
}
...

Then, recording will be handled by ew-vodingest.

DASH thumbnails for recordings

To generate DASH thumbnails, an option must be added when executing ew-vodingest. Unfortunately, this option cannot be added directly into ingest_script configuration above, so we should use this script:

#!/bin/bash
set -e
export PATH="$PATH:/usr/bin"
/usr/bin/ew-vodingest --thumbnails "$@"

The change in PATH variable above means to allow ew-vodingest to search for ffmpeg command. /usr/bin can be changed to wherever ffmpeg is placed.

Put this script into any bash script file, for example /usr/bin/ew-custom-recorder, and the configuration will look like this

...
ccmi.webtv_recorders {
  liveingest1 {
    host = 1.2.3.4
    port = 8090
    live_buffer = 600
    storage = storage1
    ingest_script = /usr/bin/ew-custom-recorder
    channel_groups = { swlive }
  }
}
...

Notes:: The above script is just a simple example and can be modified for more purposes.

Integrating for uploading

From Convoy 2.30, you can change the configuration of Convoy to change the uploading script:

upload.repackager_bin = /usr/bin/ew-vodingest

For older Convoy, you should change the symlink /usr/bin/recorder, and make it point to ew-vodingest.

ln -sf /usr/bin/ew-vodingest /usr/bin/recorder
DASH thumbnails for uploadings

Because ew-check-variant-aligment, a legacy tool of ew-recorder, are still being used for checking the asset before uploading it, a similar solution with recording above cannot be used since thumbnails tracks are not accepted by that tool.

Thus, thumbnails generating has to be executed after the checking step, and that can be achieved by using this script:

#!/bin/bash
set -e
/usr/lib64/ew-repackaging-recorder/bin/ew-check-variant-alignment "$@"
/usr/bin/ew-thumbnails-generator -i "$2"

Put above script into a file, for example /usr/bin/ew-custom-checker, and change the symlink:

ln -sf /usr/bin/ew-custom-checker /usr/bin/ew-check-variant-alignment

Notes::

  • The above script is just a simple example and can be modified for more purposes.

  • ew-check-variant-aligment are going to be replaced in the future.

For ew-vodingest-1.14.0 and newer

How to install ew-vodingest in Convoy?

Because from 1.14.0, ew-vodingest is intended to replace completely ew-recorder, some conflict errors will occur when installing ew-vodingest with bare yum or dnf, these command will help out:

dnf install -y --allowerasing <path to ew-vodingest RPM>

To rollback to ew-repackaging-recorder:

rpm -e --nodeps ew-vodingest
dnf install -y <path to ew-repackaging-recorder RPM>

IMPORTANT thing to add to Convoy upgrading and downgrading steps: Because the constrain in Convoy’s RPMs, conflict errors will occur when upgrading or downgrading the version older than Convoy-3.4.0 if ew-vodingest-1.14.0 or newer has been already installed on Convoy machines, so, please rollback to ew-repackaging-recorder before performing upgrading or downgrading, then you can reinstall ew-vodingest afterward.

After installing ew-vodingest, it can be upgraded or downgraded by:

yum install -y <path to ew-vodingest RPM>

Integrating for recording

DASH thumbnails for recordings

Change the ingest_script of each recorders to

...
ccmi.webtv_recorders {
  liveingest1 {
    host = 1.2.3.4
    port = 8090
    live_buffer = 600
    storage = storage1
    ingest_script = /usr/bin/ew-recorder-with-thumbnails
    channel_groups = { swlive }
  }
}
...

Notes:: The above script is just a simple sample and can be modified for more purposes.

Collecting input files on recording failure

When recording fails and --logfile is explicitly defined, which is the default call at Convoy, the relevant input files are collected in same folder as --logfile. For example, log file is at /tmp/XXX then collected files is at /tmp/XXX-input-files.

Those files are also collected to --output directory if --leavepartial is defined.

Integrating for uploading

From Convoy 2.30, you can change the configuration of Convoy to change the uploading script:

upload.repackager_bin = /usr/bin/ew-vodingest

For older Convoy, you should change the symlink /usr/bin/recorder, and make it point to ew-vodingest.

ln -sf /usr/bin/ew-vodingest /usr/bin/recorder
DASH thumbnails for uploadings

From Convoy 2.30, this configuration can be set:

upload.repackager_bin = /usr/bin/ew-recorder-with-thumbnails

With the older version:

ln -sf /usr/bin/ew-recorder-with-thumbnails /usr/bin/recorder