Streamer2
Table of Contents
Project Descriptions
Streamer2 is a Node.js(v4.4.7) app, developed on OS X El Capitan 10.11.5, that serves video streams to webpage. It recieves variant streams of different resolution types from Recorder and serves adaptive streaming to web app with HLS technique.
The goal of Streamer2 is to replace Streamer1 to enhance some aspects:
- Shorten boot time to 5 seconds or less.
- Lower request-response cycle to 100 milliseconds or less.
- Use JavaScript objects instead of in-memory database to store data.
- Add adaptive streaming control.
Public API
Put a video
PUT
/cameras/:cameraId/streams/:streamIdx/resolutions/:resolutionType
{
"resolution": {
"width": [integer],
"height": [integer]
},
"rateControl": {
"bitrate": [integer],
"frameRateLimit": [float]
},
"duration": [float],
"data": [array]
}
URL parameters:
-
cameraId
: ID of the camera, should consist of 24 alphanumeric characters -
streamIdx
: index of the stream, should be an integer -
resolutionType
: 'native' or 'vga' (if width*height is greater than 307200(640*480), please send videos of both resolution types; otherwise, send only 'native' resolution type)
JSON content:
-
width
: width of the video resolution -
height
: height of the video resolution -
bitrate
: video bitrate in Kbps that is used to determine bandwidth value in HLS playlists which controls adaptive streaming -
frameRateLimit
: video frame rate in fps that is used to determine stream info in HLS master playlist which describes maximum frame rate of the stream, specify 0 if there is no frame rate limit -
duration
: video length in seconds that is used to determine video duration in HLS playlists which also determines target duration.(MUST NOT be less than 1/30; also, rounded to the nearest integer, MUST be less than or equal to the target duration) -
data
: binary data of a .ts file in byte array of decimals or octets
If there is no corresponding master playlist, it will create one master playlist and one alternate playlist. Otherwise, it will update the corresponding alternate playlist.
Response if the playlist created:
HTTP/1.1 201 Created
or if the playlist updated:
HTTP/1.1 200 OK
or if the request URL contains invalid parameters:
HTTP/1.1 400 Bad Request
Example request:
PUT
/cameras/567da100f2f90000011fa0c5/streams/0/resolutions/native
{
"resolution": {
"width": 1280,
"height": 720
},
"rateControl": {
"bitrate": 192,
"frameRateLimit": 30.000
},
"duration": 10.0,
"data": [116, 102, 115, 116,...]
}
Get a video
GET
/cameras/:cameraId/streams/:streamIdx/resolutions/:resolutionType/:fileSequence.ts
URL parameters:
-
cameraId
: ID of the camera, should consist of 24 alphanumeric characters -
streamIdx
: index of the stream, should be an integer -
resolutionType
: 'native' or 'vga' -
fileSequence
: index of the file, should be an integer
Response if the viedo exists:
HTTP/1.1 200 OK
with a .ts file.
or if the request URL contains invalid parameters:
HTTP/1.1 400 Bad Request
or if the video does not exist:
HTTP/1.1 404 Not Found
Example request:
GET
/cameras/567da100f2f90000011fa0c5/streams/0/resolutions/native/0.ts
Get a master playlist
GET
/cameras/:cameraId/streams/:streamIdx/all.m3u8
URL parameters:
-
cameraId
: ID of the camera, should consist of 24 alphanumeric characters -
streamIdx
: index of the stream, should be an integer
Response if the playlist exists:
HTTP/1.1 200 OK
with all.m3u8 file, like:
#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=974098,CODECS="avc1.4d601f",RESOLUTION=1280x720,FRAME-RATE=15.000
/cameras/567da100f2f90000011fa0c5/streams/0/native.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=160176,CODECS="avc1.4d601f",RESOLUTION=640x480,FRAME-RATE=15.000
/cameras/567da100f2f90000011fa0c5/streams/0/vga.m3u8
(no FRAME-RATE tag if frame rate limit is 0)
or if the request URL contains invalid parameters:
HTTP/1.1 400 Bad Request
or if the video does not exist:
HTTP/1.1 404 Not Found
Example request:
GET
/cameras/567da100f2f90000011fa0c5/streams/0/all.m3u8
Get a playlist
GET
/cameras/:cameraId/streams/:streamIdx/:resolutionType.m3u8
URL parameters:
-
cameraId
: ID of the camera, should consist of 24 alphanumeric characters -
streamIdx
: index of the stream, should be an integer -
resolutionType
: 'native' or 'vga'
Response if the playlist exists:
HTTP/1.1 200 OK
with prog_index.m3u8 file, like:
#EXTM3U
#EXT-X-TARGETDURATION:10
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:10.0,
/cameras/567da100f2f90000011fa0c5/streams/0/resolutions/native/0.ts
#EXTINF:10.0,
/cameras/567da100f2f90000011fa0c5/streams/0/resolutions/native/1.ts
#EXTINF:10.0,
/cameras/567da100f2f90000011fa0c5/streams/0/resolutions/native/2.ts
#EXTINF:10.0,
/cameras/567da100f2f90000011fa0c5/streams/0/resolutions/native/3.ts
#EXTINF:10.0,
/cameras/567da100f2f90000011fa0c5/streams/0/resolutions/native/4.ts
or if the request URL contains invalid parameters:
HTTP/1.1 400 Bad Request
or if the video does not exist:
HTTP/1.1 404 Not Found
Example request:
GET
/cameras/567da100f2f90000011fa0c5/streams/0/native.m3u8
Get health status
GET
/health
Response:
HTTP/1.1 200 OK
with:
{
"streamIds": [array],
"cpuLoad": [float],
"mem": {
"load": [float],
"current": [integer],
"total": [integer]
}
}
JSON content:
-
streamIds
: an array of strings which consist of cameraId and streamIdx -
cpuLoad
: CPU usage in percentage -
load
: memory usage in percentage -
current
: current memory usage amount in MB -
total
: total memory size in MB
Heartbeat Mechanism
Register
POST
/streamers
{
"port": [integer],
"publicIp": [string]
}
JSON content:
-
port
: port number of the service of Streamer2 -
publicIp
: public IP address of Streamer2
Streamer2 registers to data API when it starts to run and will receive a streamer ID like 567da0e045800b0001142588
after registration.
Example request:
POST
/streamers
{
"port": 15925,
"publicIp": "123.123.123.123"
}
Report
PUT
/streamers/:streamerId
{
"streamIds": [array],
"cpuLoad": [float],
"mem": {
"load": [float],
"current": [integer],
"total": [integer]
}
}
URL parameters:
-
streamerId
: ID of the streamer recieved from Recorder after registration, should consist of 24 alphanumeric characters
JSON content:
-
streamIds
: an array of strings which consist of cameraId and streamIdx -
cpuLoad
: CPU usage in percentage -
load
: memory usage in percentage -
current
: current memory usage amount in MB -
total
: total memory size in MB
Streamer2 reports to data API at intervals determined by CLIO_HEARTBEAT_INTERVAL
, or whenever health information changes.
Example request:
PUT
/streamers/567da0e045800b0001142588
{
"streamIds": [
"567da100f2f90000011fa0c5/0",
"567da11313dc8d0001a53afc/0",
"567da12ef0be2400011a2843/1"
],
"cpuLoad": 5.5,
"mem": {
"load": 42.7,
"current": 922,
"total": 4096
}
}
Environment Variables
Required:
-
CLIO_DATA_API_URL
: the URL of data API (for example, 'http://data-api'), MUST be provided for Streamer2 to register to data API so that it can start running.
Optional:
-
CLIO_STREAMER_PLAYLIST_LENGTH
: number of video segments kept on one playlist -
CLIO_STREAMER_RETAINED_LENGTH
: number of video segments not kept on playlist but retained in memory per playlist -
CLIO_HEARTBEAT_INTERVAL
: intervals at which Streamer2 reports to data API -
CLIO_STREAMER_PORT
: port number of the service of Streamer2 -
CLIO_STREAMER_LOG_TRANSPORT
: the storage device for logs, can be 'console', 'file', or 'all' -
CLIO_STREAMER_LOG_PATH
: the filename of the logfile to write output to -
CLIO_STREAMER_LOG_LEVEL
: the maximum level of messages that should be logged, can be 'debug', 'info', or 'error'
Default values as follows:
export CLIO_STREAMER_PLAYLIST_LENGTH=3
export CLIO_STREAMER_RETAINED_LENGTH=6
export CLIO_HEARTBEAT_INTERVAL=10
export CLIO_STREAMER_PORT=15925
export CLIO_STREAMER_LOG_TRANSPORT=all
export CLIO_STREAMER_LOG_PATH=$PWD
export CLIO_STREAMER_LOG_LEVEL=info
Compilation Note:
During tests, load test script attempts to fire up instance on Google Compute Engine according to
you local gcloud installation and settings. Default project set by gcloud will be used and load test
instance will be launched in the zone according to your environment variable $ZONE
value.
Load test relies on ab
from apache2-utils
package to function.