Learn how to programmatically convert Action Message Format (AMF) timed metadata to Event Message (emsg) timed metadata using the Wowza Streaming Engine™ media server software Java API. This allows metadata ingested with a live RTMP source stream or injected by Wowza Streaming Engine to be delivered with live streams played using the MPEG-DASH protocol.
Note:
- Wowza Streaming Engine 4.8.10 or later is required.
- If you'd like to convert AMF metadata to ID3 metadata in a CMAF live stream for delivery over HLS, you can use this custom module. This module wraps the ID3 tags inside an emsg, then uses the emsg format to deliver the data packet to the player.
About converting timed metadata for MPEG-DASH streaming
Producing interactive live streams often requires the use of timed metadata to include captions, advertisements, overlay images, and more. Wowza Streaming Engine can ingest metadata over RTMP in Action Message Format (AMF)—a binary format developed by Adobe for exchanging messages between servers—from any encoder that supports AMF metadata. In order for Wowza Streaming Engine to ingest AMF metadata, the metadata must be wrapped in a top-level AMF data object (AMFDataObj).
Wowza Streaming Engine can also inject AMF metadata directly into a stream as it enters the Wowza Streaming Engine server. To inject AMF metadata into a stream sent to Wowza Streaming Engine, see Inject timed metadata using a Wowza Streaming Engine HTTP provider.
Whether Wowza Streaming Engine ingests the metadata from an encoder or injects it into a stream, once the AMF metadata is in the stream, it must be converted to emsg metadata in order for it to be used when the stream is played with an MPEG-DASH client. An emsg contains timing information, a string payload, and an event scheme identifier that differentiates between different types of metadata events (for example, captions, slide transitions, etc.). The MPEG-DASH manifest identifies any event schemes with the InbandEventStream element and schemeIdUri and value attributes. The player then knows which metadata to retrieve from the stream.
Before you start
You should be aware of the following limitations:
- Wowza Streaming Engine currently does not support using emsg metadata with the following features:
- VOD or nDVR streams
- Stream targets (push publishing to CDN destinations) – To send a stream with emsg metadata to a CDN, the CDN must pull the stream from a Live HTTP Origin application in Wowza Streaming Engine.
You should also complete the following tasks:
- Enable MPEG-DASH or CMAF MPEG-DASH packetization for your live-streaming application:
Convert AMF metadata to emsg metadata in a live stream
In this example, Wowza Streaming Engine is broadcasting a live stream of a marathon. When a runner crosses the finish line, metadata for the runner's name and the time they crossed should appear as an overlay in the player.
This example is a follow-up to the example for injecting AMF metadata into a stream with Wowza Streaming Engine in nject timed metadata using a Wowza Streaming Engine HTTP provider. Using the following sample code, Wowza Streaming Engine will
- Listen for an AMF data event and then parse it
- Map the AMF data event to an emsg and then insert the emsg into an audio segment
- Append the emsg to the appropriate event scheme in the MPEG-DASH stream's manifest
The sample code uses a single event scheme and emsg version 0 but includes additional code for registering multiple event schemes and emsg version 1 commented out.
1. Create a custom Java module
- Create a custom Java module with the following example code by compiling it and packaging it into a .jar file in the [install-dir]/lib folder. This module is called during MPEG-DASH packetization and segment creation. See Use Wowza Streaming Engine Java modules for more information about extending Wowza Streaming Engine functionality with Java modules.
Note: Refer to the code comments in the example and the following packages in the Wowza Streaming Engine Java API reference documentation for information about the classes used in this example and how to adapt the code to suit your needs:
- com.wowza.wms.amf
- com.wowza.wms.httpstreamer.mpegdashstreaming.file
package com.example; import com.wowza.wms.amf.*; import com.wowza.wms.application.IApplicationInstance; import com.wowza.wms.httpstreamer.model.LiveStreamPacketizerPacketHolder; import com.wowza.wms.httpstreamer.mpegdashstreaming.file.*; import com.wowza.wms.httpstreamer.mpegdashstreaming.livestreampacketizer.*; import com.wowza.wms.media.metadata.emsg.*; import com.wowza.wms.module.ModuleBase; import com.wowza.wms.stream.livepacketizer.*; public class ModuleMPEGDashLiveMarathonAMFToEmsg extends ModuleBase { /* The scheme URI identifies a unique event scheme that differentiates between different types of metadata events (for example, captions, slide transitions, etc.). */ private static final String EVENT_SCHEME1_URI = "urn:com:example:raceevent"; /* The value is something meaningful to whomever defines the schema of the SCHEME_URI. It can also be an empty string. */ private static final String EVENT_SCHEME1_VALUE = "1"; class LiveStreamPacketizerDataHandler implements IHTTPStreamerMPEGDashLivePacketizerDataHandler { // Identifies the AMF metadata to convert to emsg. private static final String AMF_PAYLOAD_TYPE = "onRaceFinisher"; private LiveStreamPacketizerMPEGDash liveStreamPacketizer = null; private boolean hasRegisteredEventStreams = false; private int uniqueIDEventScheme1 = 0; private long segmentStart; public LiveStreamPacketizerDataHandler(LiveStreamPacketizerMPEGDash liveStreamPacketizer) { this.liveStreamPacketizer = liveStreamPacketizer; } @Override public void onFillSegmentStart(long startTimecode, long endTimecode, InbandEventStreams inbandEventStreams) { // Only do this once with the first segment's start. if (! this.hasRegisteredEventStreams) { this.hasRegisteredEventStreams = true; /* If you want to put emsgs in video segments rather than audio segments, uncomment the following line or use the mpegdashDataEventsTrackType property in your application's configuration. */ // inbandEventStreams.inAudio = false; /* Register one or more event schemes. Uncomment the second call to register two event schemes. */ inbandEventStreams.registerEventStream(new InbandEventStream(EVENT_SCHEME1_URI, EVENT_SCHEME1_VALUE)); // inbandEventStreams.registerEventStream(new InbandEventStream(EVENT_SCHEME2_URI, EVENT_SCHEME2_VALUE)); } this.segmentStart = startTimecode; } @Override public void onFillSegmentDataPacket(LiveStreamPacketizerPacketHolder holder, AMFPacket packet, InbandEventStreams inbandEventStreams) { String payload = extractPayload(packet); if (payload == null) return; // Extract AMF metadata strings and create emsgs using emsg version 0. int id = uniqueIDEventScheme1++; // A unique ID for each emsg. Each emsg per event scheme needs a unique ID. long time = packet.getAbsTimecode(); int delta = Math.max(0, (int) (time - this.segmentStart)); int duration = 0; // The emsg has no duration. The value of duration is in the specified timescale. int timescale = 1000; // The timescale is in milliseconds (1000 ticks per second). IEmsgFrame emsg = this.createEmsgV0(EVENT_SCHEME1_URI, EVENT_SCHEME1_VALUE, id, timescale, delta, duration, payload); // To create emsgs using emsg version 1, use the following call instead. // IEmsgFrame emsg = this.createEmsgV1(EVENT_SCHEME1_URI, EVENT_SCHEME1_VALUE, id, timescale, time, duration, payload); // // Add emsgs to segments. // EmsgFrames emsgFrames = inbandEventStreams.getEmsgFrames(); emsgFrames.addFrame(emsg); } @Override public void onFillSegmentEnd(long endTimecodeVideo, long endTimecodeAudio, InbandEventStreams inbandEventStreams) { } @Override public void onFillSegmentMediaPacket(LiveStreamPacketizerPacketHolder holder, AMFPacket packet) { } protected String extractPayload(AMFPacket packet) { byte[] buffer = packet.getData(); if (buffer == null) return null; if (packet.getSize() <= 2) return null; int offset = 0; if (buffer[0] == 0) offset++; AMFDataList amfList = new AMFDataList(buffer, offset, buffer.length-offset); if (amfList.size() <= 1) return null; if (amfList.get(0).getType() != AMFData.DATA_TYPE_STRING || amfList.get(1).getType() != AMFData.DATA_TYPE_OBJECT) return null; String metaDataStr = amfList.getString(0); AMFDataObj dataObj = amfList.getObject(1); if (! metaDataStr.equalsIgnoreCase(AMF_PAYLOAD_TYPE)) return null; AMFData amfData = dataObj.get("payload"); if (!(amfData instanceof AMFDataItem)) return null; return amfData.toString(); } private IEmsgFrame createEmsgV0(String schemeUri, String schemeValue, int id, int timescale, long delta, int duration, String data) { // Convert from milliseconds to timescale. delta = ((int)convertMillisToTimescale(timescale, delta)); duration = ((int)convertMillisToTimescale(timescale, duration)); // Wowza Streaming Engine uses the EmsgBuilder to create an emsg using emsg version 0. IEmsgFrame emsg = new EmsgBuilder(0, schemeUri, schemeValue) .setId(id) .setTimescale(timescale) .setTime(delta) .setEventDuration(duration) .setMessage(data) .build(); return emsg; } // To create emsgs using emsg version 1, use the following call instead. /* private IEmsgFrame createEmsgV1(String schemeUri, String schemeValue, int id, int timescale, long time, int duration, String data) { // Convert from milliseconds to timescale. time = ((int)convertMillisToTimescale(timescale, time)); duration = ((int)convertMillisToTimescale(timescale, duration)); // Wowza Streaming Engine uses the EmsgBuilder to create an emsg using emsg version 1. IEmsgFrame emsg = new EmsgBuilder(1, schemeUri, schemeValue) .setId(id) .setTimescale(timescale) .setTime(time) .setEventDuration(duration) .setMessage(data) .build(); return emsg; } */ private long convertMillisToTimescale(long timescale, long millis) { return timescale*millis/1000; } } class LiveStreamPacketizerListener extends LiveStreamPacketizerActionNotifyBase { // When the packetizer starts, register data handler for MPEG-DASH packetizer. public void onLiveStreamPacketizerCreate(ILiveStreamPacketizer liveStreamPacketizer, String streamName) { if (liveStreamPacketizer instanceof LiveStreamPacketizerMPEGDash) { getLogger().info("ModuleMPEGDashLiveMarathonAMFToEmsg.onLiveStreamPacketizerCreate["+((LiveStreamPacketizerMPEGDash)liveStreamPacketizer).getContextStr()+"]"); ((LiveStreamPacketizerMPEGDash)liveStreamPacketizer).setDataHandler(new LiveStreamPacketizerDataHandler((LiveStreamPacketizerMPEGDash)liveStreamPacketizer)); } } } // When application starts, register to listen to packetizer events. public void onAppStart(IApplicationInstance appInstance) { appInstance.addLiveStreamPacketizerListener(new LiveStreamPacketizerListener()); getLogger().info("ModuleMPEGDashLiveMarathonAMFToEmsg.onAppStart["+appInstance.getContextStr()+"]"); } }
2. Add the Java module to your application
- Open your application's Application.xml file in a text editor.
- Add the ModuleMarathonLiveAMFToEmsg module within the <Application>/<Modules> container element.
<Module> <Name>ModuleMarathonLiveAMFToEmsg</Name> <Description>Module to convert AMF marathon metadata to emsg metadata in live streams</Description> <Class>com.wowza.wms.plugin.ModuleMPEGDashLiveMarathonAMFToEmsg</Class> </Module>
- Save your changes.
3. Control where to add emsgs (optional)
To control whether emsgs are added to video segments or audio segments for streams sent to your live application, complete the following steps.
Note: The sample module adds emsgs to audio segments unless you uncomment this line: inbandEventStreams.inAudio = false;. If this line is uncommented, it overrides the value of the mpegdashDataEventsTrackType property.
- Open your application's Application.xml file in a text editor.
- Add the mpegdashDataEventsTrackType property within the <Application>/<LiveStreamPacketizer>/<Properties> container element and set the value to audio or video.
<Property> <Name>mpegdashDataEventsTrackType</Name> <Value>video</Value> <Type>String</Type> </Property>
- Save your changes.
4. Verify the conversion to emsg
After sending a stream to your live application with AMF metadata added by an encoder or injected by Wowza Streaming Engine, you can confirm the module converts the AMF metadata to emsgs by requesting the MPEG-DASH manifest with a curl command.
Sample request
curl http://[address]:1935/[application-name]/[stream-name]/manifest.mpd
Sample response
This manifest returned should include:
- The InbandEventStream element with schemeIdUri and value attributes in either the audio or video adaptation set, depending on if the emsgs were inserted into the audio or video segments.
- Multiple InbandEventStream elements for each event scheme if you registered multiple event schemes.
The following manifest excerpt includes the event scheme registered in the sample code identified by the InbandEventStream element within the audio adaptation set.
... <AdaptationSet id="1" group="2" mimeType="audio/mp4" lang="eng" segmentAlignment="true" startWithSAP="1" subsegmentAlignment="true" subsegmentStartsWithSAP="1"> <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/> <InbandEventStream schemeIdUri="urn:com:example:raceevent" value="1"/> ...
To test playback, use an MPEG-DASH player that supports emsgs such as Bitmovin or Dash.js. When configuring the player for emsgs, you need to specify any event schemes' URIs.
