You can use the Wowza Streaming Engine™ media server software Java API to convert AMF metadata to ID3 (Interactive Data Display) metadata tags. This allows metadata ingested with an RTMP source stream or injected by Wowza Streaming Engine to be delivered to streams played using the Apple HLS format.
Note: The instructions in this article for converting AMF metadata to ID3 tags require the Wowza Streaming Engine Java API 4.5.0 or later.
About converting timed metadata for HLS
Producing interactive live streams for the purposes of synchronization, ad insertion, or content enrichment often requires the use of timed metadata. Wowza Streaming Engine ingests timed metadata over RTMP in Action Message Format (AMF), a binary format developed by Adobe for exchanging messages between servers. Wowza Streaming Engine can receive AMF metadata from any encoder that supports it, or you can inject AMF metadata directly into a stream as it enters the Wowza Streaming Engine server.
Either way, once the AMF metadata is in the stream, it must be converted to ID3 to be used when the stream is played with Apple HLS in a web browser or mobile app.
Before you start
Before you work with the examples in this article, it may be helpful to review the Use Wowza Streaming Engine Java modules page and Use custom modules section.
Convert AMF metadata to ID3 metadata in a live stream
Both examples in this section create custom modules that convert AMF-timed metadata from MPEG-TS or CMAF streams to ID3 tags for delivery with your live stream when it's played over HLS.
MPEG-TS live stream example
For this example, Wowza Streaming Engine is broadcasting an MPEG-TS live stream of a marathon. When a runner crosses the finish line, his or her name and the time that they cross should appear as an overlay in the player.
This example is a follow-up to the example for injecting AMF timed metadata into a stream with Wowza Streaming Engine in Inject timed metadata using a Wowza Streaming Engine HTTP provider. For a similar example of converting AMF timed metadata to emsg for MPEG-DASH playback, see Convert timed metadata from AMF to emsg using the Wowza Streaming Engine Java API.
In the following code, the Wowza Streaming Engine module handles data processing for the Cupertino live stream packetizer and listens for an AMF data event. It parses the AMF metadata and maps the event to an ID3 tag. Then, the ID3 tag is appended to the appropriate chunk in the HLS stream's .m3u8 HLS playlist.
Note: Refer to the code comments in the example and the following packages in the Wowza Streaming Engine Java API reference documentation for information about the classes used in this example and how to adapt the code to suit your needs:
- com.wowza.wms.amf
- com.wowza.wms.httpstreamer.cupertinostreaming.livestreampacketizer
- com.wowza.wms.media.mp3.model.idtags
Make sure you have configured Apple HLS packetization for your live application. Then, add the module to the Modules section of Application.xml:
<Module> <Name>ModuleMarathonLiveAMFToID3</Name> <Description>Module to convert AMF marathon data to ID3 in MPEG-TS live streams</Description> <Class>com.wowza.wms.plugin.ModuleCupertinoLiveMarathonAMFToID3</Class> </Module>
Then use this code for the module:
package com.wowza.wms.plugin;
import com.wowza.wms.amf.*;
import com.wowza.wms.application.*;
import com.wowza.wms.httpstreamer.cupertinostreaming.livestreampacketizer.*;
import com.wowza.wms.media.mp3.model.idtags.*;
import com.wowza.wms.module.*;
import com.wowza.wms.stream.livepacketizer.*;
public class ModuleCupertinoLiveMarathonAMFToID3 extends ModuleBase
{
class LiveStreamPacketizerDataHandler implements IHTTPStreamerCupertinoLivePacketizerDataHandler2
{
private static final String PAYLOAD_TYPE = "onRaceFinisher";
private LiveStreamPacketizerCupertino liveStreamPacketizer = null;
public LiveStreamPacketizerDataHandler(LiveStreamPacketizerCupertino liveStreamPacketizer)
{
this.liveStreamPacketizer = liveStreamPacketizer;
}
public void onFillChunkStart(LiveStreamPacketizerCupertinoChunk chunk)
{
// getLogger().info("ModuleCupertinoLiveMarathonAMFToID3.onFillChunkStart["+chunk.getRendition().toString()+":"+liveStreamPacketizer.getContextStr()+"]: chunkId:"+chunk.getChunkIndexForPlaylist());
//
// // This is how to add custom M3U tag to chunklist header
// // (Not needed by marathon example, so commented out)
// CupertinoUserManifestHeaders userManifestHeaders = liveStreamPacketizer.getUserManifestHeaders(chunk.getRendition());
// if (userManifestHeaders != null)
// userManifestHeaders.addHeader("MY-USER-HEADER-DATA", "LAST-CHUNK-TIME", (new Date()).toString());
// // This is how to add ID3 tag to start of chunk
// // (Not needed by marathon example, so commented out)
// ID3Frames id3Frames = liveStreamPacketizer.getID3FramesHeader();
// id3Frames.clear();
// ID3V2FrameTextInformation comment = new ID3V2FrameTextInformation(ID3V2FrameBase.TAG_TIT2);
// comment.setValue("LAST-CHUNK-TIME: "+(new Date()).toString());
// id3Frames.putFrame(comment);
}
public void onFillChunkEnd(LiveStreamPacketizerCupertinoChunk chunk, long timecode)
{
// This is where to add ID3 tag to end of chunk
}
public void onFillChunkMediaPacket(LiveStreamPacketizerCupertinoChunk chunk, CupertinoPacketHolder holder, AMFPacket packet)
{
}
public void onFillChunkDataPacket(LiveStreamPacketizerCupertinoChunk chunk, CupertinoPacketHolder holder, AMFPacket packet, ID3Frames id3Frames)
{
byte[] buffer = packet.getData();
if (buffer == null)
return;
if (packet.getSize() <= 2)
return;
int offset = 0;
if (buffer[0] == 0)
offset++;
AMFDataList amfList = new AMFDataList(buffer, offset, buffer.length-offset);
if (amfList.size() <= 1)
return;
if (amfList.get(0).getType() != AMFData.DATA_TYPE_STRING || amfList.get(1).getType() != AMFData.DATA_TYPE_OBJECT)
return;
String metaDataStr = amfList.getString(0);
AMFDataObj dataObj = amfList.getObject(1);
if (!metaDataStr.equalsIgnoreCase(PAYLOAD_TYPE))
return;
AMFData amfData = dataObj.get("payload");
if (!(amfData instanceof AMFDataItem))
return;
String payload = amfData.toString();
// Create ID3
ID3V2FrameTextInformationUserDefined id3 = new ID3V2FrameTextInformationUserDefined();
id3.setDescription(PAYLOAD_TYPE);
id3.setValue(payload);
id3Frames.putFrame(id3);
getLogger().info("ModuleCupertinoLiveMarathonAMFToID3.onFillChunkDataPacket["+chunk.getRendition().toString()+":"+liveStreamPacketizer.getContextStr()+"] Send payload: "+payload);
}
}
class LiveStreamPacketizerListener extends LiveStreamPacketizerActionNotifyBase
{
// When packetizer starts, register as a data handler
public void onLiveStreamPacketizerCreate(ILiveStreamPacketizer liveStreamPacketizer, String streamName)
{
if (liveStreamPacketizer instanceof LiveStreamPacketizerCupertino)
{
getLogger().info("ModuleCupertinoLiveOnTextToID3#MyLiveListener.onLiveStreamPacketizerCreate["+((LiveStreamPacketizerCupertino)liveStreamPacketizer).getContextStr()+"]");
((LiveStreamPacketizerCupertino)liveStreamPacketizer).setDataHandler(new LiveStreamPacketizerDataHandler((LiveStreamPacketizerCupertino)liveStreamPacketizer));
}
}
}
// When application starts, register to listen to Packetizer events
public void onAppStart(IApplicationInstance appInstance)
{
appInstance.addLiveStreamPacketizerListener(new LiveStreamPacketizerListener());
getLogger().info("ModuleCupertinoLiveOnTextToID3.onAppStart["+appInstance.getContextStr()+"]");
}
}
CMAF live stream example
The following example leverages the Wowza Streaming Engine Java API to define a module that processes a live CMAF stream, extracts metadata from the stream's AMF packets, and converts that AMF data to ID3 tags when the stream plays over HLS. The module generates Event Message (emsg) frames to contain the ID3 metadata. It then sends the data packet to the player using the emsg format.
Before you complete the steps in this section, ensure you're using Wowza Streaming Engine 4.8.26 and later. Create a live application in Wowza Streaming Engine Manager and enable CMAF packetization for your live application. Then, continue with these steps.
- By default, emsg data is added to the audio track. To add the emsg data to the video track, open your application's Application.xml file in a text editor. Add the cmafDataEventsTrackType property within the <Application>/<LiveStreamPacketizer>/<Properties> container element and set the value to video.
<Property> <Name>cmafDataEventsTrackType</Name> <Value>video</Value> <Type>String</Type> </Property>
- Then, add the module to the Modules section of the Application.xml file in [install-dir]/conf/[application name]/.
<Module> <Name>ModuleLiveAMFToID3Cmaf</Name> <Description>Module to convert AMF metadata to ID3 metadata in live CMAF HLS streams</Description> <Class>com.wowza.wms.plugin.ModuleCmafId3</Class> </Module>
- Use the following code to create the module. For help, see Use custom modules.
package com.wowza.example.cmaf_id3;
// Import classes and interfaces from the Wowza Streaming Engine API
import com.wowza.wms.amf.*;
import com.wowza.wms.application.IApplicationInstance;
import com.wowza.wms.httpstreamer.cmafstreaming.livestreampacketizer.LiveStreamPacketizerCmaf;
import com.wowza.wms.httpstreamer.model.LiveStreamPacketizerPacketHolder;
import com.wowza.wms.httpstreamer.mpegdashstreaming.file.*;
import com.wowza.wms.httpstreamer.mpegdashstreaming.livestreampacketizer.IHTTPStreamerMPEGDashLivePacketizerDataHandler;
import com.wowza.wms.logging.WMSLogger;
import com.wowza.wms.media.metadata.emsg.*;
import com.wowza.wms.media.mp3.model.idtags.*;
import com.wowza.wms.module.ModuleBase;
import com.wowza.wms.stream.livepacketizer.*;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.atomic.AtomicBoolean;
import static com.wowza.wms.amf.AMFData.*;
// Declare public ModuleCmafId3 class that extends ModuleBase
public class ModuleCmafId3 extends ModuleBase
{
private static final Class<ModuleCmafId3> CLASS = ModuleCmafId3.class;
private static final String CLASSNAME = CLASS.getSimpleName();
private IApplicationInstance appInstance;
private WMSLogger logger;
// Invoke on app start and register listener to live stream packetizer events
public void onAppStart(IApplicationInstance appInstance)
{
this.appInstance = appInstance;
logger = getLogger(CLASS, appInstance);
logger.info(String.format("%s.onAppStart [%s]", CLASSNAME, appInstance.getContextStr()));
// Set custom DataHandler for processing data packets
appInstance.addLiveStreamPacketizerListener(new LiveStreamPacketizerActionNotifyBase()
{
@Override
public void onLiveStreamPacketizerCreate(ILiveStreamPacketizer packetizer, String streamName)
{
if (packetizer instanceof LiveStreamPacketizerCmaf)
((LiveStreamPacketizerCmaf)packetizer).setDataHandler(new DataHandler(streamName));
}
});
}
// Declare class to handle data processing for CMAF live packetizers
private class DataHandler implements IHTTPStreamerMPEGDashLivePacketizerDataHandler
{
// https://aomediacodec.github.io/id3-emsg/
private static final String EVENT_SCHEME_URI = "https://aomedia.org/emsg/ID3";
// The value is something meaningful to whomever defined the schema of the SCHEME_URI, such as versioning
private static final String DEFAULT_VALUE = "";
private static final int DEFAULT_VERSION = 1;
private static final int DEFAULT_TIMESCALE = 1000;
private static final int DEFAULT_DURATION = 0;
private final AtomicBoolean hasRegisteredEventStream = new AtomicBoolean(false);
private final String context;
private int uniqueId = 0;
public DataHandler(String streamName)
{
this.context = appInstance.getContextStr() + "/" + streamName;
}
@Override
public void onFillSegmentStart(long startTimecode, long endTimecode, InbandEventStreams inbandEventStreams)
{
// Register our data event tracks
if (!hasRegisteredEventStream.getAndSet(true))
inbandEventStreams.registerEventStream(new InbandEventStream(EVENT_SCHEME_URI, DEFAULT_VALUE));
}
// Invoke to extract metadata from AMF packets, create ID3 framces from the metadata,
// serialize the ID3 frames into byte arrays, and construct emsg frames with ID3 data
@Override
public void onFillSegmentDataPacket(LiveStreamPacketizerPacketHolder holder, AMFPacket packet,
InbandEventStreams inbandEventStreams)
{
byte[] buffer = packet.getData();
if (buffer == null)
return;
if (packet.getSize() <= 2)
return;
int offset = 0;
if (buffer[0] == 0)
offset++;
AMFDataList amfList = new AMFDataList(buffer, offset, buffer.length - offset);
logger.debug(String.format("%s::DataHandler.onFillSegmentDataPacket [%s] data: %s", CLASSNAME, context, amfList));
if (amfList.size() <= 1)
return;
if (amfList.get(0).getType() != DATA_TYPE_STRING && amfList.get(1).getType() != DATA_TYPE_OBJECT)
return;
String handlerName = amfList.getString(0);
AMFDataObj data = amfList.getObject(1);
ID3Frames id3Frames = new ID3Frames();
ID3V2FrameTextInformationUserDefined id3 = new ID3V2FrameTextInformationUserDefined();
id3.setDescription(handlerName);
id3.setValue(data.toString());
id3Frames.putFrame(id3);
byte[] id3Bytes = id3Frames.serialize(true, false, ID3Frames.ID3HEADERFLAGS_DEFAULT);
// ID3 is serialized as a byte array but emsg requires a string value.
String id3String = new String(id3Bytes, StandardCharsets.UTF_8);
IEmsgFrame emsg = new EmsgBuilder(DEFAULT_VERSION, EVENT_SCHEME_URI, DEFAULT_VALUE)
.setId(uniqueId++)
.setTimescale(DEFAULT_TIMESCALE)
.setTime(packet.getAbsTimecode())
.setEventDuration(DEFAULT_DURATION)
.setMessage(id3String)
.build();
logger.debug(String.format("%s::DataHandler.onFillSegmentDataPacket [%s] emsg: [id: %d, time: %d, data: %s]",
CLASSNAME, context, emsg.getId(), emsg.getTime(), emsg.getMessage()));
EmsgFrames emsgFrames = inbandEventStreams.getEmsgFrames();
emsgFrames.addFrame(emsg);
}
}
}
Convert AMF metadata to ID3 metadata for a VOD asset
First, add the module to the Modules section of Application.xml:
<Module> <Name>ModuleMarathonVODAMFToID3</Name> <Description>Module to convert AMF marathon data to ID3 in VOD streams</Description> <Class>com.wowza.wms.plugin.ModuleCupertinoVODMarathonAMFToID3</Class> </Module>
Then use this code for the module:
package com.wowza.wms.plugin;
import com.wowza.wms.httpstreamer.model.*;
import com.wowza.wms.module.*;
import com.wowza.wms.amf.*;
import com.wowza.wms.application.*;
import com.wowza.wms.httpstreamer.cupertinostreaming.file.*;
import com.wowza.wms.httpstreamer.cupertinostreaming.httpstreamer.*;
import com.wowza.wms.httpstreamer.cupertinostreaming.livestreampacketizer.*;
import com.wowza.wms.media.mp3.model.idtags.*;
public class ModuleCupertinoVODMarathonAMFToID3 extends ModuleBase
{
private static final String PAYLOAD_TYPE = "onRaceFinisher";
class VODActionNotify implements IHTTPStreamerCupertinoVODActionNotify2
{
IApplicationInstance appInstance = null;
public VODActionNotify(IApplicationInstance appInstance)
{
this.appInstance = appInstance;
}
public void onCreate(IHTTPStreamerCupertinoIndex fileIndex, IHTTPStreamerApplicationContext appContext, IHTTPStreamerSession httpStreamerSession, String rawStreamName, String streamExt, String streamName)
{
}
public void onInit(IHTTPStreamerCupertinoIndex fileIndex, IHTTPStreamerApplicationContext appContext, IHTTPStreamerSession httpStreamerSession, String rawStreamName, String streamExt, String streamName)
{
}
public void onOpen(IHTTPStreamerCupertinoIndex fileIndex, IHTTPStreamerApplicationContext appContext, IHTTPStreamerSession httpStreamerSession, String rawStreamName, String streamExt, String streamName)
{
}
public void onIndex(IHTTPStreamerCupertinoIndex fileIndex, IHTTPStreamerApplicationContext appContext, IHTTPStreamerSession httpStreamerSession, String rawStreamName, String streamExt, String streamName)
{
}
public void onFillChunkStart(IHTTPStreamerCupertinoIndex fileIndex, IHTTPStreamerCupertinoIndexItem item, LiveStreamPacketizerCupertinoChunk chunk, boolean audioOnly)
{
}
public void onFillChunkEnd(IHTTPStreamerCupertinoIndex fileIndex, IHTTPStreamerCupertinoIndexItem item, LiveStreamPacketizerCupertinoChunk chunk, boolean audioOnly)
{
}
public void onDestroy(IHTTPStreamerCupertinoIndex fileIndex)
{
}
public void onFillChunkDataPacket(IHTTPStreamerCupertinoIndex fileIndex, IHTTPStreamerCupertinoIndexItem item, LiveStreamPacketizerCupertinoChunk chunk, boolean audioOnly, AMFPacket packet, ID3Frames id3Frames)
{
byte[] buffer = packet.getData();
if (buffer == null)
return;
if (packet.getSize() <= 2)
return;
int offset = 0;
if (buffer[0] == 0)
offset++;
AMFDataList amfList = new AMFDataList(buffer, offset, buffer.length-offset);
if (amfList.size() <= 1)
return;
if (amfList.get(0).getType() != AMFData.DATA_TYPE_STRING || amfList.get(1).getType() != AMFData.DATA_TYPE_OBJECT)
return;
String metaDataStr = amfList.getString(0);
AMFDataObj dataObj = amfList.getObject(1);
if (!metaDataStr.equalsIgnoreCase(PAYLOAD_TYPE))
return;
AMFData amfData = dataObj.get("payload");
if (!(amfData instanceof AMFDataItem))
return;
String payload = amfData.toString();
// Create ID3
ID3V2FrameTextInformationUserDefined id3 = new ID3V2FrameTextInformationUserDefined();
id3.setDescription(PAYLOAD_TYPE);
id3.setValue(payload);
id3Frames.putFrame(id3);
getLogger().info("ModuleCupertinoVODMarathonAMFToID3.onFillChunkDataPacket["+appInstance.getContextStr()+"] Send payload: "+payload);
}
}
// When application starts, register to listen to VOD actions
public void onAppStart(IApplicationInstance appInstance)
{
IHTTPStreamerApplicationContext appContext = appInstance.getHTTPStreamerApplicationContext("cupertinostreaming", true);
if (appContext == null)
return;
if (!(appContext instanceof HTTPStreamerApplicationContextCupertinoStreamer))
return;
HTTPStreamerApplicationContextCupertinoStreamer cupertinoAppContext = (HTTPStreamerApplicationContextCupertinoStreamer)appContext;
cupertinoAppContext.addVODActionListener(new VODActionNotify(appInstance));
getLogger().info("ModuleCupertinoVODMarathonAMFToID3.onAppStart["+appInstance.getContextStr()+"]");
}
}
Convert AMF metadata to ID3 metadata for an nDVR asset
First, add the module to the Modules section of Application.xml:
<Module> <Name>ModuleMarathonDvrAMFToID3</Name> <Description>Module to convert AMF marathon data to ID3 in DVR streams</Description> <Class>com.wowza.wms.plugin.ModuleCupertinoDVRMarathonAMFToID3</Class> </Module>
Then use this code for the module:
package com.wowza.wms.plugin;
import com.wowza.wms.amf.*;
import com.wowza.wms.application.*;
import com.wowza.wms.dvr.*;
import com.wowza.wms.httpstreamer.cupertinostreaming.livestreampacketizer.*;
import com.wowza.wms.media.mp3.model.idtags.*;
import com.wowza.wms.module.*;
import com.wowza.wms.stream.livedvr.*;
public class ModuleCupertinoDVRMarathonAMFToID3 extends ModuleBase
{
private static final String PAYLOAD_TYPE = "onRaceFinisher";
class LiveStreamPacketizerDataHandler implements IHTTPStreamerCupertinoLivePacketizerDataHandler
{
IApplicationInstance appInstance = null;
public LiveStreamPacketizerDataHandler(IApplicationInstance appInstance)
{
this.appInstance = appInstance;
}
public void onFillChunkDataPacket(CupertinoPacketHolder holder, AMFPacket packet, ID3Frames id3Frames)
{
byte[] buffer = packet.getData();
if (buffer == null)
return;
if (packet.getSize() <= 2)
return;
int offset = 0;
if (buffer[0] == 0)
offset++;
AMFDataList amfList = new AMFDataList(buffer, offset, buffer.length-offset);
if (amfList.size() <= 1)
return;
if (amfList.get(0).getType() != AMFData.DATA_TYPE_STRING || amfList.get(1).getType() != AMFData.DATA_TYPE_OBJECT)
return;
String metaDataStr = amfList.getString(0);
AMFDataObj dataObj = amfList.getObject(1);
if (!metaDataStr.equalsIgnoreCase(PAYLOAD_TYPE))
return;
AMFData amfData = dataObj.get("payload");
if (!(amfData instanceof AMFDataItem))
return;
String payload = amfData.toString();
// Create ID3
ID3V2FrameTextInformationUserDefined id3 = new ID3V2FrameTextInformationUserDefined();
id3.setDescription(PAYLOAD_TYPE);
id3.setValue(payload);
id3Frames.putFrame(id3);
getLogger().info("ModuleCupertinoVODMarathonAMFToID3.onFillChunkDataPacket["+appInstance.getContextStr()+"] Send payload: "+payload);
}
}
class LiveStreamPacketizerListener implements IDvrStreamManagerActionNotify2
{
IApplicationInstance appInstance = null;
public LiveStreamPacketizerListener(IApplicationInstance appInstance)
{
this.appInstance = appInstance;
}
@Override
public void onDvrStreamManagerCreate(IDvrStreamManager dvrMgr)
{
getLogger().info("ModuleCupertinoDVRMarathonAMFToID3#LiveStreamPacketizerListener.onDvrStreamManagerCreate["+dvrMgr+"]");
// When DVR Stream manager is created, register to listen to data handling events
dvrMgr.setDataHandler(new LiveStreamPacketizerDataHandler(appInstance));
}
@Override
public void onDvrStreamManagerInit(IDvrStreamManager dvrMgr)
{
}
@Override
public void onDvrStreamManagerDestroy(IDvrStreamManager dvrMgr)
{
}
@Override
public void onDvrStreamManagerInitStreamName(IDvrStreamManager dvrManager, String streamName)
{
}
}
// When application starts, register to listen to DVR Packetizer events
public void onAppStart(IApplicationInstance appInstance)
{
appInstance.addDvrStreamManagerListener(new LiveStreamPacketizerListener(appInstance));
getLogger().info("ModuleCupertinoDVRMarathonAMFToID3.onAppStart["+appInstance.getContextStr()+"]");
}
}
Test stream playback
You can test any of the live streams you've created and processed as a result of the instructions on this page. To do so, use the player in our Flowplayer sandbox environment, where you must enable the ID3 Metadata plugin. Make sure to modify the player configuration to add the event listener and console logging, as in this example:
const player = flowplayer("#player", {
src: "https://example.com/live/myStream/playlist.m3u8"
});
player.on(flowplayer.id3.events.ID3, async ({data})=> {
console.log(data);
});
