Scale video-on-demand streaming with Wowza Streaming Engine Media Cache

Media Cache is a read-through caching mechanism built-in with Wowza Streaming Engine™ media server software that increases the scalability of video-on-demand (VOD) streaming. This article provides information about configuring the Media Cache feature in Wowza Streaming Engine Manager in production environments. It describes the default configuration on Wowza Streaming Engine and how to change where content is stored and retrieved when using Media Cache.

Before you start

---

You should complete the following tasks:

• Create a VOD Edge application according to Create applications.

Configure Media Cache in Wowza Streaming Engine Manager

---

Wowza Streaming Engine automatically enables Media Cache when it starts; however, additional configuration is required in a production environment. 

In Wowza Streaming Engine Manager, click the Server tab at the top of the manager, and then select Media Cache in the contents panel. The Media Cache page shows the default status as Enabled.

Disable Media Cache

Leaving Media Cache enabled has no adverse effect on performance and we recommend that you leave it enabled unless you have a specific reason to turn it off.

If you decide to disable Media Cache, click Disable Media Cache. You will see the following notification:

Media Cache is disabled but before your changes can take effect, you must restart the server by clicking Restart Now. It may take several seconds for the server to restart, but Media Cache will continue to show that the restart is required until this process is complete.

Enable Media Cache

You can enable Media Cache functionality by clicking Enable Media Cache. If you decide to re-enable Media Cache, the following notification is displayed.

Media Cache is now enabled. However, before your changes can take effect, you must restart Wowza Streaming Engine by clicking Restart Now. It may take several seconds for Wowza Streaming Engine to restart, but Media Cache will continue to show that the restart is required until this process is complete.

Configure Media Cache stores

---

Stores define where on demand file sources are cached by Wowza Streaming Engine when requested by clients. The default settings for Media Cache define a store location in a mediacache folder in the Wowza Streaming Engine installation directory ([install-dir]). This folder has a maximum size of 10 GB, a maximum write rate of 16 MB per second, and a write rate bucket size of 64 MB. These default settings are adequate for testing and for very low utilization; however, a production environment may require more space and resources.

Note: We recommend that you do not change the default store unless you are an advanced user and have a specific need to do so. You can, however, create, edit, and delete other stores.

To see these settings, click the Stores tab, and then click the Edit (pencil) icon for the default store.

The Path setting allows the use of Wowza Streaming Engine variables. For example, the variable defined as ${com.wowza.wms.context.ServerConfigHome}/mediacache provides a folder named mediacache in the server configuration location. You can change this location and use the standard set of variables that are supported. If the folder you define doesn't exist, Wowza Streaming Engine will create it if you have the necessary permissions.

Add a Media Cache store

You can add more than one store for use with Media Cache. In most cases, this isn't required. However, if you have multiple disks and want to use them for cache storage, you must add a new store for each location.

1. Click the Stores tab, and then click Add Media Cache Store.

2. On the Basic tab, enter the following values:
    
   • Store Name – The name of the Media Cache store, which must be unique and can’t contain less-than (<), greater-than (>), colon (:), quotation (' and "), forward slash (/), backslash (\), pipe (|), question mark (?), asterisk (*), double-dot (..), and tilde (~) characters.
   • Description – An optional text description of the Media Cache store.
   • Path – The path to the storage location where files are cached. The path value must be unique and can't contain less-than (<), greater-than (>), quotation (' and "), backslash (\), pipe (|), question mark (?), and asterisk (*) characters. Always use forward slashes when defining paths, for example, ${com.wowza.wms.context.ServerConfigHome}/mediacache. Cached files are stored in a two-tier directory structure.
   • Maximum Size – (gigabytes) The maximum amount of data that is stored in this cache store. A general recommendation for determining the size of the store is to use the following equation:

     (average file size x number of unique streams played in the maximum time to live period) + 10–20%

     For example, if your average file size is 1 GB and you have 100 unique files played within the maximum time to live period, the maximum size of the store should be 110–120 GB. Logs show the number of unique files played within the maximum time to live period.
   • Write Rate – (megabytes per second) The maximum rate that content can be written to the cache store. Throttling the write rate helps control the flow of content that enters the cache so that it doesn't overwhelm the file system, impeding the flow of content that's being served out of the cache.
   • Write Rate Max Bucket Size – (megabytes) The maximum amount of content that is cached before the write rate is throttled. This value should be set to six times the write rate.
3. Click Add.

You can see the same information as before about the store; however, now you should also see the Delete (trash can) icon, providing you the option to delete the store.

Edit or delete a Media Cache store

• To edit a Media Cache store, click the Edit icon for the store.
• To delete a Media Cache store, click the Delete icon for the store. You can delete any Media Cache store except the default store. To completely remove the store, restart Wowza Streaming Engine as prompted. The store remains in use until the server is restarted.

Configure Media Cache sources

---

Sources are the locations from which Wowza Streaming Engine retrieves the content to be cached. Media Cache can retrieve content from three different types of sources, providing very flexible deployment options. The sources can be any of the following:

• File - Content is retrieved via standard file reads from network-attached file systems, including network file systems and any device that's recognized as a disk by the operating system.
• HTTP - Content is retrieved via standard HTTP requests from HTTP-based servers that support HTTP/1.1 range requests. The HTTP connection can be encrypted using Secure Sockets Layer (SSL) certificates from a certificate authority.
• Amazon S3 HTTP - Content is retrieved from an Amazon S3 bucket via HTTP requests that conform to Amazon S3 specifications. The HTTP connection can be encrypted using SSL certificates from a certificate authority.
• Microsoft Azure - Content is retrieved from a Microsoft Azure Blob storage account.
• Google Cloud Storage - Content is retrieved from a Google Cloud Storage Bucket connected to a Google service account.

Click Sources to see a list of your sources, as shown below:

The default dvrorigin source is already defined. This Media Cache source can't be removed as it allows interaction with Wowza nDVR. This entry can be ignored for the purposes of this tutorial.

Click Add Media Cache Source, and you'll be presented with the following options:

The Source Name must be a unique name across all sources. The optional Description can include as much or as little detail as you want about the source. Then, select a Source Type as the input fields available for the source will change when selected.

Add a File source

A Media Cache File source reads content from a file location. This source type is designed to allow retrieval from slower disk sources such as single large disks with lower RPMs, network attached storage (NAS), or even NFS storage locations. The options required are shown below:

• Prefix – The first part of the URL that clients use to access content from this source. The prefix must be unique across all sources and end with a forward slash (/). A default of path1/ is provided in the Prefix field.
• Base Path – The location where the content can be found. The base path along with the prefix controls how content is mapped back to a source configuration.
• Validate Configuration – The full address to the location that Media Cache uses to find this source. This field updates automatically as you specify the prefix and base path.
• Minimum Time To Live – Minimum time that an item will remain in the cache when it hasn't been accessed, and space is needed. The default setting is 14400000 milliseconds or four hours. We recommend setting this as low as possible, to a few seconds. Lower values make it easier to remove items when the cache is full or to add new items to the cache when needed.
• Maximum Time To Live – Maximum time that an item will remain in the cache when it hasn't been accessed regardless of space requirements. We recommend setting this to equal to the average duration of your content. The time-to-live counter starts when the last client to view the content stops viewing the content. If no other viewers view this content within the time-to-live window, the content is purged from the cache.

Add an HTTP source

A Media Cache HTTP source retrieves content from an HTTP 1.1-compliant web server. This source type is designed to allow retrieval from web servers to help you manage your content outside of Wowza Streaming Engine. No authentication is performed when using this method to make HTTP requests. Both HTTP and HTTPS (SSL) requests are supported.

• Prefix – The first part of the URL that clients use to access content from this source. The prefix must be unique across all sources and end with a forward slash (/). A default of http/ is provided in the Prefix field.
• Base Path – The location where the content can be found. The base path along with the prefix controls how content is mapped back to a source configuration. For HTTP sources, this is an HTTP or HTTPS URL.
• Validate Configuration – The full address to the location that Media Cache uses to find this source. This field updates automatically as you specify the prefix and base path.
• Minimum Time To Live – Minimum time that an item will remain in the cache when it hasn't been accessed, and space is needed. The default setting is 14400000 milliseconds or four hours. We recommend setting this as low as possible, to a few seconds. Lower values make it easier to remove items when the cache is full or to add new items to the cache when needed.
• Maximum Time To Live – Maximum time that an item will remain in the cache when it hasn't been accessed regardless of space requirements. We recommend setting this to equal to the average duration of your content. The time-to-live counter starts when the last client to view the content stops viewing the content. If no other viewers view this content within the time-to-live window, the content is purged from the cache.

Add an Amazon S3 HTTP source

A Media Cache AmazonS3 HTTP source retrieves content from an HTTP 1.1 compliant web server. This source type is designed to allow retrieval from web servers that use S3-style authentication so you can manage your content outside of Wowza Streaming Engine. Both HTTP and HTTPS (SSL) requests are supported.


 

• Prefix – The first part of the URL that clients use to access content from this source. The prefix must be unique across all sources and end with a forward slash (/). A default of amazons3/ is provided in the Prefix field.
• Base Path – The location where the content can be found. The base path along with the prefix controls how content is mapped back to a source configuration. For Amazon S3 sources, this is an HTTP or HTTPS URL, and the default value is http://s3.amazonaws.com/.
• AWS Access Key ID and AWS Secret Access Key – Enter your Amazon S3 key ID and secret access key to enable Media Cache to sign HTTP requests so that they're authenticated for access to protected content. These values can't contain spaces.
• Use Amazon S3 bucket name in domain – Select this option to include the bucket name in the domain name. Otherwise, the bucket name is included in the URL path.
• Validate Configuration – The full address to the location that Media Cache uses to find this source. This field updates automatically as you specify the prefix and base path.
• Minimum Time To Live – Minimum time that an item will remain in the cache when it hasn't been accessed, and space is needed. The default setting is 14400000 milliseconds or four hours. We recommend setting this as low as possible, to a few seconds. Lower values make it easier to remove items when the cache is full or to add new items to the cache when needed.
• Maximum Time To Live – Maximum time that an item will remain in the cache when it hasn't been accessed regardless of space requirements. We recommend setting this to equal to the average duration of your content. The time-to-live counter starts when the last client to view the content stops viewing the content. If no other viewers view this content within the time-to-live window, the content is purged from the cache.

Note: Some Amazon buckets are only available in specific regions or support a specific authentication version. Refer Amazon's S3 documentation to determine if you need to insert the region into the bucket name or if you need to specify the signature version for Amazon S3 bucket authentication. If you need to do either, refer to Configure source properties for Wowza Streaming Engine Media Cache to learn how to use the Amazon S3 custom properties awsSigningVersion and awsDefaultRegion. Only advanced users should change properties.

Add a Microsoft Azure source

A Media Cache Azure source retrieves content from an Azure Blob storage account. This source type is designed to allow retrieval from an Azure Blob account using the API key and container name so you can manage your content outside of Wowza Streaming Engine.

• Prefix – The first part of the URL that clients use to access content from this source. The prefix must be unique across all sources and end with a forward slash (/). A default of http/ is provided in the Prefix field.
• Blob Storage Account Name – Your Azure Blob account name. This value can't contain spaces.
• Blob Storage Container Name – The name of a container in your Azure Blob storage. A Media Cache Source can point to only one container in Azure Blob storage.
• Blob Storage Account Access Key – The primary or secondary access key for the Azure Blob account.
• Minimum Time To Live – Minimum time that an item will remain in the cache when it hasn't been accessed, and space is needed. The default setting is 14400000 milliseconds or four hours. We recommend setting this as low as possible, to a few seconds. Lower values make it easier to remove items when the cache is full or to add new items to the cache when needed.
• Maximum Time To Live – Maximum time that an item will remain in the cache when it hasn't been accessed regardless of space requirements. We recommend setting this to equal to the average duration of your content. The time-to-live counter starts when the last client to view the content stops viewing the content. If no other viewers view this content within the time-to-live window, the content is purged from the cache.

Add a Google Cloud Storage source

A Media Cache GoogleCloudStorage source retrieves content from a Google Cloud Storage Bucket. This source type is designed to allow retrieval from a Google Cloud service account using either JSON or P12 private key certification (JSON is recommended) so you can manage your content outside of Wowza Streaming Engine.

Note: You can create and access Google Cloud service accounts through the Google Developers Console. For more information about Google Cloud service accounts, see the Google Cloud Storage Authentication documentation.

• Prefix – The first part of the URL that clients use to access content from this source. The prefix must be unique across all sources and end with a forward slash (/). A default of http/ is provided in the Prefix field.
• Google Service Account Email Address – The email address (or account ID) used with your service account.
• Google Private Key Type – The type of private key file certificate that you selected and downloaded when creating your service account (JSON or P12).
• Minimum Time To Live – Minimum time that an item will remain in the cache when it hasn't been accessed, and space is needed. The default setting is 14400000 milliseconds or four hours. We recommend setting this as low as possible, to a few seconds. Lower values make it easier to remove items when the cache is full or to add new items to the cache when needed.
• Maximum Time To Live – Maximum time that an item will remain in the cache when it hasn't been accessed regardless of space requirements. We recommend setting this to equal to the average duration of your content. The time-to-live counter starts when the last client to view the content stops viewing the content. If no other viewers view this content within the time-to-live window, the content is purged from the cache.

If you're using a JSON private key file for certification, select JSON as the Google Private Key Type, open the downloaded JSON private key file in a text editor, and then copy and paste the file contents into the Google JSON Private Key box.

If you're using a P12 private key file for certification, copy the downloaded P12 private key file to your Wowza Streaming Engine [install-dir]/conf directory. Then select P12 as the Google Private Key Type, enter the name of the file (including the file extension) in Google P12 Private Key File Name, and enter the password to access the private key Google P12 Private Key Password.

Edit or delete a source

• To edit a Media Cache source, click the Edit icon for the source.
• To delete a Media Cache source, click the Delete icon for the source.

To completely remove the source, restart Wowza Streaming Engine as prompted. The source remains in use until the server is restarted.

Configure an application to use Media Cache

---

With Media Cache configured and enabled, you can use it in an application. Media Cache must use the VOD Edge application type.

1. In Wowza Streaming Engine Manager, click the Applications tab, and then select your VOD Edge application in the contents panel.
2. From the Setup tab of your application, click Edit.
3. Under Media Cache Sources, choose one of the following:
    
   • All Media Cache Sources are available – Makes all sources available to the application.
   • Some Media Cache Sources are available – Lets you select the source or sources you want to make available to the application.
4. Click Save, and then restart Wowza Streaming Engine.

Configure additional Media Cache properties

---

There are several properties that control Media Cache, store, and source behavior. Users with administrative permissions can access the Properties tabs to customize these settings. Only advanced users should change properties. The primary reason to edit properties is to improve performance.

For more information on Media Cache properties, see the following articles:

• Wowza Streaming Engine MediaCache.xml configuration reference
• Configure source properties for Wowza Streaming Engine Media Cache

Log Media Cache store activity

---

In Wowza Streaming Engine 4.8.8.01 and later, you can enable logging to track available space in Media Cache stores. When Media Cache store logging is enabled, a message similar to the following is added to the wowzastreamingengine_access.log file each time a new item enters the store.

MediaCacheItemBase.getStoreItem[AWS S3] Item added to store: mediaName:amazons3/sample.mp4 length:58958409 store:{name:"default",fullness:0.55%,currSize:58961505,maxSize:10737418240,itemsInStore:1}

If the store is full or there isn't enough space left, the following message is displayed:

MediaCacheItemBase.getStoreItem[AWS S3] No room for item in store(s): mediaName:amazons3/sample.mp4 length:58958409

To enable logging Media Cache store activity, complete the following steps:

1. In Wowza Streaming Engine Manager, click the Server tab, and then click Media Cache in the contents panel.
2. On the Properties tab, click Custom in the Quick Links bar.
3. In the Custom area, click Edit.
4. Click Add Custom Property, specify the following settings in the Add Custom Property dialog box, and then click Add:
    
   • Path – Select /Root/MediaCache.
   • Name – Enter logStoreCapacityInterval.
   • Type – Select Integer.
   • Value – Enter a value, in milliseconds, to specify how often the store should be checked.
5. Click Save, and then restart the server when prompted to apply the changes.

The property is added to the MediaCache.xml file within the <MediaCache>/<Properties> container element as in the following example:

<Property>
  <Name>logStoreCapacityInterval</Name>
  <Value>10000</Value>
  <Type>Integer</Type>
</Property>

Test playback

---

The following table lists URL formats for supported protocols using an example application named mediacache and an example file named sample.mp4. For each URL, [source] should be replaced with the appropriate prefix for the source type:

• File – path1
• HTTP – http
• Amazon S3 – amazons3
• Azure – https
• Google Cloud Storage – https/[bucket-name]

Playback URL formats

More resources

---

• Wowza Media Cache Deployment Architecture (PDF)
   
• Control how items expire from Wowza Streaming Engine Media Cache
• Purge an item from Wowza Streaming Engine Media Cache
   
• Media Cache REST API query parameters