Growing Your CDN Dynamically with Stream Resolver

One of the biggest hurdles people face when growing their own CDN is dynamically scaling production based on demand. The liverepeater configuration gets you partway there by allowing you to scale your edges to handle a larger volume of requests and sourcing its streams to a few origins. Unfortunately, as you publish more streams and put up new origins, you will need to reconfigure your edges to learn about the changes made to your origin infrastructure.

The StreamResolver module aims at solving part of that equation by allowing you to dynamically determine the best-suited origin for a particular stream request. The process for obtaining the origin is as follows:

1. New stream is requested on your edge.
2. The module handles the request and checks all known origins simultaneously.
3. The first positive response is assumed the most appropriate origin.
4. A connection is made from the edge to the origin.

On a heavily burdened edge, these requests have to wait until the edge has time to figure where this stream resides. However, the module takes this into consideration by caching the result so subsequent requests to valid connections are resolved to the same location(s).

With growth of your CDN, you may want to add an origin. You can do this by updating your edge’s origins.txt file, and it will see the additions in real time versus requiring a restart of the given application. You could extend this to be even more dynamic by adding an announcement from the origins to known edges.

StreamResolver also solves the problem of an origin going down or failing to respond and subsequent requests still being routed to that origin. StreamResolver makes it so the origin will no longer respond to requests from the edges, and republishing can happen on an alternative origin to ensure continued playback.

While the StreamResolver module doesn’t solve the entire problem of scalability, it is a good starting point for your own CDN.