OPENdj System Design Document

This document presents a detailed look at the actual system architecture of the OPENdj system, and is organized as follows:


Audio System Overview


In the diagram above, the large boxes represent separate systems: the DJ (typically a single computer), the OPENdj server (typically multiple networked computers), and the listeners (typically one machine per listener). The empty space between the boxes is the Internet (or some other network).

This design has several advantages:

  1. As much work as possible is done at the broadcaster's end. Particularly, the CPU-intensive audio encoding (using the LAME MP3 encoder) is done at the client end. This also conserves bandwidth because raw audio is very large and cumbersome to send over networks.
  2. The control protocol (ODSP) is very lightweight to maximize the bandwidth available to the data (MP3) stream. A detailed description of the OPENdj Streaming Protocol is presented in the OPENdj Streaming Protocol Specification.
  3. The data stream is an MP3/shoutcast stream that has good audio quality even at low bitrates. This means that broadcasters without high-speed Internet access are still able to use OPENdj. A 56Kbps modem connection is sufficient to broadcast. Using MP3 also creates flexibility on the listener's end, where a wide variety of MP3 player software is available.

The diagram presents two distinct kinds of data flows: audio data flows and control data flows. The audio data must somehow get from the broadcaster's live feed to the listener's speaker. The control data flows ensure that this happens properly and in a secure manner.

Audio Data Flow

  1. Raw audio data is arriving on the DJ's line-in port on their sound card.
  2. OPENdj-client captures the raw audio signal and passes it to the MP3 encoder.
  3. OPENdj-client reads the MP3-encoded audio from the encoder.
  4. OPENdj-client sends the MP3 audio to JRev, a software library that knows how to communicate with shoutcast servers (like Icecast).
  5. JRev connects to the Icecast server, authenticates, and begins sending MP3 audio data.
  6. Icecast receives the audio data and relays it to any listeners.

The description and accompanying diagram just presented are not entirely true. They are accurate for the Win32 broadcaster using JRev. For the UNIX broadcaster using LiveIce, it is slightly different:

  1. Raw audio data is arriving on the broadcaster's line-in port on their sound card.
  2. OPENdj-client starts up LiveIce.
  3. LiveIce captures the raw audio data.
  4. LiveIce sends the raw audio data to the MP3 encoder.
  5. LiveIce reads the MP3-encoded audio from the encoder.
  6. LiveIce connects to the Icecast server, authenticates, and begins sending MP3 audio data.

So, to summarize the difference in Win32 vs. UNIX implementations: The Win32 version of the broadcasting software spawns a separate process for the LAME encoder and "manually" handles audio capture, piping the raw audio to the MP3 encoder, reading the MP3 stream from the MP3 encoder, and (finally) piping the MP3 stream to JRev which relays it to a shoutcast server. The UNIX version simply spawns a liveice process, which handles all the audio capture, MP3 encoding, and shoutcast streaming responsibilities itself.

Control Data Flow

But just understanding the audio-data flow is not enough. For the audio to flow properly, a control conversation is also occurring between DJServer and OPENdj-client. The control conversation is much more interesting:

  1. Pre-Broadcast Activity
    1. The DJ starts up OPENdj-client shortly before a scheduled broadcast commences.
    2. The DJ, using OPENdj-client, logs in to DJServer.
    3. DJServer sends OPENdj-client information on which channel the DJ will be broadcasting on, where the Icecast server is (host and port), what the Icecast authentication information is, the time at which the broadcast will commence, and other important configuration data.
    4. OPENdj-client periodically asks DJServer how much longer until broadcasting will commence. This is to compensate for any possible clock drift between the two machines.
    5. As the time to broadcast nears, OPENdj-client inquires more frequently to DJServer to ask how much time remains until broadcasting commences.
    6. When OPENdj-client asks this question and less than a few seconds remain, DJServer does the following before sending OPENdj-client any response:
      1. DJServer waits until the exact moment that the broadcast should begin.
      2. DJServer tells the firewall to stop allowing upstream broadcasting traffic from any previous DJ who might have been finishing up a show.
      3. DJServer tells the firewall to start allowing upstream broadcasting traffic from the DJ who is about to go on the air.
    7. DJServer responds to OPENdj-client with a message indicating that broadcasting should commence immediately.
    8. OPENdj-client receives this response and starts up JRev (or LiveIce) to send the audio stream to the Icecast server.
    9. Icecast notices that the last DJ's upstream connection has been dropped, but a new DJ has connected to the same mount point. Icecast transparently transfers all listeners from the old broadcast to the new.
    10. DJServer notices (from communicating with Icecast) that the new DJ has arrived. DJServer tells Icecast to begin archiving the broadcast. DJServer makes an initial entry in the database for the archive, marking it as in-progress.
  2. Activity During Broadcasting
    1. DJServer
      1. Periodically monitors Icecast to:
        1. Ensure that the upstream bitrate is within acceptable limits. If the bitrate is too high or too low, DJServer tells Icecast to kick the DJ off.
        2. Count the number of current listeners. During the course of the broadcast, OPENdj-client periodically asks DJServer how many listeners are tuned in. DJServer returns two numbers: the current number of listeners, and the average number of listeners over the entire broadcast.
      2. Responds to commands from OPENdj-client. These commands are typically used to:
        1. Ask how much time remains in the broadcast.
        2. Ask how many listeners are listening to the broadcast.
        3. Update the meta-data description of the broadcast.
    2. Icecast
      1. Accepts connections from new listeners.
      2. Broadcasts the audio stream to all listeners.
      3. Continues to archive the broadcast.
      4. Responds to commands from DJServer.
    3. OPENdj-client:
      1. Continues to stream MP3 audio to Icecast.
      2. Periodically asks DJServer how much time is left to broadcast. It uses DJServer's response to populate its user interface and give the DJ a count-down clock.
      3. Periodically asks DJServer how many listeners are tuned in. It uses DJServer's response to populate its user interface and give the DJ a listener count.
      4. Allows the DJ to provide a description of the current broadcast in progress. The DJ can specify a genre, mood, and textual description of the broadcast. When the DJ is done entering this information, OPENdj-client tells DJServer to update the meta-data for the archive. This updating can be done any number of times during a broadcast, in case a DJ changes their mind. Any changes completely overwrite the previous meta-data.
  3. Final Broadcasting Activity
    1. When the time to complete broadcasting approaches, OPENdj-client asks DJServer how much time is left more frequently.
    2. When OPENdj-client asks this question when there is less than a few seconds remaining, DJServer does the following:
      1. DJServer waits until the exact moment that the broadcast should end.
      2. DJServer tells OPENdj-client to cease broadcasting.
      3. DJServer tells the firewall to stop allowing upstream broadcasting traffic from the DJ who is finishing their broadcast.
      4. Unless another DJs is broadcasting immediately after this DJ has finished, Icecast will clean up any resources associated with the DJ after a timeout period has passed. Remember that Icecast will notice the DJ is no longer connected when the firewall stops allowing packets from that DJ into the system.
    3. When OPENdj-client receives the response from DJServer (3.2.2 above), OPENdj-client stops JRev/LiveIce and changes its user interface to notify the DJ that broadcasting has stopped.

Chat System Overview


As in the audio system overview, the large boxes represent separate systems: the DJ, the OPENdj server, and the listener(s). Decoupling the audio flow from the chat flow provides for a more flexible architecture allowing the design and implementation of each to vary without disrupting the other. For complete details on the OPENdj Chat Protocol, read the spec.

Chat for Listeners

Listeners can enter the chat by clicking on a link from the same web page that starts up the listener's audio player. The link resolves to a simple web page that contains a Java applet. The browser downloads and executes the applet code. The applet asks the listener to pick a username for the chat. If the name is taken, the user is asked to choose another. Once an acceptable name has been selected, the listener enters the chat room. The listener then sees any comments made by other chatters, and can make comments of their own.

Chat for DJs

When OPENdj-client is told by DJServer to start broadcasting, it also starts up an instance of the chat program. In this case, it is not running as a Java applet, but rather within OPENdj-client. In this situation, the chat program uses an auto-login protocol to log in to the ChatServer. This is possible because the DJ has already logged in to DJServer. Once logged in, the DJ can see and comments that listeners make, and can send comments of their own.


Component Specifications


This section describes each component of the OPENdj system. For components provided by third parties, only the protocols and other public interfaces required are described.

Media Source

The actual content has to come from somewhere. This architecture assumes it can be anything you can plug into your sound card through some sort of line-in jack. Typically, this would be an RCA jack or perhaps a headphone-type jack. On the computer side of things, the signal from that card must be available through normal operating system interfaces.

Media Encoder

An MP3 encoder used in the OPENdj system. The current implementation uses the LAME encoder, but other encoders could be used. The only requirement OPENdj assumes is that those encoders expose a command line interface that allows the programs can handle receiving signals on standard input and writing the encoded signal on to standard output.

Broadcasting Software

The broadcasting software coordinates all the efforts between:

  1. On the operating system side: media capture and encoding
  2. On the network/server communications side: negotiating a connection with a DJServer, interacting with the DJServer throughout the connection, and transmitting the actual encoded content to the media server.
  3. On the user interface side, providing the DJ an informative and easy to use application to work with.

The broadcasting software is called OPENdj-client, and runs on Win32 and UNIX (though Linux is to date the only one tested, other flavors should require very little porting effort). The details of the design of OPENdj-client are provided in the DJServer Clients document.

Source-to-Media Server Transport

The broadcasting software does not directly send the signal to the media server. Rather, a wrapper library is used which sits on top of real media transport library. The current incarnation of OPENdj assumes that the media server understands the shoutcast protocol. For the UNIX version of OPENdj-client, the wrapper library is built on top of LiveIce, for the Win32 version, the wrapper library is built on top of JRev.

Server Operations

The DJServer is responsible for:

  1. Authenticating and managing OPENdj-clients connections.
  2. Acting on behalf of OPENdj-clients to satisfy requests such as initiating a broadcast or updating show meta-data.
  3. Coordinating access to the media server. This includes communicating with the media server (currently via Telnet using Expect) and configuring the firewall on the fly (currently done with ipchains/iptables and Expect).

A detailed description of the OPENdj Streaming Protocol is presented in the OPENdj Streaming Protocol Specification.

Media Server

The Media Server used in the current OPENdj system is Icecast. Icecast was chosen because (using Expect), external software (e.g. the DJServer) can programmatically query and control it. OPENdj requires that the media server be able to programmatically provide interfaces for these operations:

  1. Given a channel (mount point), retrieve the IP address and bitrate for the source currently broadcasting.
  2. Retrieve a list of all listeners, which must at least include which channel (mount point) each listener is listening to.

On the listening side, Icecast supports the shoutcast protocol. Any media server which meets the criteria above and supports the shoutcast protocol could be used in place of Icecast, with minor changes to the OPENdj system.

Audio Player

On the listening end, any player that supports the shoutcast/MP3 protocol will work. These currently include: RealPlayer and WinAmp for Windows, Macast and Soundjam for Macintosh, XMMS and FreeAmp for UNIX.


Copyright 2001 DAX Interactive, LLC.
$Author: jonathan $
$Revision: 1.1.1.1 $