This section summarizes the motivation and results of my diploma thesis. The thesis is about specializing the Network Multimedia Middleware (NMM) for IBMs Cell Broadband Engine (CBE) in order to realize complex and computationally intensive multimedia applications. The Cell Broadband Engine is used in Sonys PS3.

My diploma thesis raised from a cooperation between the IBM laboratory in Stuttgart, the University of Technology in Saarbrücken and my home university in Aachen (RWTH).

Over the last few years the expectations to modern multimedia applications and devices increased enormously. The television in the living room changes to become a multimedia-center which provides much more possibilities than just watching a single TV program. Many users think about using their television for surfing the Internet, playing DVDs and listen to music. Furthermore users may want to see High Definition Television (HDTV) programs, use the Electronic Program Guide (EPG) to get detailed program information, record movies to connected network locations and much more (see here).

In the past many special hardware- and software solutions have been developed, set-top box with integrated Graphical User Interface (GUI)) to satisfy the new requirements. The problem with those special solutions is their inflexibility with respect to new requirements and their low value of reuse. Constantly new expectations and requirements come up. Customers want to decode multiple H.264 video streams in parallel to get live program previews. Also transcoding of video content to mobile devices is evolving.

To cope with the frequently changing requirements in the domain of multimedia more generic and flexible solutions are required (see figure 1). For this reason general purpose high-speed multi-core processors for multimedia processing are developed. The required computational power for realizing sophisticated and flexible multimedia applications is reached by making use of parallel computing concepts.

This is possible since multimedia processing can well be decomposed into different sub algorithms. For example, a typical video stream processing chain consists of receiving, de-multiplexing, decoding, de-interlacing, scaling, color space conversion and displaying. Those different processing steps can be distributed to different cores of the multi-core processor. Then all the cores process their corresponding subproblems in parallel.

One of those generic high-speed multi-core processors is the Cell Broadband Engine\texttrademark (CBE) developed by Sony, Toshiba and IBM. The CBE is a heterogenous multi-core processor with eight sub cores and one main core. The CBE is especially designed for multimedia applications. Each sub core has its own memory and can be programmed completely independently. The sub cores can exchange data among each other using a highly optimized ringbus. The main core controls the execution of the programs running on the different sub cores. In this thesis the CBE is used as a representative of those generic high-speed multi-core processors.

This new freedom of programmability also has it's price. Developers now have to care about data movements, synchronization and platform specific details completely themselves. Hence, in this context a scalable transparency which provides only the required flexibility to the developer would be desirable. So a developer could just focus on developing an application without caring about all the platform specific details but also can customize special platform parameters if necessary. For this reason it would be desirable to have a flexible and high-performance framework which simplifies the development of multimedia applications on generic multi-core processors.

Conceptually, this is similar to what so called multimedia middleware does for distributed systems. In a distributed system computers are connected over a network which is used for exchanging information. Existing open multimedia middleware solutions provide customizability, extendability and a strict distinction between data processing and data movements in order to simplify the development of network distributed multimedia applications \cite{lohse:ip2002}. In a multi-core processor the different cores are also connected by an on-chip network. Over this network the cores can communicate to each other on chip-level.

This similar structure and similar software requirements lead to the idea of mapping concepts from open multimedia middleware systems to multi-core architectures to get similar design benefits. In order to provide a comfortable design but also a platform typical performance the idea of this thesis is to specialize a multimedia middleware applying the concepts of strict distinction between data movement and processing to the new platform with respect to the technical constraints like limited memory sizes and data alignments.

The overall goal is to evaluate how efficient this is possible.

The results show that a multimedia middleware can be specialized for systems with hard technical constraints if the required parts of the middleware are flexible and extendable designed.

In NMM the openness and the strict separation between communication tasks and processing tasks were the key concepts.

However using the multimedia middleware is not for free. In this implementation a performance loss of 10-12 percent in comparison to a highly optimized implementation is measured. Still, in a real-world scenario the use of multimedia middleware was even better than the reference case.

For a big set of software projects it is more important to have flexible and reusable components than reaching absolute peak performance. For those projects where a performance loss of 10-12 percent is acceptable using a multimedia middleware can be an option since the development speed, the quality of code and the reusability can be increased.

Extending the NMM has shown the relevant concepts a software has to provide to allow a specialization. It turned out that one of the most powerful design patterns used by software to allow this is the strategy pattern.

However, the results also show that code reuse is not always possible because of technical constraints. In this case some parts had to be reimplemented on SPE because the limited LS size and absence of multithreading did not allow a reuse. Hence, one of the most important things when specializing a software are generic interfaces. Those allow reimplementation of similar concepts on a different platform. However, it needs experience to create interfaces which are valid for different specialisations and requires the identification of high level concepts.

NMM provides generic interfaces and hence allows a reimplementation of concepts as described above. Hence, in sum, NMM is an appropriate software for realizing multimedia applications which run on systems with hard technical constraints.

The CBE is a powerful platform if the available hardware features are used correctly. Concerning multimedia processing there exist some promising programming models. Especially the streaming model is applicable for this kind of problem. However, programming the CBE from scratch is challenging and time consuming. Because of its architecture and the technical constraints it cannot be programmed like a conventional processor. To get a suitable performance one must understand and use the provided hardware features. Overall, the CBE performance combined with powerful middleware can result in dynamic and ambitious multimedia applications.