LAME: Asynchronous file processing
The encoding process of LAME on file level is shown in the following picture:
The uncompressed file is read in blocks and the compressed file is written in blocks.
Here, read and write operations alternate.
Although this approach and the use of synchronous I/O operations is not optimal
the encoding speed of one single file is not affected.
The process can be described in pseudo code in an imperative programming language as follows:
MP3EncodeFile(wavFilename, mp3Filename)
{
infile = OpenFile(wavFilename);
outfile = OpenFile(mp3Filename);
for (i = 0; i < maxBlocks; ++i) {
inblock = ReadBlock(infile);
outblock = EncodeBlock(inblock);
WriteBlock(outfile, outblock);
}
}
In general, it's possible to call the function "MP3EncodeFile" in parallel on different threads for several files.
This is suitable for two or three threads.
With even more threads, too many I/O operations will compete for a serial data device,
so that the effiency losses exceed the benefits of more processors.
Fiber Pool has an I/O scheduler to handle such problems.
An arbitrary number of tasks can be passed to it that are processed with the best possible I/O performance, then.
The pseudo code in an asynchronous programming language could be as follows:
MP3EncodeFile(wavFilename, mp3Filename)
{
Memory inMemory;
Memory outMemory;
ReadFile(wavFilename, inMemory);
WriteFile(mp3Filename, outMemory);
Encode(inMemory, outMemory);
}
By calling this function the three tasks "ReadFile", "WriteFile" and "Encode" are passed to
the underlying system (here: Fiber Pool) for further processing.
When processing several files the function can be called arbitrarily, so that for n files 3n tasks
will be passed to the system.
Unlike the imperative solution in which the sight distance of a thread is the currently executed
program line and no knowledge about further threads exists, the asynchronous system obtains
sufficient information in order to develop strategies on how to use CPU, I/O and memory up to the next
synchronization point.
Implementation in fpMP3:
The asynchronous function "MP3EncodeFile" was implemented as C++ class "MP3EncodingTask". The I/O strategy
was defined to prefer read operations rather than write operations (see documentation of "SetDefaultFileIOSchedulingStrategy").
The CPU and memory strategy can be set through the command line.
|