MSDN Magazine - October 2007 - (Page 72)

This can be written more concisely using the C# query comprehension syntax, in which case the PLINQ version looks like this: IEnumerable data = ; var q = from x in data.AsParallel() where p(x) orderby k(x) select f(x); foreach (var e in q) a(e); To use PLINQ, the query would have to be rewritten like this: IEnumerable data = …; var q = ParallelEnumerable.Select( ParallelEnumerable.OrderBy( ParallelEnumerable.Where(data.AsParallel(), (x) => p(x)), (x) => k(x)), (x) => f(x)); foreach (var e in q) a(e); Once you have made this change, PLINQ will transparently execute the Where, OrderBy, and Select on all of the available processors using classic data parallel evaluation techniques. PLINQ uses deferred execution just like LINQ, meaning the query doesn’t begin running until you foreach over it, call GetEnumerator directly, or force the results into a list through some Using language other API such as ToList comprehensions and or ToDictionary. When the extension methods is a more query executes, PLINQ will arrange for parts of the queconvenient way of writing ry to run on the available queries and makes it easier processors through the hidto move to PLINQ. den use of multiple threads. You don’t even need to understand how all of this is done; you just see better performance and better utilization of the available processors. Though it’s not obvious, the inferred type of q differs between the ordinary LINQ-to-Objects and PLINQ queries shown previously. In the first example, q is typed as IEnumerable , where U is whatever type the f method passed to the Select operator returns. In the second example, however, q is typed as IParallelEnumerable . This usually doesn’t matter: if you’ve declared q to explicitly have type IEnumerable , for instance, the change to AsParallel will still work since IParallelEnumerable derives from IEnumerable . But it does mean that any subsequent use of q will be treated like an IParallelEnumerable . For example, if you subsequently query it, PLINQ will be chosen as the query provider. Note that some LINQ operators are binary—they take two IEnumerable s as input. Join is a perfect example of such an operator. In these cases, the type of the left-most data source determines whether LINQ or PLINQ is used. Thus you need only call AsParallel on the first data source for your query to run in parallel: IEnumerable var q = from x join y select leftData = , rightData = ; in leftData.AsParallel() in rightData on x.a == y.b f(x, y); For obvious reasons, using comprehensions and extension methods is a more convenient way of writing your queries and carries the added benefit of making it much easier to move to PLINQ. And that’s it! These are the only changes required to use PLINQ in place of LINQ-to-Objects. Because LINQ-to-XML exposes XML documents as IEnumerable data structures, everything said so far applies to querying XML content, too. Processing Query Output As already noted, due to deferred evaluation, parallelism doesn’t get introduced until you start processing the output of the query. If you’re familiar with IEnumerable , this equates to calling the GetEnumerator method. There are three basic ways that PLINQ queries can be processed, each leading to a slightly different model for parallelism. The first is pipelined processing, in which case the thread doing the enumeration is separate from the threads devoted to running the query. PLINQ will use many threads for query execution, but will reduce the degree of parallelism by one so that the enumerating thread is not interfered with. For instance, if you have eight processors available, seven of them will run the PLINQ query while the remaining processor runs the foreach loop on the output of the PLINQ query as elements become available. This carries the benefit of allowing more incremental processing of output, thus reducing the memory requirements necessary to hold the results; however, having many producer threads and only a single consumer thread can often lead to uneven work distribution, resulting in processor inefficiency. The second model is stop-and-go processing. In this model, the thread that starts the enumeration joins all of the other threads to execute the query. Once all threads have finished producing the complete set of output, the thread then proceeds to enumerating the output. This has the benefit that all processing power is devoted to creating the output as quickly as possible. It is also slightly more efficient than pipelined processing because there is less incremental synchronization overhead in the implementation: PLINQ can be more intelligent about these things because it knows exactly where the output data is going and how it will be accessed. If the work distribution is uneven between the consumers and producer, this is usually a more appropriate method of consumption. Finally, there is inverted enumeration. A lambda function is provided to PLINQ that is run in parallel, once for each element in the output. This is the most efficient mechanism because there is much more parallelism exposed to PLINQ, and the implementation avoids costly operations such as merging output from multiple threads. But it has the drawbacks that you cannot simply use a foreach loop, you must use a special ForAll API, and you must be careful that the lambdas do not rely on shared state. Otherwise, All of this discussion has assumed you’re using extension methods to write your queries. If you have chosen instead to call methods on the Enumerable class directly, then you will have a little more work to do to move over to PLINQ. In addition to the call to AsParallel, you must also reference the ParallelEnumerable type. For example, imagine the above query was written by calling Enumerable directly: IEnumerable data = …; var q = Enumerable.Select( Enumerable.OrderBy( Enumerable.Where(data, (x) => p(x)), (x) => k(x)), (x) => f(x)); foreach (var e in q) a(e); 72 msdnmagazine PlINQ

Table of Contents Feed for the Digital Edition of MSDN Magazine - October 2007

Cover Contents Toolbox CLR Inside Out Basic Instincts Data Points Cutting Edge Pooled Threads WPF Threads Parallel Linq Parallel Performance Mobile Apps Test Run Foundations Windows with C++ Netting C++ .NET Matters { End Bracket } Net Nuptials

MSDN Magazine - October 2007

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.