MSDN Magazine - October 2007 - (Page 73)

introducing parallelism will render your queries incorrect and can cause unpredictable crashes or data corruption. If you can express your problem in terms of this API, however, it is always the preferred method. Which of these three models you use depends on what you do with the query results. The default is the first—pipelined processing. As soon as MoveNext is called on the resulting query enumerator, a set of additional worker threads will execute the query, and results are returned from this and all subsequent MoveNext calls as they become available. If a call to MoveNext is made and no output is ready from the query producer threads, the calling thread will block until an element is available. If you just use foreach to process the output of a PLINQ query, this is what you will get: var q = some query ; foreach (var e in q) { a(e); // this runs in parallel with the execution of ‘q’ } is used as the default: if you use the ToArray or ToList methods, these operators will internally force a stop-and-go operation. If you have a sort in your query, stop-and-go will be used instead because pipelining the output of a sort is wasteful. A sort exhibits extremely high latency (because it generally needs to sort the whole input before producing a single output element), and so PLINQ prefers to devote all processing power to completing the sort as quickly as possible. In order to use inverted enumeration, you must use a different PLINQ-specific API: public static class System.Linq.ParallelEnumerable { public static void ForAll ( this IParallelEnumerable source, Action action); the other standard query operators } Using the ForAll API looks quite a bit like using a foreach loop, as you just saw: var q = … some query …; q.ForAll(e => a(e)); The IParallelEnumerable interface actually offers an overload of GetEnumerator that takes a bool argument called pipelined, allowing you to choose stop-and-go processing instead (true means pipelined and false means stop-and-go). Upon the first subsequent call to MoveNext, the entire query will be executed and the call will only return when all output is available. Subsequent calls to MoveNext just enumerate a buffer containing the output: var q = some query ; using (var e = q.GetEnumerator(false)) { while (e.MoveNext()) { // after the 1st call, the query is finished executing // we then just enumerate the results from an in-memory list a(e.Current); } } Concurrent Exceptions Although the statements earlier about PLINQ’s parallelization process being entirely transparent were mostly true, there are a small number of places where the use of parallelism can leak through the simple abstractions presented above. These are the gotchas that were alluded to earlier. Happily, most of them are minor, but you should be aware of them anyway. Any lambdas or query operators that throw an exception stop sequential LINQ queries from executing right away. That’s because when only one processor is used to run the query, elements are processed one after the other, sequentially: if an operator fails on There are a few special cases where stop-and-go processing Compelling Scenarios While reading this article, you have probably already begun to imagine some ways to use PLINQ in your own applications. Perhaps you are already using LINQ today and want to improve your application’s scalability on machines with multiple processors or multiple cores. Of course, PLINQ can make your current programs run faster, but it also allows you to do more compute work and operate on larger data sizes in the same amount of time, while processing streams of data at a faster rate. In doing all of this, the new PLINQ technology might open up new application possibilities that you could not previously attempt. Let’s look at a few illustrations of scenarios in which multicore and PLINQ open up new doors. Consider a music producer working in a sound studio who wants to apply a series of effects on raw instrument sounds to produce a more aesthetically pleasing, production-quality master track. The company that provides his mixing software could apply these effects using PLINQ. These effects are usually composed of filters and projections over large streams of data (the raw music). PLINQ could greatly speed up the production time and utilize more powerful hardware as it becomes available. This approach might even allow music transformations in near real time, instead of doing complete post-production processing. Likewise, consider a foreign exchange currency trader who looks for arbitrage conditions (inefficiencies in the market) in order to make a profit. Such shifts are minute and disappear as the market is constantly reaching equilibrium, requiring very fast trades. Querying stock trade information using parallelism via PLINQ could enable close to real time decision making, informed by large amounts of data and complicated analysis and computations. These are just a few samples of the ways in which the speedup provided by PLINQ on multi-core hardware can provide a business advantage. Other domains offer similar opportunities, such as healthcare, economics, geological modeling, scientific computing, traffic control and simulations, gaming, artificial intelligence, machine learning, linguistic analysis, and the list goes on. PlINQ october2007 73

Table of Contents Feed for the Digital Edition of MSDN Magazine - October 2007

Cover Contents Toolbox CLR Inside Out Basic Instincts Data Points Cutting Edge Pooled Threads WPF Threads Parallel Linq Parallel Performance Mobile Apps Test Run Foundations Windows with C++ Netting C++ .NET Matters { End Bracket } Net Nuptials

MSDN Magazine - October 2007

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.