Dr. Dobb's Journal - November 2007 - (Page 22)

d11zeid_p6db 9/10/07 11:03 AM Page 22 Core Technology by Timothy Hoehn and Bob Zeidman Grid-Enabling Resource-Intensive Applications From one computer to many In this article, we discuss how we took an application designed to run on a single PC and created Tim is a research engineer at Zeidman Consulting. He can be reached at tim@ zeidmanconsulting.com. a framework that enabled it to distribute its processing over a network grid. The original application is called “CodeMatch” and the grid version we created is “CodeGrid.” CodeMatch detects software plagiarism by comparing two directories of possibly thousands of source-code files, then producing a report showing all matching patterns found (www.ddj.com/dept/architect/ 184405734). CodeMatch is used in court cases to help prove or disprove copyright infringement. Because its algorithms are very rigorous, they are also very time consuming, and CodeMatch can take a long time to compare large numbers of files. For this reason, we were seeking a way to speed it up and came up with the idea of a network grid for resource sharing. The problem we were trying to solve can be defined as follows: • Distribute and execute a set of x tasks on y number of nodes of a network. • Collect the results generated by the nodes and integrate them into a single report. • Implement load balancing. • Automatically scale the solution to the number of available nodes in the network. • Integrate the existing CodeMatch software with minmal modification to the existing code base. • Run on Windows. • Allow fault-tolerance in case a node has problems during execution. • Develop in a short amount of time. There are many different ways to share resources between multiple computers over a network, and in our research, we compared some of these by listing the respective advantages, disadvantages, and the trade-offs of choosing one way over another (see Table 1). The main advantages of distributed computing are increased performance by sharing the load of resource-intensive applications, improved scalability, and fault tolerance. These advantages come at a cost however of added complexity, potential security problems, and increased manageability issues. To develop a successful distributed system, careful planning is necessary as remote calls can be 1000 times slower than a local call. There has to be a balance between the added overhead of the network management and the size of the job itself. In the search for the best approach to this problem, we explored several possible solutions, using different programming methodologies (such as procedural and object-oriented programming) and different frameworks (such as .NET and Java frameworks). We looked at the various software tools available, and different communications methods such as Remote Method Invocation (RMI) and Remote Procedure Calls (RPC). We also looked at different algorithms for distributing the work such as the masterslave paradigm and the push-pull model. While our design choices are by no means the only ones, they represent what we felt were the best choices in this particular project. Bob is the president of Zeidman Consulting. He can be reached at bob@zeidmanconsulting.com. Architectures Distributed programming has several basic common architectures: client-server, peer-to-peer, distributed objects, clustering, and grid computing. These architectures are not all mutually exclusive, and particular implementations may be combinations of these architectures. 22 Dr. Dobb’s Journal l www.ddj.com l November 2007 http://www.ddj.com/dept/architect/184405734 http://www.ddj.com/dept/architect/184405734 http://www.ddj.com

Table of Contents Feed for the Digital Edition of Dr. Dobb's Journal - November 2007

Contents Hmmmm Alia Vox Developer Diaries Developer’s Notebook Smart Compilers - But Smart Enough? Conversations Grid-Enabling Resource-Intensive Applications Distributed Computing: Windows and Linux Adobe AIR: Desktop/Web Convergence Transparency on Demand Reusable Associations Effective Concurrency The Agile Edge Swaine’s Flames

Dr. Dobb's Journal - November 2007

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.