Dr. Dobb's Journal - November 2007 - (Page 32)

d11moha_p5db 9/10/07 10:49 AM Page 32 Core Technology DISTRIBUTED COMPUTING: WINDOWS AND LINUX processor of the Linux cluster. We had several options: • Connect to each machine using PUTTY and run the command line. However, this approach required a lot of user interaction, so it was discarded. • Connect to all the machines using SSH and start a program that executes scripts—which is what we opted to do. We wrote the Cluster program (Listing Three) to invoke the Scheduler program (Listing Four) on each node of the Linux Figure 1: Overview of the Windows/Linux distributed system. cluster. The Scheduler executes script files prepared by the script generator program. This is done using the fork() function to create a child process for each connection. The connections to the Linux nodes using SSH are set to use automatic login and the Cluster program is run on the cgywin emulator on the windows machine. The script generator program generates scripts to be executed by the Linux schedule. A script processes each job and the number of the scripts is stored in a max.txt file. The script files are generated in the form #.sh, where # is the number of the script. These files are then transferred to the directory “script” on the Linux cluster and their access mode is changed to be executable using the chmod +x *.sh command executed from a PUTTY terminal. The Scheduler program first opens the max.txt file to get the maximum number of scripts. It then enters the loop where it sequentially checks all the status files with the form “#.do.” When it finds a nonexisting status file, then it directly assigns it to itself by creating it and executes the corresponding script file. After finishing the script file, it rewrites the status file with the string “DONE” in it, then searches for the next status file until the maximum is reached. It then stops. The max is read in each loop (as sometimes we increased it) as we introduce new experiments (with new files). Only one job is assigned per script. If one node stops working due to any reasons, only one file will not be processed, and can be processed later. The generated scripts look like the script in Listing Five. After the Cluster program finishes, we check the status files. If any are of size 0, the corresponding file is not executed due to a node failure. We then execute this script (manually or by deleting this status file(s) and calling the Cluster program). If the Windows machine is restarted for some reason, we restart the Cluster program. In the worse case, two scripts execute at the same time on each node. Because the Linux cluster is not dedicated to our processes, we use the lowest priority of execution using the 32 Dr. Dobb’s Journal l www.ddj.com l November 2007 http://www.accusoft.com/ddj http://www.accusoft.com/ddj http://www.ddj.com

Table of Contents Feed for the Digital Edition of Dr. Dobb's Journal - November 2007

Contents Hmmmm Alia Vox Developer Diaries Developer’s Notebook Smart Compilers - But Smart Enough? Conversations Grid-Enabling Resource-Intensive Applications Distributed Computing: Windows and Linux Adobe AIR: Desktop/Web Convergence Transparency on Demand Reusable Associations Effective Concurrency The Agile Edge Swaine’s Flames

Dr. Dobb's Journal - November 2007

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.