Dr. Dobb's Journal - December 2008 - (Page 45)

d12babk_p4db 10/10/08 10:53 AM Page 45 descriptor. Then the reader thread would read or close the wrong file! For the socket connections, a solution is to use the shutdown() system call. It stops the communication without freeing the file descriptor. The logic for stopping the reader thread is shown in Listing Two. On some operating systems, this is also the only working solution: On FreeBSD, a close() without shutdown() does not wake up the processes waiting in a read() or select(). The reader thread would receive the shutdown indication and close the socket before exiting, so afterwards it’s safe to destroy the state without causing races. Another issue to consider is that closing by shutdown() or by close() may not be considered a read event in the OS. If a library does select() or its more modern analog poll() waiting for read events only, then it would not be woken up. Closing a socket always counts as an error event and as a write event, but somehow not as a read event. So the select() waiting for a readready event just continues sitting there and waiting. Check the library you use, and if needed, modify it to check for the error events, too. However, shutdown() works only on the sockets with established connections, not on the ones listening for new connections nor on the other kinds of file descriptors. If you can change the code of the library (which is thankfully an option for open source libraries), can we still make it work without the shutdown() call? Yes: We need to add a mutex for accessing the state; see Listing Three. Now the reader thread checks the state before doing its system calls and avoids doing any more calls on the closed socket. But not always! Note that there is still a little time between unlock() and read(), which becomes a race window. And there is no way to close it reliably. It would be very nice if the OS allowed the release of locks atomically when entering sleep on the read() system call. But it doesn’t. So the best attempt to fix this is by adding a little sleep in the revocation code, as in Listing Four. It’s not pretty, but it works okay in practice. The same approach can be used even without access to the source code of the Listing Five class WrapSocket : protected LibSocket { protected: mutex mutex_; bool open_; // flag: this socket is open public: int read(char *buf, int blen) { bool ok; mutex_.lock(); // LibSocket::fd accesses the field in the parent class ok = open_ && LibSocket::fd >= 0; mutex_.unlock(); if (!ok) return 0; // EOF return LibSocket::read(buf, blen); } void stop_reader() { mutex_.lock(); if (open_) { int workfd = LibSocket::fd ; // prevent LibSocket from messing up things LibSocket::fd = -1; open_ = false; usleep(20000); // 20 msec ::close(workfd); } mutex_.unlock(); } } Listing Six // stopping code: lock(state.mutex); if (state.open) { int nullfd; nullfd = open("/dev/null", O_RDONLY); if (nullfd < 0) { state.fd = false; // should not happen but just in case usleep(20000); // 20 msec close(state.fd); } else { dup2(nullfd, state.fd); close(nullfd); } } unlock(state.mutex); library (other than to get the file descriptor, and even that can be worked around with dirty tricks). To do it, create a wrapper API. In the C++ code, you can define a wrapper class over the library class. This wrapper class would have a mutex and wrap the underlying calls safely. Listing Five is the skeleton of such a class. The only extra trick is that when you take away the socket, you need to set the library’s idea of it to a known invalid value “–1” pre, venting the underlying library from doing unsafe things. A better solution has been suggested to me by Robert Watson of the FreeBSD Project. The dup2() system call takes a file descriptor and copies it to another file descriptor, in the process closing the file that used to be referred to from the new file descriptor. So it’s a perfect atomic operation to avoid the race. The stopping code is shown in Listing 6. Opening the device /dev/null provides a file descriptor that returns EOF all the time. And opening it with a read-only flag makes sure that it can’t be written-to either. It’s a perfect placeholder descriptor that does nothing. Many UNIX implementations support similar functionality directly in the kernel. It’s called “revocation.” When a filesystem gets forced-unmounted, all the file descriptors of the open files on it are revoked: They return an error on any future calls, yet the descriptors themselves aren’t closed. A similar thing is done when the controlling TTY process exits— revoke() causes the file descriptors of this TTY in all the other processes to be revoked. But the programs waiting for input from this TTY would not necessarily wake up until this TTY gets actually closed. What the dup2() call gives us is a sort of application-level revocation. DDJ 45 December 2008 l www.ddj.com l Dr. Dobb’s Journal http://www.ddj.com

Table of Contents Feed for the Digital Edition of Dr. Dobb's Journal - December 2008

Dr. Dobb's Journal - December 2008 Contents Friday Night Fish Fry Alia Vox Developer Diaries Conversations The Man Who Sold the Sky Performance on Rails LINQ-to-SQL and T-SQL A Remote Java RMI Registry Beyond B-Trees File Descriptors and Multithreaded Programs Effective Concurrency The Agile Edge Swaine's Flames

Dr. Dobb's Journal - December 2008

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.