Dr. Dobb's Journal - May 2008 - (Page 38)

d05gor_p3db 3/13/08 9:07 AM Page 38 State of the Art by Andrei Gorine and Alexander Krivolapo Kernel-Mode Databases A DBMS technology for high-performance applications The evolution of database management systems has been driven by the goal of eliminating Andrei is Chief Technology Officer and Alexander a software engineer at McObject. They can be reached at gor@mcobject.com and kid@mcobject .com, respectively. latency and increasing the prioritization of data management tasks. New physical database structures—ISAM, VSAM, clustered indexes, and column-based databases—are responses to the need for faster data retrieval. Innovations in storage modality have also played a role: Caching was introduced as a means to keep selected data in memory for faster access, while inmemory database systems (IMDS) offered the ability to store entire databases in main memory, eliminating the overhead and unpredictability of disk I/O and caching logic. More than any other software component, operating system kernels embody the kind of high-priority, zerolatency responsiveness sought by database system developers. Typically viewed as the lowest-level software abstraction layer, the kernel is responsible for resource allocation, scheduling, low-level hardware interfaces, network, security, and other integral tasks. Security applications (access-control systems, firewalls, and the like) and operating-system monitors commonly place their functions in the operating-system kernel and have a need for local, high-performance data sorting, storage, and retrieval. In typical access-control application scenarios, for instance, the data structures and queries are inherently complex, yet the lookups and updates must be nearly instantaneous. However, the kernel has generally been off-limits for database management systems because DBMS overhead (file and disk I/O, locking, cache management, and related logic) could overwhelm kernel resources and disrupt OS-critical tasks. Kernel module developers have been left to either reinvent the wheel of fundamental database capabilities for use in the kernel (albeit in a limited, lightweight fashion), or deploy a complete DBMS in user-mode space and rely on expensive (in performance terms) context switches whenever kernel-mode processes require data lookup. The advent of ultra-small footprint and resourceconserving embedded in-memory databases presents an efficient alternative, making it possible to integrate database engines with the operating-system kernel. McObject (the company we work for) has developed the eXtremeDB Kernel Mode Edition that, to the best of our knowledge, is the first commercial off-the-shelf database-management system designed for such deployment. The tool consists of the standard eXtremeDB in-memory embedded database runtime adjusted for kernel usage. The adjustments are relatively minor: Database locking is implemented via kernel-mode spinlocks rather than through synchronization primitives available in the user space, such as semaphores and spinlocks, and the database runtime does not use the C runtime. To facilitate the implementation of user-mode applications accessing the kernel-mode database, eXtremeDB-KM provides a simple interface compiler utility that is conceptually similar to the well-known Remote Procedure Call Interface Definition Language compilers (RPC IDL). To illustrate its use, the eXtremeDB-KM distribution includes a sample application that creates a kernel-mode database to enforce access rules as part of an access-control system. The reference design includes another kernel module that intercepts filesystem calls and provides a file access authorization mechanism to the system. This “filter module” exports two types of interfaces: the “direct” API available to other kernel modules and drivers, and the “indirect” API that implements the ioctl interface to the database module. The Challenge Data-management code that operates inside the kernel faces considerable challenges, whether this module is custom-made or off-the-shelf. Kernel-mode drivers or application components have to be nonintrusive to the system. Therefore, the integrated data management cannot write data to a hard disk, even on systems where a large filesystem cache is present, because the transition commit on disk I/O would 38 Dr. Dobb’s Journal l www.ddj.com l May 2008 http://www.ddj.com

Table of Contents Feed for the Digital Edition of Dr. Dobb's Journal - May 2008

Dr. Dobb's Journal - May 2008 Contents Friday Night Fish Fry Alia Vox Developer Diaries Software Development Goes to the Movies Cat: A Functional Stack-Based Little Language Mojax: Mobile Ajax Framework Kernel-Mode Databases Getting Better Search Results Effective Concurrency The Agile Edge

Dr. Dobb's Journal - May 2008

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.