Threading and Penn

This is dedicated to Sketch. ;)

Penn is a single-thread program (Though on Unix/Linux systems it is a multi-process one, to the same effect... at least when it comes to doing potentially slow and blocking hostname lookups.) It is certainly POSSIBLE to make it a multi-threaded program. It might not be desirable to, though.

How and why:

The command queues and softcode parser and anything related needs to be a single thread, running one queue entry after another. Having multiple commands executing at once would cause meltdown; all existing softcode assumes --- heck, depends --- on being the only thing happening at that moment.

The way to make a multithreaded mush is to have each of several conceptually independent parts running in its own thread, passing messages back and forth.

Network I/O can be one or more threads. One possible model: One thread listening for connections. One thread listening for text coming from sockets. One sending text to sockets. Eventually, SSL connections will be handled by a separate process to allow those connections to persist over a restart of the main mush process. A thread dedicated to talking to that concentrator might be a possibility too.

When the listen thread gets a new connection, it passes it onto a hostname lookup thread (Which can also do sitelock checks), which either drops the connection, or adds it to the list of ones used by the I/O threads.

The main game thread, running queue entries, doing dbcks, saves, and so forth.

A thread for running SQL queries, using an asynchronous interface that queues a new command for each row in a query as it runs. This eliminates a major issue with the current inline SQL system, where a slow query hangs the entire game.

Aside from the SQL thread, none of this would result in any performance increases to speak of; the vast majority of processor time would be spent, as it is now, running the queue in a single thread. It's merely an organizational model.

Wether it's beneficial or not... I don't know. The current model works fine, though I'm interested in replacing the core select() with libevent's callback model, or writing our own polling code to wrap more efficient ways to tell if network activity or a timeout has happened: epoll() on Linux, kqueue() on *BSD, or poll() on other unixes, and otherwise working to support a larger number of connections. That's orthogonal to threads, though.

Threading does mean adding a lot of synchronization, which is a pain and leads to hard-to-catch bugs when forgotten. For example, every access to the list of connected players requires a mutex or read/write lock so only one thread is looking at it at a time. Portability is also an issue; the standard thread libraries on unixes and Windows are different. Some compilers/OS combinations require additional arguments to the compiler when using threads. Using something like glib's thread library is an option to push most of this work onto someone else.

Other problems: Older GNU/Linux installs using the LinuxThreads library will not work with Penn, due to conflicting use of signals. Its replacement, NPTL is probably fine. NetBSD has long had problems with threads on some architectures. Using a user-mode thread library like Pth on those doesn't help when it comes to the blocking hostname lookups and SQL queries.

Right now, I don't think the effort is worth it. Maybe if starting from scratch with a new mush server...