The RegexPolicyDaemon (rxpd) can be used to efficiently check data against different lists of regular expressions. This can be used to build whitelists/blacklists to protect many kinds of Internet services. It uses a simple textual protocol that is easily implementable in scripting languages. Example usages are access and content control (spam filtering) for CGI scripts, wikis, email, revision control systems, IRC servers and clients, and so on.

Rxpd encourages users to distribute their lists in a friend2friend network. It has features to fetch, updaten, filter and merge lists. The idea is that users/administrators maintain manageable lists which cover single topics and then merge them together.

Rxpd targets to be simple and efficently validating data against Regular expressions. It has (yet) no configuration file for the daemon itself and is controlled by commandline options. Most management of regular expression lists can be done remotely over a simple protocol. By itself it has has no authentication but there is a policy check which validates incoming requests against an special regex list which then defines if the client is allowed to do a certain task. Any further management like distributing the lists, authenticate sessions more strongly and so on should be done by other means and are not planned to be included in rxpd.

The goal it to create a common place which applications can use to validate any kind of data. This works efficently because short lived programs like CGI scripts take the advantage of regular expressions which are precompiled in core and generally such lists might be shared between different applications.

Release tarballs are attached to the wiki at: http://www.pipapo.org/pipawiki/RegexPolicyDaemon?action=AttachFile

I am using gpg signed tarballs for distribution. As first step one has to check the signature

This will produce a rxpd-X.Y.tar.gz and report if the signature could be validated.

Since the package is built with gnu autotools, the usual build and install procedure works:

The development version is available via git from git://git.pipapo.org/rxpd or mirrored at repo.or.cz git://repo.or.cz/rxpd.git.

After you cloned the repository you need to bootstrap the autotools first

Then the usual configure / make will work.

There is a special makefile target make meta to bring several files (README, AUTHORS, NEWS, TODO) in sync with the Rxpd Documentation wiki and update the Change``Log.

Rxpd requires gnu-pth and its development headers.

A single executable called rxpd will be installed in $prefix/bin.

One list of rules can be used to define access policies for the rxpd itself (-p option). Each command will be extended with access protocol (one of tcp4, tcp6 or unix) and the peer address and then checked against this policy list. When this check yields in an ACCEPT:.. rule, the command is allowed, for everything else will result in an error and drop the connection.

For example if -p policy is used:

We want to protect a wiki or such against vandalism: blacklists are in $blacklists.d/ lets say /etc/blacklists.d/

The wiki engine builds a tuple hostname;ip which is checked against a blacklist which classify the hosts

this is /etc/blacklist.d/hosts

so printf("CHECK:hosts\n%s;%s\n", hostname, ipaddr) send to the blacklist daemon will result in either allow, deny or check send back. The first both (allow/deny) results are handled obliviously. With the check result the edited content will be filtered against another list /etc/blacklists.d/content

Rxpd acts on following signals:

Lines in can be at most 4095 bytes long.

Example list file, let’s name this example:

Matches will later report the line matched, without the atime and first colon part. "Macrosoft" matches "M.*soft" thus "reject:M.*soft" will be returned.

Note that the first accept rule has no atime, to initiate atimes they can be initalized with 0 the daemon will update them on access and rewrite the List files with the SAVE command or when it recieves a SIGTERM.

When there is an error in a regular expression, it will be replaced with #ERROR:, followed by the cause of the error, followed by the rules string in quote.

A client can indicate that it is finished by sending !EXIT on a single line for all commands which take multi line input except CHECK:.

Rxpd is intended to be used as distributed system, this makes clear that data put into it has to follow some rules to avoid clashing with foreign data. While rxpd doesn’t enforce any naming rules I write some suggestions/proposals here.

Rule lists in rxpd are files, with 0.3 they are organized as directory hierachy under the rules-basedir.

Most importantly we need to distinguish the authority for a rules list, this can be either a specifc user or some server. For users we use the users email as unique identifier, for servers their hostname will suffice. Further we need to address the purpose or protocol for which a list is primary used and finally the list shall have a descriptive name.

This triple forms a uniqe identifier for any list.

Rxpd uses a simple line based text protocol. The first line is always the command and list which will be used on the following data, it is not possible to change the command throughout a session. Each session will generate at least one line of response. When no other output is available #OK: is send, in case of an error a line starting with #ERROR: is send.

Lines end with any combination of the newline and/or carriage return character.

The protocol is line based where lines which are longer than 4095 characters are broken (may be word-wraped on the last whitespace character in the line in future). Many commands take multiple lines as input, all this commands except CHECK: can be exited by sending a !EXIT statement.

Lists are autoloaded on demand and automatically saved when they are already exist on disk.

DUMP:list\n dump the content of list.

Contributions welcome! If anyone out there needs one of this features, drop me note and implement it when you dont wan’t to wait until I do it.