Emacs Code

This page contains the Emacs lisp packages I develop and maintain (see below for more detailed descriptions):

Predictive Completion

The Emacs Predictive Completion package adds a new minor-mode to the GNU Emacs editor. When enabled, predictive mode exploits the redundancy inherent in languages in order to complete words you are typing before you've finished typing them (somewhat like the IntelliSense feature in some IDEs). It is highly customisable, and works happily alongside other Emacs major modes. See the documentation for more details.

Predictive mode only works under GNU Emacs, not under XEmacs. It may be possible to get it to work under XEmacs with a modicum of work. (At the very least, the overlay compatibility package would be required.) If you're interested in attempting this, then I'm happy to answer questions, but I have no plans to do it myself.

Documentation

Full documentation is available in a variety of formats, for download and viewing online.

If you're interested in using predictive mode, you should read the extensive Predictive User Manual.

The Predictive Programmer Manual contains low-level information for people who want to write their own lisp code to enhance predictive mode, for instance to make predictive completion work in other major modes. However, there is one reason why you might not want to read the programmer manual: it doesn't exist yet. Some of the information that would be in the programmer manual, were it to exist, is described in the auto-overlay manual (which does exist!), and the Completion-UI manual would contain further relevant information, were it to exist!

Predictive User Manual

Predictive Programmer Manual

Other sources of information

The Predictive Mode page on the Emacs Wiki contains various useful pieces of information about the Predictive Completion package. (And if you don't find the information you want, you can always add it yourself and then read it.)

Mathias Dahl has also posted a nice tutorial about using predictive mode on his blog.

Download and Installation

The current release of the Predictive Completion package is version 0.24 (released February 2013). It's still under active development, so don't forget to check back here for updates every so often. If you want to live on the bleeding edge, the latest "development" version of the Predictive package is hosted in a git repository.

Note that the git repository URL is a git repository, not a web-site. You cannot view it in a web browser. To grab the latest development version, clone the repository using something like:

     git clone http://www.dr-qubit.org/git/predictive.git

Whether you downloaded the package or cloned the git repository, you need to perform some further steps in order to install the Predictive Completion package on your system. See the Obtaining and Installing section of the manual and the INSTALL file included in the package for installation instructions.

Bugs and Feature Requests

Bugs, feature requests, and any other feedback should be addressed to toby-predictive@dr-qubit.org. (Note that this address is protected by the TMDA spam-reduction system. The first time you send an email, you will have to verify your address by replying to a confirmation message.)

If you have code you would like to contribute to the Predictive package, either send a patch against the latest development version to toby-predictive@dr-qubit.org, or, better still use git, rebase your work against the latest git master, and let me know where to find your clone of the Predictive repository so that I can pull your changes.

A list of known bugs and feature suggestions is included at the end of the user manual.

Undo Tree

Emacs has a powerful undo system. Unlike the standard undo/redo system in most software, it allows you to recover any past state of a buffer (whereas the standard undo/redo system can lose past states as soon as you redo). However, this power comes at a price: many people find Emacs' undo system confusing and difficult to use, spawning a number of packages that replace it with the less powerful but more intuitive undo/redo system. (See the Emacs Wiki.)

Both the loss of data with standard undo/redo, and the confusion of Emacs' undo, stem from trying to treat undo history as a linear sequence of changes. It's not. The undo-tree-mode provided by this package replaces Emacs' undo system with a system that treats undo history as what it is: a branching tree of changes. This simple idea allows the more intuitive behaviour of the standard undo/redo system to be combined with the power of never losing any history. An added side bonus is that undo history can in some cases be stored more efficiently, allowing more changes to accumulate before Emacs starts discarding history.

It gets better. You don't have to imagine the undo tree, because undo-tree-mode includes an undo-tree visualizer which draws it for you, and lets you browse around the undo history.

The only downside to this more advanced yet simpler undo system is that it was inspired by Vim. But, after all, most successful religions steal the best ideas from their competitors!

Documentation

Details of the undo-tree-mode commands and key bindings can be found in the Commentary section at the top of the undo-tree.el file itself, along with an extended explanation (with diagrams!) of the differences between the undo-tree-mode, standard Emacs' undo, and undo/redo systems.

The commands themselves are all documented using the standard, built-in Emacs documentation features. Customization options can be found under the undo-tree customization group.

Download and Installation

The current "stable" release of the Undo-Tree package is version 0.6.3 (released December 2012). It's still under active development, so don't forget to check back here for updates every so often. (A few older versions are also available below, but are no longer supported.)

In recent versions of Emacs (≥24.1), you can also install the Undo-Tree package from within Emacs itself, via GNU ELPA. Use M-x list-packages and take it from there. (The ELPA version occasionally lags slightly behind the latest version available here.)

If you want to live on the bleeding edge, the latest "development" version of the Undo-Tree package is hosted in a git repository.

Note that the git repository URL is a git repository, not a web-site. You cannot view it in a web browser. To grab the latest development version, clone the repository using something like:

     git clone http://www.dr-qubit.org/git/undo-tree.git

Regardless of where you obtained it, to install the Undo-Tree package simply save the undo-tree.el file to a directory in your Emacs load-path, and add the line:

   (require 'undo-tree)

to your .emacs file. "M-x undo-tree-mode" will then enable undo-tree-mode in the current buffer.

If you want to replace the standard Emacs' undo system with the Undo-Tree system in all normal buffers, you can enable undo-tree-mode globally by adding:

   (global-undo-tree-mode)

to your .emacs file.

Bugs, Feature Requests, and Contributing

Bugs, feature requests, and any other feedback should be addressed to toby-undo-tree@dr-qubit.org. (Note that this address is protected by the TMDA spam-reduction system. The first time you send an email, you will have to verify your address by replying to a confirmation message.)

If you have code you would like to contribute to Undo-Tree, either send a patch against the latest development version to toby-undo-tree@dr-qubit.org. Or, better still, use git, rebase your work against the latest git master, and let me know where to find your clone of the Undo-Tree repository so that I can pull your changes.

Completion User Interface

The Completion User Interface package is a library that implements user-interfaces for in-buffer completion.

Typically, a lot of code in packages providing some kind of text completion deals with the user interface. The goal of Completion-UI is to be the swiss-army knife of in-buffer completion user-interfaces, which any source of completions can plug in to, thus freeing completion package writers to concentrate on the task of finding the completions in the first place. In fact, Completion-UI is even better than a swiss-army knife, because it's also extensible: it's easy to add new user-interfaces, as well as new completion sources.

Various completion user-interfaces and commands are provided, which can be separately enabled, disabled and tweaked by the Emacs user via the usual bewildering array of customization variables:

Dynamic completion
Provisionally inserts the best completion candidate in the buffer, highlighting the completed portion.
Completion hotkeys
Single-key selection of a completion.
Cycling
Cycle through completion candidates.
Tab-completion
"Traditional" expansion to longest common substring.
Echo area
Display a list of completion candidates in the echo-area.
Tooltip
Display a list of completion candidates in a tool-tip located below the point.
Pop-up frame
Allow completion candidates to be selected from a pop-up frame displayed below the point.
Completion menu
Allow completion candidates to be selected from a drop-down menu located below the point.
Completion browser
Allow completion candidates to be selected from a hierarchically organised stack-of-cards menu structure.
auto-completion-mode
Automatically complete words as you type.

The philosophy of Completion-UI is that customization of the user-interface should be left up to users. They know what they want better than you do! And by providing a universal user-interface that can be used by all completion packages, Completion-UI lets users customize their in-buffer completion user interface once-and-for-all to suit their tastes, rather than having to learn how to customize each new package separately.

Completion-UI comes with built-in support for dabbrevs, etags, Elisp and file-name completion, as well as supporting (if installed) CEDET's Semantic completion, nxml-mode, and the predictive completion package.

For each source of completions, Completion-UI provides an interactive command that completes the word next to the point using that source, e.g: complete-dabbrev to complete using dabbrevs, complete-etags to complete using etags, complete-elisp to complete Elisp symbols, complete-files to complete file names, complete-semantic to use Semantic completion, complete-nxml and complete-predictive to use nxml-mode and predictive-mode completion, respectively.

The complete-<name> commands are not bound to any key by default. As with any Emacs command, you can run them via "M-x complete-<name>", or you can bind them to keys, either globally or in a minor-mode keymap. E.g. to globally bind "M-/" to complete-dabbrev, you would put the following line in your .emacs file:

     (global-set-key [?\M-/] 'complete-dabbrev)

To bind "M-" to complete-elisp in emacs-lisp-mode, you would bind the command in the emacs-lisp-mode-map keymap:

     (define-key emacs-lisp-mode-map [?\M-\t] 'complete-elisp)

You're free to bind the complete-<name> commands to any keys of your choosing, though "M-" or "M-/" fit best with the default Completion-UI key bindings that are enabled when you're completing a word. These are:

M-<tab> M-/
Cycle through completions.
M-S-<tab> M-?
Cycle backwards through completions.
C-<ret>
Accept the current completion.
C-<del>
Reject the current completion.
<tab>
Traditional tab-completion, i.e. insert longest common substring.
C-<tab>
Accept current completion and re-complete the resulting word.
S-<down>
Display the completion tooltip (then use <up> and <down> to cycle).
M-<down>
Display the completion menu.
C-<down>
Display the completion pop-up frame.
S-<up> C-<up> M-<up> (in pop-up frame)
Dismiss the completion pop-up frame.
M-/ (in pop-up frame)
Toggle between displaying all completions.

Documentation

If you're an Emacs user interested in using Completion-UI (as opposed to an Elisp programmer interesting in extending Completion-UI or making use of it in your own package), then the first section of the Completion-UI manual would contain all the information you could possible need...were it to exist. In the meantime, the description given above should get you started, and all the commands and customization options are reasonably well documented via the standard, built-in, Emacs documentation features. All the relevant customization options can be found in the completion-ui customization group (and subgroups thereof).

If you're interested in extending Completion-UI with a new user-interface or new completion source, or want to make use of Completion-UI in your own package, then full documentation will shortly be available in a variety of formats, for download and viewing online. In the meantime, the docstrings for completion-ui-register-interface and completion-ui-register-source should get you started.

Completion-UI Manual

Download and Installation

The current release of the Completion-UI package is version 0.12 (released February 2013). However, it's still under active development at the moment, so don't forget to check back here for updates every so often. If you want to live on the bleeding edge, the latest "development" version of the Completion-UI package is hosted in the same git repository as the Predictive Completion package.

Note that the git repository URL is a git repository, not a web-site. You cannot view it in a web browser. To grab the latest development version, clone the repository using something like:

     git clone http://www.dr-qubit.org/git/predictive.git

To install, simply extract the files to a directory in your Emacs load-path, and add the line:

   (require 'completion-ui)

to your .emacs file.

Automatic Overlays

The Automatic Overlays package allows you to define overlays that are created (and updated and destroyed) automatically when text in a buffer matches a regular expression.

Various classes of automatic overlay are provided, to make it easy to define matches for different text regions: words, lines, regions enclosed by start and end tags, or regions enclosed by delimiters. You can also define your own custom classes.

The overlays are updated just before any buffer modification. The built in overlay classes only update as much as is necessary to ensure that overlays covering the point are consistent. Therefore the overlays at the point are guaranteed to be correct before any buffer modification takes place there, but updating the overlays takes very little time and causes no noticeable delay.

Documentation

Full documentation is available in a variety of formats, for download or for viewing online. Since Auto-Overlays is an Elisp library, intended to be used by other Elisp packages, the documentation is aimed at Elisp programmers, not Emacs users.

Auto-Overlay Manual

Download and Installation

The current release of the Automatic Overlays package is version 0.10.8 (released February 2013). However, it's still under active development at the moment, so don't forget to check back here for updates every so often. If you want to live on the bleeding edge, the latest "development" version of the Automatic Overlays package is hosted in the same git repository as the Predictive Completion package.

Note that the git repository URL is a git repository, not a web-site. You cannot view it in a web browser. To grab the latest development version, clone the repository using something like:

     git clone http://www.dr-qubit.org/git/predictive.git

To install, simply extract the files to a directory in your Emacs load-path.

Data Structures

These packages provide basic (and not so basic) data structures.

Documentation

The functions these packages provide are well documented using Emacs' built-in documentation features. Brief descriptions of the data structures follow:

Heaps

A heap is a form of efficient self-sorting tree. In particular, the root node is guaranteed to be the highest-ranked entry in the tree. (The comparison function used for ranking the data can, of course, be freely defined). They are often used as priority queues, for scheduling tasks in order of importance, and for implementing efficient sorting algorithms (such as heap-sort).

Queues

A queue can be used both as a first-in last-out and as a first-in first-out stack, i.e. elements can be added to and removed from the front or back of the queue. (This library is an updated re-implementation of the old Elib queue library.)

Tagged Non-deterministic Finite state Automata

*Regexps are not regular expressions! Features of modern regexp implementations, including Emacs', mean they can recognise much more than regular languages. This comes with a big downside: matching regexps can be very inefficient.

A tagged, non-deterministic finite state automata (NFA) is an abstract computing machine that recognises regular languages. In layman's terms, they are used to decide whether a string matches a regular expression*. The "tagged" part lets the NFA do group-capture: it returns information about which parts of a string matched which subgroup of the regular expression. Why re-implement regular expression matching when Emacs comes with extensive built-in support for regexps? Primarily, because some algorithms require access to the NFA states produced part way through the regular expression matching process. Secondarily, because Emacs regexps only work on strings, whereas regular expressions can equally well be used to specify other sequence types.

Tries

The ternary search tree package has been obsoleted by the trie package. The former is no longer supported.

A trie stores data associated with "strings" (not necessarily the string data type; any ordered sequence of elements can be used). It stores them in such a way that both storage size and data lookup are reasonably space- and time- efficient, respectively. But, more importantly, advanced string queries are also very efficient, such as finding all strings with a given prefix, or finding all strings matching a regular expression, returning results in alphabetical order, or any other sort-order, or returning only the first few results, etc.

Dictionary trees

The dictionary tree data structures are a hybrid between tries and hash tables. Data is stored in a trie, but results that take particularly long to retrieve are cached in hash tables, which are automatically synchronised with the trie. The dictionary package provides persistent storage of the data structures in files, and many other convenience features.

Download and Installation

These packages are all relatively stable, though bug-fixes and new features are added occasionally. (Latest update: February 2013).

In recent versions of Emacs (≥24.1), you can also install all the non-obsolete packages from within Emacs itself, via GNU ELPA. Use M-x list-packages and take it from there. (The ELPA version might occasionally lag slightly behind the latest version available here.)

If you want to see if there are any bleeding-edge changes, the latest "development" versions of the data structure libraries are hosted in the same git repository as the Predictive Completion package.

  • Git repository: http://www.dr-qubit.org/git/predictive.git
  • heap.el (version 0.4)
  • queue.el (version 0.1)
  • avl-tree.el (already included in Emacs ≥ 24.1)
  • tNFA.el (version 0.1.1; requires queue.el)
  • tstree.el (obsolete! Use trie.el instead)
  • trie.el (version 0.2.6; requires everything except dict-tree.el)
  • dict-tree.el (version 0.12.8; requires everything else)

Note that the git repository URL is a git repository, not a web-site. You cannot view it in a web browser. To grab the latest development version, clone the repository using something like:

     git clone http://www.dr-qubit.org/git/predictive.git

To install, simply copy the files to a directory in your Emacs load-path.

Note that the avl-tree.el library is an enhanced version of the library of the same name that comes with Emacs, and is a drop-in replacement for that original version. If you don't want to overwrite that version, make sure you save this replacement version to a location that appears earlier in your Emacs load-path than the directory in which the original version is located (usually something like /usr/share/emacs/lisp/emacs-lisp/ on *nix systems).

You can view the current load path using C-h v load-path from within Emacs, and check that the new version shadows the original one by looking at the output of M-x list-load-path-shadows.

Recent versions of Emacs (≥24.1) already include this updated version of avl-tree.el.

Miscelaneous

These packages provide miscelaneous features I needed at some point. So I coded them. Currently, they're all to do with displaying useful information in the mode line.

show-point-mode displays the current value of the point in the mode line. I primarily find it useful when debugging Elisp code that uses overlays and markers.

wc-mode displays output similar to the Unix wc command in the mode line, i.e. the character count, word count and line count for the current buffer. (Useful when writing grant applications with character or word limits! Though I'm sure it's useful for other more useful activities, too...)