Q eQuational Programming System Version 7.11 Copyright (c) 1991-2008 by Albert Graef The Q programming system is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2, or (at your option) any later version. The Q programming system is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. (See the file "COPYING" included in the distribution for the GNU General Public License.) Author's address: Albert Graef Johannes Gutenberg-Universitaet Mainz Schmittpforte 11 Musikwissenschaftliches Institut 55599 Wonsheim/Germany Bereich Musikinformatik Dr.Graef@t-online.de 55099 Mainz/Germany ag@muwiinfa.geschichte.uni-mainz.de URL: http://www.musikinformatik.uni-mainz.de/ag ABOUT Q ===== = Q is a powerful and extensible programming language based on term rewriting which offers advanced symbolic processing and functional programming capabilities. Q's main features: - SIMPLE: Programs are just collections of equations which are used to evaluate expressions in a symbolic fashion. - POWERFUL: Despite its conceptual simplicity, Q is a full-featured functional programming language with a modern syntax, curried function applications, built-in support for lambda abstractions (as of Q 7.1), dynamic object- oriented typing including (as of Q 7.7) support for Wadler-style views, exception handling and POSIX style multithreading. As of version 7.0, Q also fully supports unicode environments. - EXTENSIBLE: Q has a libtool-based C interface which makes it easy to extend the interpreter with your own primitives. As of version 6.0, Q also supports SWIG, the "Simplified Wrapper and Interface Generator", which makes it fairly easy to wrap complex C/C++ libraries in Q modules. - EMBEDDABLE: Q can also be embedded in C/C++ programs, in order to employ Q as a macro language or term rewriting engine in your applications. - PORTABLE: Q runs on BeOS, FreeBSD, Linux, Mac OS X, Solaris and Windows. Porting to other modern Unix-based platforms should be a piece of cake. - FAST: As an interpreted language Q is certainly not as fast as native machine code, but it has an efficient interpreter which byte-compiles scripts in an eye blink and executes them about as fast as interpreted Lisp or Haskell. - EASY TO USE: Just throw together some equations, run the interpreter and start to evaluate expressions. The interpreter also includes a symbolic debugger which makes it easy to run your programs in a step-wise fashion. Q scripts can be run from the command line or within GNU Emacs. For Windows, a graphical IDE for editing and running Q scripts is also available. - BATTERIES INCLUDED: Q comes with a comprehensive standard library (written in Q itself) which provides complex and rational numbers, additional list processing functions, "streams" (a lazy variant of lists), container data structures (sets, dictionaries, etc.), and a PostScript interface. The distribution also includes a system interface written in C and other add-on modules for interfacing to various third-party tools and libraries, which makes Q a powerful tool for scientific programming, computer music, multimedia, and other advanced applications. The source distribution includes the Q programming tools, the standard library, a collection of useful add-on modules for interfacing to GNU Octave, GNU dbm, ODBC, Curl, GGI, ImageMagick, Tcl/Tk, XML/XSLT and IBM's Data Explorer, as well as Q language modes for Emacs, Vim and Kate. More packages in source and binary form and additional information can be found on the Q homepage at SourceForge: http://q-lang.sourceforge.net GETTING Q ========= As of December 2003, Q has become a SourceForge-hosted project, which can be found at the following URL: http://sourceforge.net/projects/q-lang/ There you can obtain released source and binary packages, as well as the latest and greatest development sources in CVS, under the following URLs. Download area: http://sourceforge.net/project/showfiles.php?group_id=96881 CVS access: http://sourceforge.net/cvs/?group_id=96881 Note that to build Q from the cvs sources you will first have to run the autogen.sh script which creates the configury. (You need autoconf, automake and libtool to do this, see MAINTAINER'S NOTE below.) Then you proceed to configure and compile the sources as explained under "INSTALLING FROM THE SOURCES" below. INSTALLATION ============ See the INSTALL file for generic (UNIX) installation instructions. This version has been tested (at least) on Linux and Windows, and should also work (with the usual amount of tweaking) on most other modern UNIX/POSIX-based platforms. Please report any patches required to build the package on other systems so that they can be included in the next release. Binary packages for Linux and Windows are available on the Q project website. These packages also contain the various add-on modules which are available, and are the most convenient way to get up and running quickly. NOTE: Many of the add-on modules require additional libraries to work, as detailed on the Q homepage. However, you only need to install these dependencies if you actually want to use the corresponding modules. The base system, consisting of interpreter and standard library including the clib module, only requires the C library and the GNU multiprecision library to work. If necessary, you can instruct the package installer to ignore other, unsatisfied dependencies (e.g., rpm may be run with the --nodeps option). INSTALLING FROM THE SOURCES --------------------------- If you have a Bourne-compatible shell, a POSIX-compatible C library, and either gcc (recommended) or another decent C compiler with ANSI C support, chances are that you will be able to build this package from source without too much hassle. You'll need the GNU multiprecision library version 3.0 or later. If this library is not available on your system, get it at http://www.swox.com/gmp/ and install it first. As with other GNU autotools-based packages, the basic compile/install procedure is as follows: ./configure && make && make install In this case configure picks a default set of optimization/debugging flags. More aggressive optimization can make a great difference for execution speed, though, so you might wish to try something like: CFLAGS=-O3 ./configure && make && make install If all is well, this will install Q in the default location (/usr/local); you can invoke the configure script with the --prefix option to change this. There are a number of other configure options to play with; run configure with the --help option for a list of the available options. You can also run a simple test suite with: make test This is by no means an exhaustive check of the interpreter's internals, but if any of these tests fails then there's probably a portability bug that needs to be fixed. You can also run this test before actually installing the software. The same build process should also work on Cygwin, BeOS and Mac OS X. See also the comments on specific systems below. Documentation is available in texinfo format, from which you can create both online and printed manuals in a variety of formats. The online manual in info format will be installed on your system. Furthermore, additional README files with information about the installed modules can be found in /share/ q/etc. [MAINTAINER'S NOTE: The current release was prepared using autoconf 2.61, automake 1.10 and libtool 1.5.23c. If necessary, you can always grab the Q cvs sources and run autogen.sh to bootstrap the configury yourself. Note that you only need these tools if you are building from cvs sources, or if you want to work on the configure scripts and automake Makefiles.] POST-INSTALL ------------ To complete the installation, you might wish to install Emacs Q mode, which provides auto-indentation and syntax highlighting of Q scripts in GNU Emacs and XEmacs, and also lets you run Q scripts right inside the editor. To these ends, copy the q-mode.el file under /share/q/etc to your Emacs site-lisp directory (or any directory searched for elisp files), and set up your .emacs file as described in Appendix E of the manual and at the beginning of the q-mode.el file. In the etc directory you also find syntax files to enjoy Q syntax highlighting in Vim (a popular vi clone) and the advanced KDE editor Kate. For Kate, just copy the q.xml file into your kde/share/apps/katepart/syntax directory and you're set. For Vim, copy q.vim to the ~/.vim/syntax directory and follow the instructions in the file to add the necessary entry to the filetype.vim file so that Vim recognizes the '.q' file type. TROUBLESHOOTING / KNOWN ISSUES --------------- - ----- ------ The interpreter doesn't work on 64 bit systems yet. This will be fixed "Real Soon Now"(TM). When compiling from the sources, the most common problem probably is that shared libraries and external modules refuse to build on your system. In this case, you can disable dynamic modules altogether with the --disable-shared configure option. This should work on *any* system supported by libtool, and you will still be able to use the modules bundled with the Q distribution since these will be linked directly into the interpreter. If you *can* build the modules successfully, you may still encounter problems loading them in the interpreter, due to bugs in libtool's dynamic module loader, libltdl. In this case, if you have a custom libltdl on your system, you can try to use that with the --with-installed-ltdl configure option. (Use this option with care. If you do not have libltdl on your system or configure cannot locate it, the bundled libltdl will be built and installed on your system, which may cause problems with other libltdl-based applications.) Moreover, recent libtool versions apparently have problems linking the Q executable on some systems (I noticed this on FreeBSD 5.1 and Mac OS X 10.2.4). I think that this is a libtool bug; libtool versions <=1.4.2 did not exhibit this behaviour. A workaround for this is to remove ltmain.sh and the libltdl directory and rebuild the configury with the autogen.sh script (you need to have a working autoconf/automake/libtool combo installed to do this). If configure fails to locate some required or optional third-party libraries on your system, you can specify --with-PKG=LIBS and --with-PKG-includes= INCLUDES to tell configure how to link these libraries (-L, -l etc.), and which additional includes (-I) are required to access the corresponding header files. Run configure --help to find out about the optional libraries and corresponding --with options which are supported. Only the gmp library is strictly necessary for building the package. Multithreading support will be enabled by default if the POSIX thread library is found, but can be disabled with the --without-pthread option. Likewise, unicode support is enabled by default if configure finds the necessary bits and pieces on your system, but can be disabled with --without-unicode. For instance, if your gmp library is in /root/lib and the gmp.h header file in /root/include, then you would configure with: ./configure --with-gmp="-L/root/lib -lgmp" --with-gmp-includes=-I/root/include SYSTEM-SPECIFIC NOTES ===================== BEOS ---- You'll need a working readline library to enjoy command line editing. Moreover, the module loader in the stock libltdl is broken. I suggest that you install the readline library from the GeekGadgets package and the fixed libtool package, both available from http://www.bebits.com. Then you can build the package with: ./configure --with-installed-ltdl Otherwise you'll have to live without dynamic loader support, and you should configure as follows: ./configure --disable-shared Note that currently there's no multithreading support, as there's no strictly POSIX-compatible thread library for BeOS. Also, BeOS support is pretty much in limbo since I don't run BeOS any more, and probably won't until one of the free replacements really takes off. CYGWIN ------ Q versions 7.1 and later have been reported to compile and install cleanly on recent Cygwin versions. Older Cygwin releases had issues with dynamic modules and POSIX threads, but these seem to be fixed now, so make sure you get a recent Cygwin version if you want to run Q on this platform. FREEBSD ------- (As of version 4.3.2, tested on FreeBSD 5.1-RELEASE. The same build procedure might work on the other BSDs as well, but I have not tested this.) Make sure you have a recent gmp library, as well as the libraries for the modules that you want (gdbm, odbc, tk, etc., all available in the ports tree). You'll also have to specify some compilation flags to enable the compiler to find things in /usr/local. If you want multithreading, add --with-pthread=-lc_r to the configure command. (Recent FreeBSD versions might now include a "real" pthread library, in this case you can omit this flag.) Sample configure command (omit the Tcl/Tk include paths and --with-tk option if you don't need the tk module): CFLAGS="-O3 -I/usr/local/include -I/usr/local/include/tcl8.4 \ -I/usr/local/include/tk8.4 -L/usr/local/lib" \ ./configure --with-tk="-ltcl84 -ltk84" --with-pthread=-lc_r LIBTOOL TROUBLES: If you run into trouble when linking the Q main executable, try removing ltmain.sh and libltdl from the main source directory, run autogen.sh to rebuild the configury (you need to have autoconf/automake/libtool installed to do this) and try again. NOTE: This port has not been tested for a while, so any feedback and patches needed to make Q work on this platform are appreciated. LINUX ----- Linux is the primary development platform for this software, and the sources should build out of the box on all recent Linux distributions. RPM packages for SUSE Linux are available on the Q project website. MAC OS X --- -- - As of version 4.2, the sources can be compiled with the gcc from the Mac OS X development pack. As reported by Andrew Berg and Marco Maggesi, to get a basic installation up and running on recent OSX systems you need to install the gmp and readline libraries. You can get these, e.g., from the Fink repository at http://finkproject.org (when using Fink, don't forget to also install the corresponding development packages). Specify the compiler include and library paths paths required to compile and link against the optional software (taking Fink as an example): C_INCLUDE_PATH=/sw/include LIBRARY_PATH=/sw/lib ./configure C_INCLUDE_PATH=/sw/include LIBRARY_PATH=/sw/lib make If you want to build from CVS sources you'll also need m4, autoconf and automake, also available from the Fink repository. More packages (such as X11, Tcl/Tk, etc.) will be needed if you want to build all the included modules. NOTES: - The gcc preprocessor on some older OSX versions seems to be broken, but this can be worked around using gcc's -no-cpp-precomp option. Add this to the CFLAGS variable in the configure command and you should be set. - Depending on the OSX version you might also run into problems when linking some modules or the Q main executable. This can usually be cured by rebuilding the configury from scratch as described in TROUBLESHOOTING above. - Q-Tk applications currently require that you have an X11-based Tcl/Tk installed and must be run in an X server, which is available from http://www.apple.com/macosx/x11/. Actually it should also be possible to build an X-less version which uses the native (Aqua) Tcl/Tk, but I haven't figured out how to do this yet; any input on this will be appreciated. To get an X11 version of Tcl/Tk running on your system, you'll have to install Tcl/Tk from source. First install the X11 server so that the necessary X11 libraries are available. Then get the Tcl/Tk 8.4 sources from http://www.tcl.tk and follow the instructions for Unix installation. (*Don't* use the OS X installation procedure, otherwise you will get the Aqua version.) SOLARIS ------- The sources should compile cleanly with gcc (versions up to 4.3.1 have been tested with gcc 2.95.2 under Solaris 2.8/x86). Configure as follows: ./configure --with-pthread="-lpthread -lrt" The -lrt option (-lposix4 on older Solaris systems) is needed to get the semaphore functions, if you're building with pthread support. NOTE: This port has not been tested for a while, so any feedback and patches needed to make Q work on this platform are appreciated. WINDOWS ------- A binary release for all current 32 bit Windows systems, the Qpad package, is available from the Q project website. It provides an MSI installer package, a user-friendly Windows GUI frontend to the interpreter, extensive online documentation including the full Q language manual in html help format, and of course precompiled binaries of the Q programming tools, libraries and modules. All required libraries and other support files are included in this package, except ImageMagick and Tcl/Tk, which you will have to install separately if you want to use the corresponding modules. You can find Windows installers for these libraries at the following URLs: - ImageMagick: http://www.imagemagick.org/www/archives.html - Tcl/Tk: http://aspn.activestate.com/ASPN/Downloads/ActiveTcl/ If you would like to use q-mode.el under Windows, you should get a Win32 version of Emacs which supports processes (and hence comint). Two nice Windows ports of Emacs are NT Emacs (see http://www.cs.washington.edu/homes/voelker/ ntemacs.html) and XEmacs (http://www.xemacs.org). After installing Qpad, you can find q-mode.el in Qpad\etc under your program files directory. Copy this file to your Emacs site-lisp directory, and set up your .emacs file as described in Appendix E of the manual. If you want to build the native Windows port yourself, you need a recent version of Mingw and Msys, both available at http://www.mingw.org. To build the entire distribution, you'll also need MS Visual C/C++ and a bunch of additional libraries. To ease porting, the Q project website has a zip file with the complete build tree used to create the current Windows release, including the Qpad sources, the add-on modules, and all dependencies except ImageMagick and Tcl/Tk. OPTIONAL SOFTWARE ======== ======== The Q distribution comes with some add-on modules which can be found in the `modules' directory of the Q source tree. `Clib', Q's "system" module which provides access to some important functions from the C library, is now an integrated part of the standard library, and is documented in the Q language manual. Some other modules are provided to access various useful third-party software. Documentation for these can be found in the appropriate README files inside the etc subdirectory of the Q installation directory after installation. Note that if you are building the Q core distribution from source then only those modules will be built for which the corresponding third party libraries are available on your system. CURL ---- The `curl' module gives access to libcurl (http://curl.haxx.se/), a powerful library for transferring files with URL syntax which supports all common Internet protocols. See README-Curl for more information. GDBM ---- Q's `gdbm' module provides an interface to the GNU dbm library, see gdbm(3). If you have a Linux system then most likely you already have this library, otherwise you can get it from http://www.gnu.org or one of its mirrors. This module is also supported on FreeBSD, OS X and Windows. See README-Gdbm for more information. GGI --- The `ggi' module provides access to the GGI (General Graphics Interface) library, available from http://www.ggi-project.org. This module allows you to create raster graphics on a variety of display devices. See README-GGI for more information. Please note that GGI itself doesn't provide any custom font support for text rendering, but Q's `ggi' module does. To make this work, you'll also need version 2 of the FreeType library, available from http://www.freetype.org/. GHOSTSCRIPT ----------- Q provides access to PostScript graphics via the `graphics' script contained in the standard library. This script also provides an interface to Ghostscript, a PostScript previewer which is available for a great variety of systems and can be obtained from http://www.ghostscript.com. IMAGEMAGICK ----------- The `magick' module lets you access most common image formats using the ImageMagick library, available from http://www.imagemagick.org. Using this module you can also, e.g., perform various image manipulation operations and render image files in a GGI visual. See README-Magick for more information. OCTAVE ------ The `octave' module provides access to John W. Eaton's Octave, a comprehensive MATLAB-like software for performing advanced numeric computations. To use this module, you must have Octave installed on your system, which is available from http://www.octave.org. You can also find some additional items including information about binary packages for Windows at the GNU Octave Repository (http://octave.sourceforge.net/). See README-Octave for more information. ODBC ---- ODBC has become the industry standard for portable and vendor independent database access. Q's ODBC interface lets you connect to ODBC-compatible databases and retrieve or modify data using SQL statements. To use this module, you need an ODBC driver manager on your system, as well as the database and corresponding ODBC driver you want to use. Two well-known open source ODBC implementations are iODBC (http://www.iodbc.org/) and unixODBC (http://www.unixodbc.org/). Two popular ODBC-compatible open source databases are MySQL (http://www.mysql.com/) and PostgreSQL (http://www.postgresql.org/). Under Windows you can also connect to various proprietary databases such as MS Access. See README-ODBC for more information. OPENDX ------ The `dxl' ("DX-Link") module provides an interface to IBM's Open Data Explorer, a powerful scientific data visualization software. This software needs Motif, and is only available for Unix and Linux systems at this time. (At least there is no straightforward way to make it run on other systems.) You can get it at http://www.opendx.org. See README-DXL for more information. SWIG ---- SWIG is the "Simplified Wrapper and Interface Generator" which allows you to create C/C++ wrapper modules for various target languages in an easy way. More information about SWIG can be found at http://www.swig.org. SWIG support is an important new feature which has been added in version 6.0 of the Q programming system. While a hand-crafted interface module allows you to tailor the interface to the Q environment, creating such a module by hand is often impractical for huge C/C++ libraries. This is where SWIG comes in handy. SWIG makes it really easy to wrap big libraries since the wrappers are generated automatically from simple interface definition files. Note that until the Q-SWIG module becomes part of the official SWIG distribution, to use SWIG with Q you'll need a SWIG version which has been patched up to add support for the Q language. For the time being, a suitable SWIG package can be found on the Q homepage. TCL/TK ------ The `tk' module provides an interface to Tcl/Tk, John Ousterhout's command language and graphical user interface toolkit. Using the `tk' module, you can employ Tcl/Tk to create functional if not fancy graphical user interfaces which are portable across most popular operating systems and windowing environments, including Unix/X11, Mac OS and MS Windows. See README-Tk for more information. Support for Gnocl, Peter G. Baum's Tcl->Gnome/GTK bridge (http://gnocl.sf.net), is also available as a separate package. XML/XSLT -------- XML (Extensible Markup Language, http://www.w3.org/TR/REC-xml) and XSLT (XML Stylesheet Language Transformations, http://www.w3.org/TR/xslt) are open standards by the W3C consortium. XML is used to represent structured data, XSLT to transform that data into other XML-based formats. XML is commonly employed to represent complex structured data in a way which facilitates data exchange across various applications and systems, while XSLT helps to present such structured data in any manner desired, e.g., as an HTML file. Q provides an interface to XML and XSLT via its `xml' module, which is implemented using the libxml2 and libxslt libraries from the GNOME project (http://xmlsoft.org). AND MORE ... --- ---- --- The auxiliary software listed above only encompasses the modules whose sources are distributed with the Q core package. Other interfaces such as the multimedia library (featuring modules to work with digital audio, signal processing, MIDI, 3D graphics and sound, etc.) and Qt/Q (Q's complete interface to Trolltech's Qt GUI toolkit) are available as separate source packages from the Q homepage. As of Q 7.0, the compiled versions of these modules are usually included in the binary packages of the Q programming system. New and/or experimental modules may also be provided as separate binary packages, or only in source form. Please check the Q homepage for more information about these add-ons. Q is also available as a plugin or module inside other environments. At the time of this writing, the available interfaces include the mod_q Apache module (which lets you run Q as a module inside the Apache web server), the Pd/Q external (which allows you to implement Pd control objects in Q) and John Cowan's Q interface for the Chicken Scheme compiler. The Apache and Pd/Q modules are available from the Q website, the Chicken module from http://www.call-with-current-continuation.org/. FEEDBACK ======== This software is provided free of charge and without any warranty, but that does *not* mean that I'm not fixing bugs or listening to new ideas. ;-) So your questions, comments, suggestions, contributions and, in particular, bug reports and patches are always welcome. I'd also like to hear about your experiences using Q, and which applications you use it for. To these ends, the SourceForge project website provides an area where you can post bug reports and patches, as well as support and feature requests. Moreover, the site also offers two mailing lists for discussing Q development and usage. Enjoy! June 2007 Albert Graef