This is ../../doc/qdoc.info, produced by makeinfo version 4.8 from ../../doc/qdoc.texi. INFO-DIR-SECTION Programming Languages START-INFO-DIR-ENTRY * Q: (qdoc). The Q programming language and system. END-INFO-DIR-ENTRY The Q Programming Language, Version 7.11, 23 February 2008. Copyright (c) 1992-2008 by Albert Graef. This manual has been published in the series `Musikinformatik und Medientechnik', Bereich Musikinformatik, Musikwissenschaftliches Institut, Johannes Gutenberg-Universitaet Mainz 55099 Mainz Germany ISSN 0941-0309 The Q programming system is distributed under the terms of the GNU General Public License. See the software license accompanying the distribution for details. Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies. Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this manual into another language, under the above conditions for modified versions, except that this permission notice may be stated in a translation approved by the author.  File: qdoc.info, Node: Exception Handling, Next: Miscellaneous Functions, Prev: Lambda Abstractions, Up: Built-In Functions 10.7 Exception Handling ======================= As already mentioned, the Q language only knows a few "hard" runtime error conditions a.k.a. "exceptions" such as stack or memory overflow. However, "soft" exceptions can also be generated and handled in any desired manner. The following functions are used to deal with all kinds of exceptions during expression evaluation: `halt' halt evaluation `quit' exit the Q interpreter `break' invoke the debugger `fail' abort the current rule `_FAIL_' abort the current reduction `catch F X' handle an exception `throw X' raise an exception `trap ACT SIG' trap signals The `halt' function never returns a result, but raises an exception which normally causes the evaluation process to be aborted immediately. This last resort is commonly used in case of a fatal error condition. The `quit' function is like `halt', but also causes exit from the interpreter and returns you to the operating system shell; this operation is often used interactively to terminate a session with the interpreter. (Note that in a multithreaded script, see *Note POSIX Threads::, `quit' only terminates the program when it is invoked from the main thread. In other threads it just acts like `halt'.) If the `break' flag is `on' (see *Note Debugging::), the `break' function interrupts an evaluation and invokes the symbolic debugger built into the Q interpreter, just as if the user typed `Ctl-C'. This operation allows you to set breakpoints in a script. For instance, foo X = break || bar X; causes the debugger to be invoked as soon as the rule is executed. If the `break' flag is `off' then this operation has no effect. In any case, the `break' function returns `()'. The `fail' function is used to abort the current rule, just like a failing qualifier. The difference is that `fail' can be used anywhere in the right-hand side or qualifier of a rule, and causes the rule to be exited _immediately_. This allows you to handle complicated error conditions which occur while a rule is already being executed, and also provides an efficient method for implementing backtracking algorithms. For instance, here is a quick solution for the famous "N queens" problem: queens N = search N 1 1 []; search N I J P = write P || writes "\n" if I>N; = search N (I+1) 1 (P++[(I,J)]) || fail if safe (I,J) P; = search N I (J+1) P if J1; = bar (1/X) otherwise; The `catch' function allows you to handle both "hard exceptions" which are raised when the interpreter encounters one of the runtime error conditions discussed in *Note Error Handling::, and "soft exceptions" which are raised by the functions `halt' and `quit' already explained above, and the `throw' function which is discussed below. Moreover, the interpreter also handles certain signals sent to it, e.g., via the `kill'(2) system call, by raising appropriate exceptions. In the current implementation, by default only the signals `SIGINT' ("break"), `SIGTERM' ("terminate") and `SIGHUP' ("hangup") are handled; the latter two are both treated as termination requests, like a call to the `quit' function. Signal handlers can also be installed by the running script using the `trap' function discussed below. Both arguments of `catch' are special. The `catch' function first evaluates its second argument, the "target expression" and, if all is well, returns the value of that expression. Otherwise, if any exception was raised during the evaluation of the target expression, it evaluates the first argument (the "exception handler"), applies it to the value of the exception, and returns the result computed by the exception handler. In the case of a runtime error condition and the exceptions raised with the `halt' and `quit' functions, the "exception value" is of the form `syserr N' where `N' is one of the integer error codes listed below: * 1: "Break": break signal, user typed `Ctl-C'. * 2: "Halt": invocation of the `halt' function, or request to halt evaluation from the debugger. * 3: "Quit": termination signal, invocation of the `quit' function, or request to exit the interpreter from the debugger. * 4: "Memory overflow": the attempt to allocate memory for the stack or a new expression on the heap fails. * 5, 6: "Stack overflow": the attempt to push an expression on the stack, or the activation of a rule fails because the stack size limit has been reached (see the comments on memory management in *Note Running Compiler and Interpreter::). 5 signals an expression stack overflow, 6 an overflow of the rule stack. * 7: "Symbol table overflow": the attempt to create a new variable symbol fails. * 8: "Conditional error": the qualifying condition of a rule does not evaluate to a truth value. * 9: "External function error": an external function (see *Note C Language Interface::) signals an error condition. Let's try this by forcing a stack overflow condition: ==> iter 100000000 (+) 1 ! Stack overflow >>> iter 100000000 (+) 1 ^ ==> catch exception (iter 100000000 (+) 1) exception (syserr 5) The `syserr' constructor belongs to the built-in `SysException' type, which in turn is a subtype of `Exception'. These types may be thought of as being predefined as follows: public type Exception; public type SysException : Exception = const syserr N; User-defined exceptions can be generated with `throw'. The `throw' function raises an exception whose value is given by its single (non-special) argument. For instance: hd [] = throw '(hd []); exception X = writes "Exception: " || write X || writes "\n"; With these definitions we have: ==> catch exception (hd [1..3]) 1 ==> catch exception (hd [1..0]) Exception: '(hd []) () ==> hd [1..0] ! Exception '(hd []) >>> hd [1..0] ^ Note that if an exception raised with `throw' is not handled with a corresponding `catch', then the interpreter aborts the evaluation and prints an error message (and it will also invoke the debugger to report the rule which generated the exception, if `break' is `on'). The `catch' and `throw' functions can also be used to implement non-local value returns. For instance, here is a variation of the N queens function which returns the first solution as soon as it has been found during backtracking: queens1 N = catch id (search1 N 1 1 []); search1 N I J P = throw P if I>N; = search1 N (I+1) 1 (P++[(I,J)]) || fail if safe (I,J) P; = search1 N I (J+1) P if J', `<=' and `>=' operators with rules for deciding equality and inequality of lists, streams and tuples, and rules for ordering lists and streams lexicographically. That is, tuples, lists and streams can be compared with `=' and `<>', which is done by checking that the operands are of the same size and recursively comparing all members of the operands. Lists and streams can also be compared "lexicographically" by recursively comparing the list or stream members; the first unequal members decide the comparison. This works just like the lexicographic comparison of strings. Thus, e.g., `[1,2]' is considered to be less than both `[1,3]' and `[1,2,1]', but more than `[1,1,3]' or `[0,5]'. Moreover, the prelude defines the syntactic inequality operator `!=' as the logical negation of the built-in `==' operator, cf. *Note Non-Linear Equations::. Like `==', `!=' is implemented as a special form. Last but not least, the standard prelude also provides default definitions for arithmetic, relational, numeric and conversion functions on `Real', so that a minimal implementation of a user-defined type derived from `Real' (or any of its subtypes) just needs to define the `float' function (which is needed to coerce values of the type to `Float') and then add definitions for those operations which need special treatment. The default definitions in `prelude.q' are at the lowest priority level so that they can always be overridden if needed. The `stdlib.q' script has those operations which will probably be used most frequently by the average programmer; it contains a lot of additional list processing functions and other useful stuff mostly adopted from [Bird/Wadler 1988]. This module is also included by many other library scripts. Historically, it was the first script written for the standard library. The `stream.q' script extends most list operations defined in `stdlib.q' to streams, i.e., lazy lists. It also provides a number of other useful stream operations. Similarly, the `string.q' and `tuple.q' scripts provide additional operations on strings and tuples, and also extend some list operations to these data types. The `stdtypes.q' script simply includes all modules which implement common container data structures; currently these are `array.q', `bag.q', `dict.q', `hdict.q', `heap.q' and `set.q', see *Note Standard Types::. Some declarations shared by these modules are in the `stddecls.q' script. The `complex.q' and `rational.q' scripts implement complex and rational number types along with the usual operations on these types. These have been designed to integrate seamlessly into Q's system of built-in number types so that expressions involving arithmetic operations with any combination of integer, floating point, complex and rational arguments work as expected. The `clib.q' and `system.q' modules give access to various useful system functions from the C library, including C-style formatted I/O, binary file I/O, process and thread management, internationalization support and regular expression routines. The `clib.q' module also provides mutable expression cells ("references"), as well as a "byte string" data type which lets you represent arbitrary C data, and it overrides various important standard library functions with much faster C versions. The operations of these modules are mostly written in C, using the C language interface discussed in *Note C Language Interface::. These modules are described in their own chapter, see *Note Clib::. As of Q 7.8, the `reftypes.q' module provides additional convenience operations to work with mutable expression sequences implemented as tuples, lists or streams of references. These are also described in the "Clib" chapter, see *Note Expression References::. The Q programming system comes bundled with some additional modules for interfacing to various third-party libraries and software packages. At the time of this writing, these are: * Curl, a library for transferring files with URL syntax * GNU dbm, a simple database library; * ODBC, the industry standard database interface; * Octave, a popular MATLAB-like open source software for doing numeric computations; * GGI, a portable raster graphics interface; * ImageMagick, a comprehensive image manipulation software; * Tcl/Tk, a portable user interface toolkit and scripting language; * IBM's Data Explorer, a scientific visualization software. Preliminary documentation for these modules can be found in the `etc' subdirectory of your Q installation directory (usually in `/usr/share/q' or `/usr/local/share/q'). Other useful add-on modules are available as separate source packages. At the time of this writing this comprises additional GUI libraries (including a complete interface to Trolltech's very nice "Qt" toolkit), a bunch of graphics and multimedia modules (including interfaces to OpenGL, OpenAL, the Xine video library, and Grame's MidiShare and Faust) and more. Ready-made plugins for using the Q interpreter from the "Chicken" Scheme compiler, the Apache webserver and Miller Puckette's graphical computer music and multimedia environment "PureData" are also available. Please check out the Q website at `http://q-lang.sourceforge.net' for more information about these. (The Q module for Chicken, which was written by John Cowan, is available separately from the Chicken website, see `http://www.call-with-current-continuation.org'.) * Menu: * Standard Functions:: * Tuple Functions:: * String Functions:: * Option Parsing:: * Type-Checking Predicates:: * Sorting Algorithms:: * Standard Types:: * Streams:: * Conditionals and Comprehensions:: * Mathematical Functions:: * Complex Numbers:: * Rational Numbers:: * Graphics:: * Diagnostics and Error Messages::  File: qdoc.info, Node: Standard Functions, Next: Tuple Functions, Prev: The Standard Library, Up: The Standard Library 11.1 Standard Functions ======================= The `stdlib.q' script provides frequently used list operations and other stuff mostly adopted from [Bird/Wadler 1988]. `abs X' absolute value of X `all P Xs' verify that each element of the list `Xs' satisfies the predicate `P' `any P Xs' verify that the list `Xs' contains an element satisfying predicate `P' `append Xs X' append a value `X' to the list `Xs' `cat Xs' concatenate a list of lists `cons X Xs' prepend an element to a list `cst X' constant-valued function: `cst X Y => X' `curry F' curry a function: turn a function operating on pairs into a function with two arguments `curry3 F' `curry' with three arguments `do F Xs' apply a function `F' to each member of a list `Xs', return `()' `dowith F Xs Ys' take two lists and apply a binary function to corresponding elements, return `()' `dowith3 F Xs Ys Zs' `dowith' with three lists `drop N Xs' remove the first `N' elements from the list `Xs' `dropwhile P Xs' remove elements from the beginning of `Xs' while the predicate `P' is satisfied `eq X Y' syntactic equality (cf. *Note Non-Linear Equations::) `filter P Xs' filter a list with a predicate `foldl F A Xs' fold-left `foldl1 F Xs' fold-left over nonempty lists `foldr F A Xs' fold-right `foldr1 F Xs' fold-right over nonempty lists `hd Xs' return the head element of a list `hds Xs' return the list of all head elements in a list of lists `id' the identity function: `id X => X' `init Xs' return list `Xs' without its last element `iter N F A' generate the list of the first `N' values `A', `F A', `F (F A)', ... `last Xs' return the last element of a list `map F Xs' apply function `F' to each member of a list `max X Y' maximum of two values `min X Y' minimum of two values `mklist X N' create a list of `N' `X''s `neg P' negate a predicate `neq X Y' syntactic inequality `null Xs' check whether a list is empty (`[]') `nums N M' generate a list of numbers in a given range `numsby K N M' generate a list of numbers with a given step size `pop Xs' remove the head element from a list `prd Xs' product of a list of numbers `push Xs X' prepend an element to a list (`cons' with arguments reversed) `reverse Xs' reverse a list `scanl F A Xs' apply `foldl' to every initial part of a list `scanl1 F Xs' apply `foldl1' to every nonempty initial part of a list `scanr F A Xs' apply `foldr' to every final part of a list `scanr1 F Xs' apply `foldr1' to every nonempty final part of a list `sgn X' sign of a number `sum Xs' sum of a list of numbers `take N Xs' select the first `N' elements from the list `Xs' `takewhile P Xs' select elements from the beginning of `Xs' while the predicate `P' is satisfied `tl Xs' remove the head element from a list `tls Xs' return a list of lists with all head elements removed `top Xs' return the head element from a list `transpose Xs' transpose a list of lists `uncurry F' uncurry a function: turn a function with two arguments into a function operating on pairs `uncurry3 F' `uncurry' with triples `until P F X' repeat applying `F' to `X' until `P' is satisfied `unzip Xs' transform a list of pairs into a pair of lists `unzip3 Xs' `unzip' with triples `while P F A' list repeated applications of `F' to `A' while `P' is satisfied `zip Xs Ys' take two lists and return a list of corresponding pairs `zip3 Xs Ys Zs' `zip' with three lists `zipwith F Xs Ys' take two lists and map a binary function to corresponding elements `zipwith3 F Xs Ys Zs' `zipwith' with three lists Note that some of these functions are actually overridden with much faster C versions in the `clib.q' module; see *Note C Replacements for Common Standard Library Functions::.  File: qdoc.info, Node: Tuple Functions, Next: String Functions, Prev: Standard Functions, Up: The Standard Library 11.2 Tuple Functions ==================== The `tuple.q' script provides some additional functions operating on tuples: `fst Xs' return first element of a tuple `mktuple X N' create a tuple of `N' `X''s `pair X Y' construct a pair `snd Xs' return second element of a tuple `trd Xs' return third element of a tuple `triple X Y Z' construct a triple `tuplecat Xs' concatenate a list of tuples Moreover, the following list functions from `stdlib.q' are overloaded to also work on tuples: `append', `cat', `cons', `do', `dowith', `dowith3', `map', `null', `pop', `push', `reverse', `top'. Most of these return tuples when applied to such arguments, except `null' (which returns a truth value), the `do'/`dowith' functions (which return `()') and `cat' (which always returns a list; use `tuplecat' if you want to concatenate a collection of tuples instead). Also note that the `reverse' and `tuplecat' operations are actually overridden with much faster C versions in the `clib.q' module; see *Note C Replacements for Common Standard Library Functions::.  File: qdoc.info, Node: String Functions, Next: Option Parsing, Prev: Tuple Functions, Up: The Standard Library 11.3 String Functions ===================== The `string.q' script provides a collection of additional string functions. Currently the following operations are implemented (most of these operations are actually overridden with much more efficient C implementations in the `clib.q' module; see *Note C Replacements for Common Standard Library Functions::): `chars S' return the list of individual characters in `S' `join DELIM Xs' concatenate a list of strings, interpolating the given `DELIM' string between each pair of consecutive strings in the list `mkstr S N' create a string consisting of `N' copies of the given string `S' `split DELIM S' split a string into a list of substrings delimited by characters in the given `DELIM' string `strcat Xs' concatenate a list of strings Moreover, as of Q 7.8 this module overloads all list operations in `stdlib.q' so that they work on strings as expected. This lets you use strings mostly as if they were just another kind of list value. For instance: ==> take 3 "abcdef" "abc" ==> dropwhile (<="c") "abcdef" "def" ==> tl "abcdef" "bcdef" ==> zip "abcdef" "ABCDEF" [("a","A"),("b","B"),("c","C"),("d","D"),("e","E"),("f","F")] ==> map (+1) "HAL" "IBM" ==> map ord "HAL" [72,65,76] ==> [C-1 : C in "IBM"] ["H","A","L"] (As the last example shows, strings can also serve as sources in the binding clauses of list comprehensions; cf. *Note Conditionals and Comprehensions::.) Here's a somewhat more practical example which illustrates the use of various list operations on strings to build a ROT13 translation table: import dict; def CHARS = strcat ["a".."z"], RCHARS = drop 13 CHARS ++ take 13 CHARS, ROT13 = dict $ zip (CHARS++toupper CHARS) (RCHARS++toupper RCHARS); rot13 C:Char = C where C:Char = ROT13!C; = C otherwise; rot13 S:String = map rot13 S; (This employs the `dict' function to create a dictionary mapping lower- and uppercase letters to their equivalents in the ROT13 encoding; see *Note Dictionaries::.) Example: ==> CHARS; RCHARS "abcdefghijklmnopqrstuvwxyz" "nopqrstuvwxyzabcdefghijklm" ==> rot13 "This is an encoded string." "Guvf vf na rapbqrq fgevat." ==> rot13 _ "This is an encoded string."  File: qdoc.info, Node: Option Parsing, Next: Type-Checking Predicates, Prev: String Functions, Up: The Standard Library 11.4 Option Parsing =================== This module provides one function, `getopt', which takes two arguments: `OPTS', a list of option descriptions in the format described below, and `ARGS', a list of strings containing the command line parameters to be parsed for options. The result is a pair `(OPTS,ARGS)' where `OPTS' is a list of pairs of options and their arguments (if any; missing arguments are returned as `()'), and `ARGS' is the list of remaining (non-option) arguments. Options are parsed using the rules of GNU `getopt'(1). If an invalid option is encountered (unrecognized option, missing or extra argument, etc.), `getopt' throws the offending option string as an exception. The `OPTS' argument of getopt is a list of triples `(LONG,SHORT,FLAG)', where `LONG' denotes the long option, `SHORT' the equivalent short option, and `FLAG' is one of the symbolic integer values `NOARG', `OPTARG' and `REQARG' which specifies whether the option has no argument, an optional argument or a required argument, respectively. In the returned option-value list, all options will be represented using their long option equivalents. Also note that both the long and short option values in the `OPTS' argument may actually be of any type, so unneeded options may be replaced with corresponding dummy values. You only have to make sure that the values given for the long options allow you to identify which option was actually specified. Finally, please note that, as already pointed out, this script does _not_ belong to the prelude, and hence has to be imported explicitly if you want to use the `getopt' operation in your program. Example (see *Note C-Style Formatted I/O::, for a description of `printf' and `fprintf'): import getopt; def OPTS = [("--help", "-h", NOARG), ("--foo", "-f", REQARG), ("--bar", "-b", OPTARG)]; invalid_option PROG OPT = fprintf ERROR "%s: invalid option %s\n" (PROG,OPT) || halt; test [PROG|ARGS] = printf "OPTS = %s\nARGS = %s\n" (str OPTS, str ARGS) where (OPTS,ARGS) = catch (invalid_option PROG) $ getopt OPTS ARGS;  File: qdoc.info, Node: Type-Checking Predicates, Next: Sorting Algorithms, Prev: Option Parsing, Up: The Standard Library 11.5 Type-Checking Predicates ============================= The `typec.q' script contains a collection of predicates which check whether an expression is of a given type. The following functions perform a purely syntactic check; they are all equivalent to the corresponding type guards: `isbool X' check for truth values `ischar X' check for single character strings `iscomplex X' check for complex numbers (cf. *Note Complex Numbers::) `isexcept X' check for `Exception' values (cf. *Note Exception Handling::) `isfile X' check for file objects `isfloat X' check for floating point numbers `isfunction X' check for lambda functions `isint X' check for integers `islist X' check for lists `isnum X' check for numbers `isrational X' check for rational numbers (cf. *Note Rational Numbers::) `isreal X' check for real numbers `isstr X' check for strings `issym X' check for function and variable symbols (special form) `istuple X' check for tuples Note that the syntactic number predicates `isint', `isfloat' etc. only check for a given representation. The `typec.q' module also provides various predicates which classify numbers according to the kind of abstract mathematical object they represent, regardless of the concrete representation: `iscompval X' check for complex values `isintval X' check for integer values `isratval X' check for rational values `isrealval X' check for real values These predicates work mostly like their Scheme counterparts. E.g., `3' will be classified not only as an integer value, but also as a rational, real and a complex value. A complex number with zero imaginary part may also be classified as an integer, rational or real, depending on its real part. A floating point number with zero fractional part and a rational number with a denominator of `1' are both also classified as an integer value. Moreover, the following predicates allow you to check whether a number is exact or inexact (inexact numbers generally involve floating point values), and whether a number represents an IEEE infinity or NaN ("not a number") value: `isexact X' check for exact numbers `isinexact X' check for inexact numbers `isinf X' check for infinite floating point values `isnan X' check for NaN floating point values Finally, two other special-purpose predicates provided in `typec.q' are: `isenum X' check for enumeration type members `issym X' check for function and variable symbols (special form)  File: qdoc.info, Node: Sorting Algorithms, Next: Standard Types, Prev: Type-Checking Predicates, Up: The Standard Library 11.6 Sorting Algorithms ======================= The `sort.q' script provides mergesort and quicksort algorithms for sorting a list using a given order predicate: `msort P Xs' mergesort algorithm `qsort P Xs' quicksort algorithm The mergesort algorithm is more involved than quicksort, but may run _much_ faster if input lists are large enough and are already partially sorted. Both algorithms take an order predicate as their first argument, which makes it possible to sort lists using different criteria. The order predicate must be a function accepting two arguments, and must return `true' iff the first argument is strictly less than the second. For instance, to sort a list in ascending order, you could say: ==> qsort (<) [1,5,3,2,4] [1,2,3,4,5] By reversing the order predicate, the list is sorted in descending order: ==> qsort (>) [1,5,3,2,4] [5,4,3,2,1] Custom order predicates also allow you to sort a list according to different sort keys. For instance, the following example shows how to sort a list of pairs using the first component of each pair as the sort key: ==> def L = [(1,2),(5,1),(3,3),(1,1),(4,5)] ==> var le1 = \X Y.fst X < fst Y ==> qsort le1 L [(1,2),(1,1),(3,3),(4,5),(5,1)] Both algorithms provided by this module are "stable", i.e., they preserve the relative order of list elements with "equal" sort keys. This is important when successive sorts are used to order elements according to different criteria. For instance: ==> var le2 = \X Y.snd X < snd Y ==> qsort le2 L [(5,1),(1,1),(1,2),(3,3),(4,5)] ==> qsort le1 _ [(1,1),(1,2),(3,3),(4,5),(5,1)]  File: qdoc.info, Node: Standard Types, Next: Streams, Prev: Sorting Algorithms, Up: The Standard Library 11.7 Standard Types =================== Please note that, as of Q 7.8 this module is no longer included in the prelude, so you have to import it explicitly if your script needs one of the data types described here. It is also possible to just import the individual data type modules (`array.q', `bag.q', etc.) if you do not need the entire collection. The `stdtypes.q' script implements a collection of efficient container data structures, which currently comprises arrays, heaps (priority queues), ordered sets and bags, and (ordered as well as hashed) dictionaries. The different data types are actually implemented in the scripts `array.q', `bag.q', `dict.q', `hdict.q', `heap.q' and `set.q'. Many operations of these modules are overloaded; the declarations of these operations can be found in the `stddecl.q' script which is included by the different modules. All data structures support equality checking with `=' and `<>', as well as the operation `#' to determine the size of an object (number of elements it contains). Furthermore, the set and bag data structures overload the `<', `>', `<=' and `>=' operators to implement subset/subbag comparisons, and the `+', `-' and `*' operators to compute the union, difference and intersection of two sets or bags, respectively. For convenience, default views are provided for all container data structures, so that they will be printed in the form of a construction function (which is implemented as a virtual constructor of the type, cf. *Note Views::) applied to a member list (such as, e.g., `set [1,2,3]' in the case of `Set', or `dict [("a",1),("b",2),("c",3)]' in the case of `Dict'). These are at a low priority, so you can easily override them when needed. * Menu: * Arrays:: * Heaps:: * Sets:: * Bags:: * Dictionaries:: * Hashed Dictionaries::  File: qdoc.info, Node: Arrays, Next: Heaps, Prev: Standard Types, Up: Standard Types 11.7.1 Arrays ------------- The `array.q' script provides a zero-based array data structure `Array' with logarithmic access times, implemented as size-balanced binary trees. The following operations are provided: `array Xs' create an array from list `Xs' `array2 Xs' create a two-dimensional array from a list of lists `emptyarray' return the empty array `mkarray X N' create an array consisting of `N' `X''s `mkarray2 X (N,M)' create a two-dimensional array with `N' rows and `M' columns `isarray X' check whether `X' is an array `null A' check whether `A' is the empty array `A1 = A2, A1 <> A2' array equality/inequality `#A' size of an array `A!I' return `I'th member of an array `A!(I,J)' two-dimensional subscript `members A, list A' list the members of `A' `members2 A, list2 A' list a two-dimensional array `first A, last A' return the first and last element of an array `rmfirst A, rmlast A' remove the first and last element from an array `insert A X' insert `X' at the beginning of `A' `append A X' append `X' at the end of `A' `update A I X' replace the `I'th member of `A' by `X' `update2 A (I,J) X' update two-dimensional array  File: qdoc.info, Node: Heaps, Next: Sets, Prev: Arrays, Up: Standard Types 11.7.2 Heaps ------------ The `heap.q' script provides an efficient heap (priority queue) data structure `Heap' implemented as size-balanced binary trees. Heaps allow quick (i.e., constant-time) access to the smallest element, and to insert new elements in logarithmic time. The present implementation does not allow fast random updates of heap members; if such functionality is required, bags should be used instead (see *Note Bags::). Heap members must be ordered by the `<=' predicate. Multiple instances of the same element may be stored in a heap; however, the order in which equal elements are retrieved is not specified. The following operations are provided: `emptyheap' return the empty heap `heap Xs' construct a heap from a list of its members `isheap X' determine whether `X' is a heap `null H' check whether `H' is the empty heap `H1 = H2, H1 <> H2' heap equality/inequality `#H' size of a heap `members H, list H' list the members of `H' in ascending order `first H' return the first element of `H' `rmfirst H' remove the first element from `H' `insert H X' insert `X' into `H'  File: qdoc.info, Node: Sets, Next: Bags, Prev: Heaps, Up: Standard Types 11.7.3 Sets ----------- The `set.q' script provides an ordered set data structure `Set' implemented as AVL trees, and thus guaranteeing logarithmic access times. The following operations are defined in `set.q': `emptyset' return the empty set `set Xs' create a set from a list of its members `isset X' check whether `X' is a set `null M' check whether `M' is the empty set `member M X' check whether `M' contains element `X' `M1 = M2, M1 <> M2' set equality/inequality `M1 < M2, M1 > M2, M1 <= M2, M1 >= M2' set comparison `M1 + M2, M1 - M2, M1 * M2' set union, difference and intersection `#M' size of a set `members M, list M' list the members of `M' in ascending order `first M, last M' return the first and last member of `M' `rmfirst M, rmlast M' remove the first and last member from `M' `insert M X' insert `X' into `M' `delete M X' delete `X' from `M'  File: qdoc.info, Node: Bags, Next: Dictionaries, Prev: Sets, Up: Standard Types 11.7.4 Bags ----------- The `bag.q' script defines the type `Bag' as a variant of the set data structure which may contain multiple instances of the same element. The operations are analogous to those of the set data structure; see *Note Sets::. The `emptybag' function returns the empty bag, and `bag Xs' constructs a bag from a list `Xs' of its members. The `isbag' predicate checks whether its argument is a bag.  File: qdoc.info, Node: Dictionaries, Next: Hashed Dictionaries, Prev: Bags, Up: Standard Types 11.7.5 Dictionaries ------------------- The `dict.q' script supplies an (ordered) dictionary data structure `Dict' which maps "keys" from an ordered set to corresponding "values". Like sets and bags, dictionaries are implemented as AVL trees, thus guaranteeing logarithmic access times to individual members of the dictionary. The following operations are defined in `dict.q': `emptydict' return the empty dictionary `dict XYs' create a dictionary from a list of key/value pairs `mkdict Y Xs' create a dictionary from a list of keys and an initial value `isdict X' check whether X is a dictionary `null D' check whether `D' is the empty dictionary `member D X' check whether `D' contains `X' as a key `D1 = D2, D1 <> D2' dictionary equality/inequality `#D' size of a dictionary `D!X' return the value `Y' associated with `X' in `D' `members D, list D' list the members (key/value pairs) of `D' in ascending order by key `keys D' list the keys of `D' in ascending order `vals D' list the corresponding values `first D, last D' return the first and last member of `D' `rmfirst D, rmlast D' remove the first and last member from `D' `insert D (X,Y)' insert a key/value pair `(X,Y)' into `D'; update an existing entry for `X' if present `delete D X' remove key `X' from `D' `update D X Y' same as `insert D (X,Y)'  File: qdoc.info, Node: Hashed Dictionaries, Prev: Dictionaries, Up: Standard Types 11.7.6 Hashed Dictionaries -------------------------- The `hdict.q' script implements hashed dictionaries (`HDict' type), a variation of the `Dict' type which uses hashed key values obtained with the built-in `hash' function. The actual key-value pairs are stored in "buckets" for each hash value. This kind of data structure is also known as "hashes" or "associative arrays" in other programming languages. The main advantage of the `HDict' type is that key values can be of any type and do not have to belong to an ordered set. For instance, `hdict [(0,1),(foo,2),("bar",3)]' is a legal `HDict' value which maps the integer `0' to `1', the symbol `foo' to `2', and the string `"bar"' to `3'. There are some other notable differences between the `Dict' and the `HDict' type. First of all, a `HDict' stores it members in an apparently random order which may depend on the order in which new entries are added to the dictionary. Hence equality testing for `HDict's is more involved than for `Dict's, as the member lists of two "equal" `HDict's may be arbitrary permutations of each other. This also means that the `first', `rmfirst', `last' and `rmlast' operations do not make much sense with hashed dictionaries and are not supported by the `HDict' type. Moreover, key values are always compared syntactically when looking up and updating entries. Hence, e.g., `0' and `0.0' are different key values in a `HDict' object, whereas they are considered to be the same for the `Dict' type. Apart from these differences, the operations of the `HDict' and `Dict' types work analogously. The `HDict' constructors are named `emptyhdict', `hdict' and `mkhdict' which take the same arguments as the corresponding `Dict' constructors, and the `ishdict' predicate checks for `HDict' values.  File: qdoc.info, Node: Streams, Next: Conditionals and Comprehensions, Prev: Standard Types, Up: The Standard Library 11.8 Streams ============ The `stream.q' script provides operations on streams, Q's lazy list data structure. Note that, as of Q 7.1, streams are now provided as a built-in data type, which is declared as follows: public type Stream = special const nil_stream, cons_stream X Xs; As explained in *Note Lists Streams and Tuples::, the Q language now also provides syntactic sugar for these structures, so that streams can be denoted just like lists, using curly braces instead of brackets. The `stream.q' module overloads most list operations so that they work on streams in a completely analogous fashion, and provides the following additional operations: `isstream X' check whether an object is a stream `stream Xs' convert a tuple or a list to a stream `list Xs' convert a stream to a list `strict Xs' force the heads and tails of a stream (see comments below) `lazy Xs' memoize the heads and tails of a stream (see comments below) `numstream N' generate the stream of all numbers `>=N' `numstreamby K N' generate a number stream with given step size `mkstream X' generate an infinite stream of `X''s `repeat X' generate an infinite stream of `X''s (works like `mkstream', but is implemented as a special form and hence doesn't evaluate `X') `repeatn ~N X' generate a stream of `N' `X''s (special form, equivalent to `take N (repeat X)') `cycle Xs' generate an infinite stream which repeatedly cycles through the members of the list or stream `Xs' `iterate F A' generate the stream of all values `A', `F A', `F (F A)', ... `streamcat Xs' concatenate a stream or list of streams and/or lists (see comments below) Operations like `#', `all', `foldl' will of course cause troubles with infinite streams since it can take them an infinite time to compute the result. The same holds for the `foldr' function unless the second argument of the folded operation is special. The `stream.q' module also extends the list concatenation function `cat' defined in `stdlib.q' to work with any mixture of streams and lists. This operation will always return a list, thus it is not suitable to work with infinite streams either. As a remedy, the `streamcat' function is provided which concatenates a stream of streams in a fully lazy manner, i.e., you can concatenate a (possibly infinite) stream of (possibly infinite) streams. The argument of `streamcat' can actually be a list or stream, which consists of any mixture of streams and lists. The result is always a stream. The `strict' function expands a stream by forcing all its heads and tails to be evaluated (using `~', see *Note Special Forms::). This has essentially the same effect as `list', but the result is still a stream rather than a list. Conversely, the `lazy' function makes a stream even "lazier" than it normally is, by causing the heads and tails of the result stream to be memoized (using `&', see *Note Special Forms::). This reduces computation times when a stream is to be traversed repeatedly. The `stream.q' module also provides some additional operations to force or memoize only the heads (`hdstrict', `hdlazy') or the tails (`tlstrict', `tllazy') of a stream, as well as the "mixed mode" operations `strict_lazy' and `lazy_strict'. See the `streams.q' script for a description of those. An example for the use of `lazy' can be found in *Note Memoization and Lazy Evaluation::.  File: qdoc.info, Node: Conditionals and Comprehensions, Next: Mathematical Functions, Prev: Streams, Up: The Standard Library 11.9 Conditionals and Comprehensions ==================================== The `cond.q' script provides the special forms needed to implement both various conditionals and tuple/list/stream comprehensions. `ifelse ~P X Y, when ~P X, unless ~P X' simple conditional expressions `cond CASES, case ~X CASES' multiway and pattern matching conditional expressions `condfun CASES, casefun CASES' multiway and pattern matching conditional abstractions `dowhile P X' simple looping construct `for CLAUSES X' iterate over lists and streams `tupleof X CLAUSES, listof X CLAUSES, streamof X CLAUSES' tuple, list and stream comprehensions All these functions are implemented as special forms. The `ifelse' function has already been mentioned in *Note Conditional Expressions and Lambdas::. It returns the value of either `X' or `Y', depending on whether the first argument is `true' or `false'. For instance, here is how the factorial function can be implemented using `ifelse': fac N = ifelse (N>0) (N*fac (N-1)) 1; Or, if you prefer to write this using the `if X then Y else Z' syntax described in *Note Conditional Expressions and Lambdas::: fac N = if N>0 then N*fac (N-1) else 1; For cases in which you are only interested in executing one of the branches, you can also use the `when' and `unless' functions which return a `()' default value if the branch condition is _not_ met: ==> def X = 1 ==> when (X>0) (writes "positive\n") positive () ==> unless (X>=0) (writes "negative\n") () As indicated, these constructs are most useful when used with functions involving side-effects; `unless' works exactly like `when', but reverses the branch condition `P'. The `if X then Y' construct is syntactic sugar for `when': ==> if X>0 then writes "positive\n" positive () A more general conditional expression is also provided, which allows you to discriminate between an arbitrary number of cases. It has the form: cond ((P1,X1),(P2,X2),...) Using the "grouping syntax" described in *Note Lists Streams and Tuples::, this can also be written more succintly as: cond (P1,X1;P2,X2;...) This special form looks and behaves like Lisp's `cond'. That is, it returns the first value `X' for which the corresponding condition `P' evaluates to `true'. A branch condition of `true' can be used to indicate the default case. If no case matches then `cond' raises a `syserr 8' exception (note that the same error code is also thrown in case of an invalid conditional in a rule, cf. *Note Exception Handling::). There is also a more Haskell-like `case' conditional in which the first expression of each case is a pattern `P' to be matched against the given expression `X': case X (P1,Y1;P2,Y2;...) The expression `X', which is passed by value, is matched against each of the patterns `P' in turn, and as soon as a matching pattern is found, the corresponding value `Y' is returned, with the variables in the pattern bound to the corresponding values using `lambda'. A pattern of `_' can be used to denote the default case. If no case matches then `case' raises a `syserr 8' exception. Some examples: fac N = cond (N>0, N*fac (N-1); true, 1); sign X = cond (X>0, 1; X<0, -1; true, 0); prod Xs = case Xs ([], 1; [Y|Ys], Y*prod Ys); reply X = case X ("y" , true; "n" , false; _ , throw "bad reply"); As of Q 7.7, there are also variations of `cond' and `case', named `condfun' and `casefun', which take the data to be analyzed as the second argument. These constructs can be used to create anonymous conditional and pattern-matching functions, respectively: condfun (P1,F1;P2,F2;...) X casefun (P1,Y1;P2,Y2;...) X Note that `casefun' works exactly like `case', but the arguments are reversed, so the above `reply' function could also be defined with an implicit parameter, as follows: reply = casefun ("y" , true; "n" , false; _ , throw "bad reply"); In contrast, `condfun' interprets each branch `(P,F)' as a pair of a predicate `P' and a function `F' which are to be applied to `condfun''s second "data" parameter `X'. Going through the tuple of branches, `condfun' returns the first value `F X' for which `P X' evaluates to `true'. Thus `condfun' works like `cond' except that the constituents of each branch are not constant values, but functions to be applied to the data parameter. For instance, here's an alternative way to define the `sign' function from above: sign = condfun ((>0), cst 1; (<0), cst (-1); cst true, cst 0); The `dowhile' function implements a simple looping construct which keeps re-evaluating the expression in the second argument as long as the condition in the first argument evaluates to `true'. Both arguments are special. The result is always `()'. This construct obviously makes sense only with expressions involving side-effects. For instance: ==> def F = fopen "/etc/passwd" "r" ==> dowhile (not feof F) (writes (freads F++"\n")) The `for' function provides a means to iterate an operation with side-effects over tuples, lists and streams. It takes a tuple of clauses in the first argument, which has the same format as the second argument of a comprehension (see below). The second argument is then evaluated for each binding, performing parameter binding using `lambda'. The result is always `()'. Example: ==> for (I in [1..3],J in [1..I]) (write (I,J) || writes " ") (1,1) (2,1) (2,2) (3,1) (3,2) (3,3) () The `listof' function allows to specify a list of values in a manner similar to the way sets are described in mathematics: listof X (Y1, Y2, ...) This construct is also commonly called a "list comprehension". For convenience, the Q language provides syntactic sugar for list comprehensions so that they can also be written as follows: [X : Y1, Y2, ...] (Note that in scripts this alternative notation is only permitted on the right-hand side of equations and variable definitions, as the colon `:' is already used to denote type guards in left-hand side expressions. So in the unlikely case that you have to extend the definition of `listof' you will have to use the first syntax from above.) The expressions `Y1, Y2, ...' may either be so-called binding clauses or conditional clauses. A "binding clause" is an expression of the form `Z in Zs', using the relational operator `in', which is declared as follows: public const (in) X Y @ 2; // relational in operator A binding clause specifies that the pattern `Z' should be matched in turn to each member of the list `Zs'; only values matching the pattern will be extracted, and free variables in the pattern are bound to their corresponding values using `lambda'. Binding clauses are considered from left to right, which means that a clause may refer to any variable introduced in an earlier binding clause. Any other expression specifies a "conditional clause" which must evaluate to a truth value; only those elements will be listed for which the conditional clause is satisfied. Note that, as of Q 7.8, binding clauses may actually draw values from any combination of tuples, lists and streams. The result of the list comprehension is always a list no matter which kind of sequences the values come from. This also works analogously with the other type of comprehensions discussed below. As an example, here is a function which computes the prime numbers up to a given integer `N' using Erathosthenes' sieve. Note the list comprehension `[Y:Y in Xs,Y mod X<>0]' in the second equation which is used to extract all numbers which are not divisible by the first list member (which is always a prime by virtue of the construction). primes N = sieve [2..N]; sieve [X|Xs] = [X|sieve [Y:Y in Xs,Y mod X<>0]]; sieve [] = []; The `tupleof' function works completely analogous, except that it generates tuples instead of lists. It also uses the same kind of syntactic sugar, but the expression is surrounded by ordinary parentheses instead of brackets. These constructs are also known as "tuple comprehensions". Similarly, the `streamof' function implements "stream comprehensions". It works like `listof', but in a lazy fashion and returns a stream instead of a list as the result. The Q language provides syntactic sugar for stream comprehensions, so that they look like list comprehensions, using curly braces instead of brackets. Here is the same prime sieve algorithm as above, formulated with streams instead of lists. Instead of a list of all primes up to a given limit it simply generates the infinite stream of _all_ prime numbers: primes = sieve {2..}; sieve {X|Xs} = {X|sieve {Y:Y in Xs,Y mod X<>0}};  File: qdoc.info, Node: Mathematical Functions, Next: Complex Numbers, Prev: Conditionals and Comprehensions, Up: The Standard Library 11.10 Mathematical Functions ============================ The `math.q' script defines the `const' variables `inf' and `nan', which denote the special IEEE floating point values for (positive) infinity and NaN ("not a number"), and the following additional operations on floating point numbers: `floor X, ceil X' round to the nearest integer below and above the given number `asin X, acos X, tan X' additional trigonometric functions `lg X, log X' base 2 and 10 logarithms `sinh X, cosh X, tanh X' hyperbolic functions `asinh X, acosh X, atanh X' inverse hyperbolic functions  File: qdoc.info, Node: Complex Numbers, Next: Rational Numbers, Prev: Mathematical Functions, Up: The Standard Library 11.11 Complex Numbers ===================== The `complex.q' script implements complex numbers. As of Q 7.7, `Complex' is now implemented as an abstract data type, very much like Rational (cf. *Note Rational Numbers::). A complex value with real part `X' and imaginary part `Y' is represented as `X+:Y', employing the virtual constructor `+:', which is implemented as a binary infix operator with the same precedence as `+'. (The alias `(:+)' for `(+:)' is provided for Haskell compatibility.) Of course, to construct a complex value you can also use the more customary notation `X+i*Y', where the constant `i', which is implemented as a `const' variable, denotes the imaginary unit `0+:1'. The script overloads the usual arithmetic operations (including exponentiation), `sqrt', `exp', logarithms, trigonometric and hyperbolic functions so that they also work with complex numbers. The following basic operations are also provided: `abs Z, arg Z' absolute value and argument `re Z, im Z' real and imaginary part `re_im Z' returns a pair with both the real and the imaginary part `conj Z' complex conjugate `cis X' the complex exponential `cis X = exp (i*X) = cos X + i*sin X' `polar R X' convert polar coordinates to complex number, `polar R X = R*cis X' Note that `cis' and `polar' are only defined on `Real' arguments. The other operations (including the virtual constructor `+:') work consistently on both real and complex numbers. The `arg' function returns the polar angle (in radians), thus `(abs Z,arg Z)' denotes the polar coordinates of a complex number `Z'. Conversely, `polar R X' converts the polar coordinates `(R,X)' back to the corresponding complex number. Some examples: ==> 2^(1/i) 0.769238901363972 +: -0.638961276313635 ==> lg _ 0.0 +: -1.0 ==> (abs _,arg _) (1.0,-1.5707963267949) Please note that the virtual constructor `+:', just like Haskell's `:+' constructor, is really an operator symbol with the same precedence as `+'. Thus an expression like `0+:1' is not "atomic", but has the same precedence and associativity as all addition operators. This yields results which might be somewhat surprising for the uninitiated. For instance, consider: ==> 1+:2*3+:4 1+:10 This result is indeed correct, as the input expression is parsed as `1+:(2*3)+:4 = (1+:6)+:4' and _not_ as `(1+:2)*(3+:4) = -5+:10'. (Note that, unlike Haskell's `:+', Q's `+:' operator can be applied to _both_ real and complex operands, so that `Z1+:Z2' will yield the same as `Z1+i*Z2' in any case.)  File: qdoc.info, Node: Rational Numbers, Next: Graphics, Prev: Complex Numbers, Up: The Standard Library 11.12 Rational Numbers ====================== Thanks to the work of Rob Hubbard, Q versions 7.2 and later also include the `rational.q' script which implements the rational number type `Rational' which is a subtype of `Real'. Please note that at present this module only provides the data type and the basic arithmetic. More advanced approximation and formatting operations for rational numbers can be found in a separate add-on package available from the Q website which also includes additional documentation [Hubbard 2006]. The module overloads the usual arithmetic operations as well as the functions `abs', `sgn', `round', `trunc', `int', `frac' and `pow' (the latter comes from the `clib' module), and provides the following additional operations: `rational X' create a rational number from an integer, or a pair of integers (numerator, denominator) `num Q, den Q' normalized numerator and denominator (numerator and denominator are coprime and the sign is always in the numerator) `num_den Q' returns a pair of integers with the normalized numerator and denominator Last but not least, the module also implements the exact division operator `%' (with the same precedence as `/') which returns a rational or complex rational number for each combination of integer, rational and complex integer/rational arguments; for any other arguments of type `Num' `%' behaves like the `/' operator. The `%' operator is actually implemented as a virtual constructor of the `Rational' type (cf. *Note Views::), and `rational.q' provides a corresponding default view for rational numbers in the format `N%D', where `N' and `D' are the normalized numerator and denominator, respectively. Examples: ==> 5%7 + 2%3 29%21 ==> 3%8 - 1%3 1%24 ==> pow (11%10) 3 1331%1000 ==> pow 3 (-3) 1%27 ==> num_den _ (1,27) ==> rational _ 1%27 Since `Rational' is a subtype of `Real', operations on complex numbers with rational components also work as expected: ==> (1+:2)%(3+:4) 11%25+:2%25 ==> var sqr = \X.X*X; sqr (5+:-5%18) 8075%324 +: -25%9  File: qdoc.info, Node: Graphics, Next: Diagnostics and Error Messages, Prev: Rational Numbers, Up: The Standard Library 11.13 Graphics ============== The `graphics.q' script implements an interface to Adobe's PostScript language [Adobe 1990]. PostScript is a page description language which has become one of the main standards in the desktop publishing world. The nice thing about PostScript is that page descriptions are _device-independent_ - the same page description can be used for different output devices such as a screen previewer or a laser printer. The `graphics.q' script allows you to produce graphics output on any PostScript device. Note that, as already pointed out, this script does _not_ belong to the set of "standard" scripts included by the prelude, and hence has to be imported explicitly if you want to use the operations described in the following. The graphics device is implemented by the `GRAPHICS' variable, which by default is assigned to standard output. This is useful for taking a look at the generated PostScript code, e.g., for debugging purposes, but in a real application you will of course redirect the output to some PostScript file or device, which can be done by assigning a suitable value to the `GRAPHICS' variable. Any file object open for writing will do, however the script also provides the following convenience functions which each return an output file or pipe which can be used as the value of the `GRAPHICS' variable: `gsdev' pipe to the `ghostscript' program (see below) `gvdev' pipe to `ghostview' (see below) `lpdev' printer device (usually a pipe to `lpr'(1)) `filedev NAME' output file with name `NAME' `nulldev' the null device (a synonym for `filedev "/dev/null"', or `filedev "nul"' under DOS/Windows) For instance, you define the graphics device as a pipe to ghostscript as follows: def GRAPHICS = gsdev; You can either add this definition to your main script, or enter it directly at the command prompt of the interpreter (see *Note Command Language::). Ghostscript is a popular PostScript previewer available for a wide range of different platforms. Ghostview is an improved X interface for ghostscript. A similar program, GSView, is also available for the Windows operating system. Note, however, that GSView cannot read its input from a pipe, hence `gvdev' is not directly supported on Windows. To preview your PostScript output under Windows, you can either use `gsdev', or employ `filedev' to set up an output file and then invoke GSView manually. For instance (assuming that the `gsview32' program is on your path): ==> def GRAPHICS = filedev "graphics.ps" ==> // output your PostScript graphics here ... ==> !gsview32 graphics.ps Also note that on most systems output to a pipe created with `popen' is buffered, thus you might have to flush buffered data when using `gsdev' or `gvdev' as the graphics device. In addition, it might be necessary to invoke `flushpage' to force an immediate display update, see *Note Overview of Graphics Operations::. This can be done as follows: ==> flushpage || fflush GRAPHICS If output goes to a printer or a file, you will probably need a minimal header which identifies the file as a PostScript document. To include such a header in the output file, use the `psheader' function _before_ invoking any other graphics operation: ==> psheader You can also set up a custom header and include other DSC and EPSF comments by means of the `ps' function; see *Note DSC and EPSF Comments::, for details. In the following we give an overview of the graphics operations in the PostScript language, as they are provided by this module, and describe the implemented functions. For more details about PostScript please refer to [Adobe 1990]. * Menu: * Coordinate System:: * Overview of Graphics Operations:: * Path Construction:: * Painting:: * Clipping:: * Graphics State:: * Miscellaneous Operations:: * DSC and EPSF Comments::  File: qdoc.info, Node: Coordinate System, Next: Overview of Graphics Operations, Prev: Graphics, Up: Graphics 11.13.1 Coordinate System ------------------------- The PostScript coordinate system has its origin `(0,0)' in the lower left corner of the output page or display window, with the positive `X' and `Y' axes extending horizontally to the right and vertically upward, respectively. The default unit length is 1 "point" which is 1/72 of an inch. The origin of the coordinate system, as well as the unit lengths and orientation of the `X' and `Y' axes can be changed by means of the `translate', `scale' and `rotate' operations, cf. *Note Graphics State::.  File: qdoc.info, Node: Overview of Graphics Operations, Next: Path Construction, Prev: Coordinate System, Up: Graphics 11.13.2 Overview of Graphics Operations --------------------------------------- The process of painting a graphics object usually consists of the following three steps: * Modify graphics parameters such as the text font, the color, or the translation and scaling of the coordinate axes. * Construct a "path" which outlines the shape of the object to be painted. * Execute the appropriate painting operation to display the object on the output page. A "path" is an ordered sequence of straight and curved line segments. The individual segments may be connected to each other or they may be disconnected. Thus a path may consist of several connected pieces which are referred to as the "subpaths" of the path. In a subpath, each line segment starts at the point where the previous segment ends. The `newpath' function is used to begin a new path. A new subpath is obtained by invoking `moveto' which specifies the first point in the subpath. Various operations are provided to add straight and curved line segments to the current subpath. For instance, a path consisting of three straight line segments may be denoted as follows: newpath || moveto 0 0 || lineto 1 0 || lineto 1 1 || lineto 0 1 The last point on the current subpath can be connected back to its starting point (usually the last point specified with `moveto') by "closing" the subpath with the `closepath' operation. For instance, a rectangle is specified as follows: newpath || moveto 0 0 || lineto 1 0 || lineto 1 1 || lineto 0 1 || closepath Having constructed a path, the `stroke' function draws the line segments contained in the path. Alternatively, `fill' may be used to fill the interior of the path (for this purpose the entire path should consist of closed subpaths). The precise appearance of stroked and filled objects on the output page is controlled by a collection of parameters referred to as the "graphics state". Various operations are provided for changing these parameters. For instance, you can set the linewidth and dash pattern used by `stroke', the color used by the `stroke' and `fill' operations, and the scale and translation of the coordinate axes. The current settings can be saved on a stack using `gsave' and restored (popped from the stack) with `grestore'. Another important parameter is the current "clipping path" which specifies the regions on the page which can be affected by the painting operations. By default, the paintable area is the whole page. In order to restrict painting to a user-defined region, a path is constructed as usual, and then the `clip' function is used to set this path as the clipping path. Subsequent paint operations will only paint the interior of the clipping path, i.e., the region which would have been filled had we applied the `fill' operation instead of `clip' to the constructed path. Multiple applications of `clip' are accumulative. That is, `clip' intersects the current clipping area (as defined by previous invocations of `clip') with the interior of the current path. The treatment of textual output is somewhat special. It is possible to define a path consisting of the outlines of the characters in a given text string by means of the `charpath' function. More commonly, however, text strings are simply displayed at a given position which is accomplished by means of the `show' function. For instance, to display a text string `S' at a position `(X,Y)' on the current page the following expression is used: moveto X Y || show S The `graphics.q' script also provides several operations which deal with a graphics page as a whole. First of all, `showpage' emits the current page, and prepares for the next page by erasing the current page. The `copypage' operation is like `showpage', but keeps the contents of the current page. This allows you to accumulate the contents of several pages. Both `showpage' and `copypage' are mainly used when output goes to a printer. Two additional operations are provided for interactive use, when output goes to a display window. The `erasepage' function causes the contents of the current page to be erased. The `flushpage' operation updates the display, like `showpage' or `copypage', but does not start a new page. (To improve performance, graphics output under the X window system is usually performed in larger chunks. The `flushpage' operation is required to synchronize the display by flushing any unwritten data.) Note that this operation is _not_ part of the PostScript standard, but only works with Ghostscript and possibly some similar Postscript viewers. If you want to achieve special effects which cannot be implemented in terms of the operations provided by `graphics.q', you can directly invoke PostScript commands by means of the `ps' function. Also, you can copy a PostScript file to the graphics device with the `psfile' operation. As an example for the `ps' function, operations to display a string right-justified or centered at the current position can be implemented as follows: showright S:String = ps (psstr S++ " dup stringwidth pop neg 0 rmoveto show\n"); showcenter S:String = ps (psstr S++ " dup stringwidth pop 2 div neg 0 rmoveto show\n"); (The `psstr' function converts a string to PostScript syntax; see *Note Miscellaneous Operations::.) The `ps' function is also useful to include DSC and EPSF comments in your graphics output if this is necessary. See *Note DSC and EPSF Comments::, for details.  File: qdoc.info, Node: Path Construction, Next: Painting, Prev: Overview of Graphics Operations, Up: Graphics 11.13.3 Path Construction ------------------------- The `graphics.q' script defines the following collection of operations to define the current path used by the painting and clipping operations: `newpath' start a new path `closepath' close the current subpath `clippath' set the current path to the current clipping path `moveto X Y' absolute move to position `(X,Y)' `rmoveto DX DY' move relatively by `DX' units in horizontal and `DY' units in vertical direction `lineto X Y' straight line segment between the current point and absolute location `(X,Y)' `rlineto DX DY' straight line segment specified by displacement `(DX,DY)' with respect to the current point `curveto X1 Y1 X2 Y2 X3 Y3' Bezier cubic section between the current point and `(X3,Y3)', using `(X1,Y1)' and `(X2,Y2)' as control points `rcurveto DX1 DY1 DX2 DY2 DX3 DY3' Bezier cubic section, with the points specified as displacements with respect to the current point `arc X Y R A1 A2' arc of a circle with radius `R' centered at location `(X,Y)' starting and ending at angles `A1' and `A2' (`0<=A1,A2<=360'), respectively; if there is a current point, it is connected by a straight line segment to the first point on the arc `narc X Y R A1 A2' negative arc; the arc is drawn in clockwise rather than in counter-clockwise direction `arct X1 Y1 X2 Y2 R' arc specified by tangent lines; the center of the arc is located within the inner angle of the tangents, with the first tangent connecting the current point and `(X1,Y1)', and the second tangent connecting `(X1,Y1)' and `(X2,Y2)' `charpath S T' path consisting of the character outlines that would result if the string S where shown at the current point using `show'; `T' is a truth value denoting whether the path should be used for stroking to draw the character outlines (`T = false') or whether the path should be adjusted for use with `fill' or `clip' (`T = true')  File: qdoc.info, Node: Painting, Next: Clipping, Prev: Path Construction, Up: Graphics 11.13.4 Painting ---------------- The following operations are provided for stroking and filling, and for displaying text: `stroke' draw the line segments in the current path `fill, eofill' fill the interior of the current path (any unclosed subpaths of the current path are closed automatically) `show S' paint string `S' at the current point The `fill' function uses the "nonzero winding number" rule for determining which points lie "inside" the current path, while `eofill' uses the "even-odd" rule. Please refer to [Adobe 1990] for the details.  File: qdoc.info, Node: Clipping, Next: Graphics State, Prev: Painting, Up: Graphics 11.13.5 Clipping ---------------- The following operations are used to determine the current clipping path, as described in *Note Overview of Graphics Operations::: `clip, eoclip' intersect the current clipping area with the interior of the current path The `clip' and `eoclip' operations use the same rules for insideness testing as `fill' and `eofill', respectively, see *Note Painting::.  File: qdoc.info, Node: Graphics State, Next: Miscellaneous Operations, Prev: Clipping, Up: Graphics 11.13.6 Graphics State ---------------------- As already indicated in *Note Overview of Graphics Operations::, the PostScript graphics state is a collection of parameters which control the behavior of the graphics operations. The `graphics.q' script provides the following operations to manipulate these parameters: `gsave' save the current graphics state `grestore' restore the previously saved graphics state `savematrix' push the current transformation matrix (CTM) on the PostScript stack; the CTM is manipulated by the `translate', `scale' and `rotate' operations, see below `restorematrix' restore the CTM from the stack `translate TX TY' move the origin of the coordinate system `TX' units in horizontal and `TY' units in vertical direction `scale SX SY' scale the unit lengths of the coordinate axes by `SX' in horizontal and `SY' in vertical direction `rotate A' rotate the coordinate system by an angle of `0<=A<=360' degrees `setlinewidth X' set the line width to `X' units `setlinecap N' set the line cap style (`0' = butt caps, `1' = round caps, `2' = projecting square caps) `setlinejoin N' set the line join style (`0' = miter joins, `1' = round joins, `2' = bevel joins) `setdash Xs DX' set the dash pattern (see below) `setgray X' set the gray shade (`0' = black, `1' = white) `setrgbcolor R G B' set the color in the RGB model (`R' = red, `G' = green, `B' = blue) `sethsbcolor H S B' set the color in the HSB model (`H' = hue, `S' = saturation, `B' = brightness) `setcmykcolor C M Y K' set the color in the CMYK model (`C' = cyan, `M' = magenta, `Y' = yellow, `K' = black) `setcolor C' set the color specified by symbolic color value `C' (see below) `setfont S X' select font `S' scaled by `X' units (see below) The `setrgbcolor', `sethsbcolor' and `setcmykcolor' functions enable you to select arbitrary colors in the RGB, HSB and CMYK model, respectively. A more user-friendly, but less flexible, routine `setcolor' is provided which allows colors to be selected from a fixed set of symbolic constants implemented by the `Color' type. Please refer to `graphics.q' for a list of the possible color values. The `setdash' function sets the dash pattern for straight and curved line segments. The first argument of `setdash' is a list of length values which alternately specify the lengths of the "on" and "off" segments of the line (i.e., dashes and gaps between the dashes). This list is cycled through by the `stroke' function. For instance, `setdash [2,1] 0' specifies the dash pattern "2 on, 1 off, 2 on, 1 off, ...". If the list is empty (`setdash [] 0') `stroke' produces solid lines. The second argument of `setdash' denotes the "phase" of the dash pattern, which is given by an offset into the pattern. E.g., `setdash [2,3] 11' denotes the pattern "1 on, 3 off, 2 on, 3 off, 2 on, ...". The `setfont' function takes as its first argument a string denoting a PostScript font name such as `"Times-Roman"' or `"Helvetica-Oblique"'. The second argument denotes the size in units to which the font should be scaled. For instance: `setfont "Helvetica" 10'. (This function is implemented by a combination of the PostScript operators `findfont', `scalefont' and `setfont'.)  File: qdoc.info, Node: Miscellaneous Operations, Next: DSC and EPSF Comments, Prev: Graphics State, Up: Graphics 11.13.7 Miscellaneous Operations -------------------------------- `showpage' emit the current page `copypage' like `showpage', but do not erase the contents of the current page `flushpage' update the display (flush any buffered graphics) `erasepage' erase the contents of the current page `copies N' number of copies to be emitted with `showpage' `psfile NAME' copy a PostScript file to the graphics device `psstr S' convert a string to PostScript syntax `psheader' output a minimal PostScript header `ps CMD' output a PostScript command The `showpage', `copypage', `flushpage' and `erasepage' functions have already been discussed in *Note Overview of Graphics Operations::. The `copies' operation determines the number of copies which should be printed when `showpage' is invoked: ==> copies 4 || showpage To submit a PostScript file to the graphics device the `psfile' operation may be used. It takes one string argument, the name of the file. For instance: ==> psfile "foo.ps" The `ps' function is used to directly invoke a PostScript command. Examples can be found in *Note Overview of Graphics Operations::. The `psstr' function converts a string to PostScript format. It takes care of embedded backslashes and parentheses. For instance: ==> writes (psstr "(silly\\) example") || writec "\n" (\(silly\\\) example) () The `psheader' function is used to begin the output file with a minimal header identifying the file as PostScript; see *Note DSC and EPSF Comments::.  File: qdoc.info, Node: DSC and EPSF Comments, Prev: Miscellaneous Operations, Up: Graphics 11.13.8 DSC and EPSF Comments ----------------------------- Appendix G and H of [Adobe 1990] define a standard set of comments which should be included in PostScript documents to make explicit the document structure, and to allow inclusion of PostScript files in other documents. These standards are known as the "document structuring conventions" (DSC) and the "encapsulated PostScript file" (EPSF) format. See [Adobe 1990] for a detailed discussion of the purpose of these formats. The `graphics.q' script currently does not provide any specialized operations for sending DSC and EPSF comments to the graphics device, with the exception of the `psheader' function which outputs a minimum header to make your printer recognize a graphics file as PostScript. Invoke this function as follows, _before_ calling any other graphics operation: ==> psheader You can also call the `psheader' function at initialization time from within a script, using a line like the following: def INIT = psheader; To include other types of DSC and EPSF comments in the graphics output, you have to specify these comments explicitly using the `ps' function. For instance, a function writing out the necessary DSC header comments of an EPS file may be implemented as follows: /* (X1,Y1) and (X2,Y2) denote the lower left and the upper right corner of the bounding box, respectively */ epsf_header X1:Real Y1:Real X2:Real Y2:Real = ps "%!PS-Adobe-3.0 EPSF-3.0\n" || ps ("%%BoundingBox: "++join " " [str X1, str Y1, str X2, str Y2]++"\n"); As indicated, each comment _must_ be terminated by a newline character. Use the `epsf_header' function in place of the `psheader' function if you are composing an EPS file which is to be used in other documents. E.g., when invoked as `epsf_header 5 5 105 105', the following header information will be written to the graphics device: %!PS-Adobe-3.0 EPSF-3.0 %%BoundingBox: 5 5 105 105  File: qdoc.info, Node: Diagnostics and Error Messages, Prev: Graphics, Up: The Standard Library 11.14 Diagnostics and Error Messages ==================================== The `assert.q' script supplies the special form `assert' for printing diagnostic messages: foo X = assert (X>0) || bar (1/X); The `assert' function verifies that the given expression evaluates to `true', in which case it returns `()'. Otherwise it uses the `error' function to abort evaluation after printing an error message of the following form: ! Error: assertion (X) failed, value (VALUE OF X) Error messages are printed using the `error' operation of the `error.q' script which prints an error message and then simply stops evaluation using `halt'. For instance: hd [] = error "hd: empty list"; This will print an error message ! Error: hd: empty list when executed, and halt evaluation.  File: qdoc.info, Node: Clib, Next: Q Language Grammar, Prev: The Standard Library, Up: Top 12 Clib ******* `Clib' is the C interface and "system" module of the Q programming language. As of Q 7.8, this component actually consists of two different modules, `clib.q' which provides basic C data structures and routines commonly used in most Q programs, and `system.q' which contains most of the POSIX system interface. Only `clib.q' is part of the prelude; for most system functions, you will have to explicitly import `system.q' in your programs. In the sections below, we will always indicate whether the `system.q' module is needed for the described operations. In difference to the other standard library modules, `clib' and `system' are "external" modules, i.e., most functions are actually implemented in C (cf. *Note C Language Interface::). Together, `clib' and `system' provide additional string operations, extended file functions, C-style formatted I/O, low-level and binary I/O, an interface to various system functions, POSIX thread functions, expression references, time functions, internationalization support, filename globbing and regular expression matching, additional integer functions from the GMP library, and, last but not least, efficient C replacements for some common standard library list and string processing functions. Even if you do not use the extra functionality provided by these modules, you will benefit from the replacement operations (which are in `clib' and thus included in the prelude), which considerably speed up basic list and string processing, sometimes by several orders of magnitude. NOTE: Not all of the following operations are implemented on all systems. The UNIX-specific operations are marked with the symbol `(U)' in the `clib.q' and `system.q' scripts. Only a portable subset of the UNIX system interface is provided, which encompasses the most essential operations found on many recent UNIX (and other POSIX) systems, as described by the ANSI C and POSIX standards as well as the Single UNIX Specification (SUS). These operations are also available on Linux and OSX systems. * Menu: * Manifest Constants:: * Additional String Functions:: * Byte Strings:: * Extended File Functions:: * C-Style Formatted I/O:: * File and Directory Functions:: * Process Control:: * Low-Level I/O:: * Terminal Operations:: * Readline Interface:: * System Information:: * Sockets:: * POSIX Threads:: * Expression References:: * Time Functions:: * Internationalization:: * Filename Globbing:: * Regular Expression Matching:: * Additional Integer Functions:: * C Replacements for Common Standard Library Functions::  File: qdoc.info, Node: Manifest Constants, Next: Additional String Functions, Prev: Clib, Up: Clib 12.1 Manifest Constants ======================= `Clib' defines an abundance of symbolic values for use with various system functions. Most of these can be found in `system.q', but a few constants related to the memory sizes of various basic C data types and the `fseek' and `setvbuf' operations can also be found in `clib.q'. System constants actually vary from system to system; only the most common values are provided as global variables here. The variables are declared `const' (read-only) and are initialized at startup time. Flag values can be combined using bitwise logical operations as usual. A complete list of the variables can be found at the beginning of the `clib.q' and `system.q' scripts. Flag values which are unavailable on the host system will be set to zero, other undefined values to -1. Thus undefined values will generally have no effect or cause the corresponding operations to fail.  File: qdoc.info, Node: Additional String Functions, Next: Byte Strings, Prev: Manifest Constants, Up: Clib 12.2 Additional String Functions ================================ These functions provide an interface to some familiar character routines from the C library. They can all be found in `clib.q' and are thus in the standard prelude. *Character predicates:* These work exactly like the corresponding C library routines, except that they work with arbitrary Unicode, not just ASCII, characters, provided that the interpreter has been built with Unicode support. public extern islower C, isupper C, isalpha C, isdigit C, isxdigit C, isalnum C, ispunct C, isspace C, isgraph C, isprint C, iscntrl C, isascii C; *String conversion:* Convert a string to lower- or uppercase (like the corresponding C functions, but work on arbitrary Unicode strings, not just on single ASCII characters). public extern tolower S, toupper S; Examples -------- Count the number of alphanumeric characters in a text: ==> #filter isalnum (chars "The little brown fox.\n") 17 Convert a string to uppercase: ==> toupper "The little brown fox.\n" "THE LITTLE BROWN FOX.\n"  File: qdoc.info, Node: Byte Strings, Next: Extended File Functions, Prev: Additional String Functions, Up: Clib 12.3 Byte Strings ================= The following type represents unstructured binary data implemented as C byte vectors. This data structure is used by the low-level I/O functions and other system functions which operate on binary data. The `ByteStr' type itself and its operations are implemented in `clib.q' and thus included in the prelude. public extern type ByteStr; public isbytestr B; // check for byte strings Byte strings are like ordinary character strings, but they do not have a printable representation, and they may include zero bytes. (Recall that a zero byte in a character string terminates the string.) They can be used to encode arbitrary binary data such as C vectors and structures. The `bytestr' function can be used to construct byte strings from integers, floating point numbers, string values or lists of unsigned byte values: public extern bytestr X; // create a byte string The `X' argument denotes the data to be encoded and can be either a list of byte values (unsigned integers in the range from 0 to 255), or an atomic data object, i.e., an integer, floating point number or string constant. In the latter case, the argument can also have the form `(X,SIZE)' indicating the desired byte size of the object; otherwise a reasonable default size is chosen. If the specified size differs from the actual size of `X', the result is zero-padded or truncated accordingly. Integer values are encoded in the host byte order, with the least significant GMP limb first; negative integers are represented in 2's complement. Floating point values are encoded using double precision by default or if the byte count is sufficient (i.e., at least 8 on most systems), and using single precision otherwise. Strings are by default encoded in the system encoding, but you can also specify the desired target encoding as `(X,CODESET)' (or `(X,CODESET,SIZE)' if you also need to specify a byte size), where `CODESET' is a string denoting the target encoding. Like ordinary character strings, byte strings can be concatenated, size-measured, indexed, sliced and compared lexicographically. Moreover, a byte string can be converted back to a (multiprecision) integer, floating point number, string value, or a list of byte values. (When converting back to a string you can specify the source encoding as in bstr `(B,CODESET)', otherwise the system encoding is assumed.) For these purposes the following operations are provided. public extern bcat Bs; // concatenate list of byte strings public extern bsize B; // byte size of B public extern byte I B; // Ith byte of B public extern bsub B I J; // slice of B (bytes I..J) public extern bcmp M1 M2; // compare M1 and M2 public extern bint B; // convert to unsigned integer public extern bfloat B; // convert to floating point number public extern bstr B; // convert to string public bytes B; // convert to list public ::list B; // dito You can use the `bytes' function to convert a byte string to a list of byte values; the `list' function is overloaded to provide the same functionality. These functions are defined as follows: bytes B:ByteStr = map (B!) [0..#B-1]; list B:ByteStr = bytes B; For convenience, the common string operators and the `sub' function are overloaded to work on byte strings as well. Thus `#B' returns the size of `B' (the number of bytes it contains) and `B!I' the `I'th byte of `B'. `B1++B2' concatenates `B1' and `B2', `sub B I J' returns the slice from byte `I' to `J', and the relational operators `=', `<', `>' etc. can be used to compare byte strings lexicographically. These operations are all implemented in terms of the functions listed above. Byte Strings as Mutable C Vectors --------------------------------- As of Q 7.11, `clib' supports a number of additional operations which allow you to treat byte strings as mutable C vectors of signed/unsigned 8/16/32 bit integers or single/double precision floating point numbers. The following functions provide read/write access to elements and slices of such C vectors: public extern get_int8 B I, get_int16 B I, get_int32 B I; public extern get_uint8 B I, get_uint16 B I, get_uint32 B I; public extern get_float B I, get_double B I; public extern put_int8 B I X, put_int16 B I X, put_int32 B I X; public extern put_uint8 B I X, put_uint16 B I X, put_uint32 B I X; public extern put_float B I X, put_double B I X; Note that the given index argument `I' is interpreted relative to the corresponding element type. Thus, e.g., `get_int32 B I' returns the `I'th 32 bit integer rather than the integer at byte offset `I'. Also note that integer arguments must fit into machine integers, otherwise these operations will fail. Integers passed for floating point arguments will be coerced to floating point values automatically. For the `get_xxx' functions, the index parameter may also be a pair `(I,J)' to return a slice of the given byte string instead of a single element (this works like `sub'/`bsub', but interprets indices relative to the element type). The `put_xxx' functions also accept a byte string instead of an element as input, and will then overwrite the corresponding slice of the target byte string `B' with the given source byte string `X'. Similar to `sub'/`bsub', these variations of `get_xxx'/`put_xxx' are "safe" in that they automatically adjust the given indices to fit within the bounds of the target byte string. Moreover, the following convenience functions are provided to convert between byte strings and lists of integer/floating point elements. public extern int8_list B, int16_list B, int32_list B; public extern uint8_list B, uint16_list B, uint32_list B; public extern float_list B, double_list B; public extern int8_vect Xs, int16_vect Xs, int32_vect Xs; public extern uint8_vect Xs, uint16_vect Xs, uint32_vect Xs; public extern float_vect Xs, double_vect Xs; Examples -------- Encode an integer as a byte string, take a look at its individual bytes, and convert the byte string back to an integer: ==> hex ==> def B = bytestr 0x01020304; bytes B; bint B [0x4,0x3,0x2,0x1] 0x1020304 (Note that this result was obtained on a little-endian system, hence the least significant byte `0x04' comes first in the byte list.) Negative integers are correctly encoded in 2's complement: ==> def B = bytestr (-2); bytes B; bint B [0xfe,0xff,0xff,0xff] 0xfffffffe To work with these binary representations you must be aware of the way GMP represents multiprecision integers. In particular, note that the default size of an integer is always a multiple (at least one) of GMP's limb size which is usually 4 or 8 bytes depending on the host system's default `long' integer type. The actual limb size can be determined as follows: ==> #bytes (bytestr 0) In order to get integers of arbitrary sizes, an explicit `SIZE' argument may be used. For instance, here is how we encode small (1 or 2 byte) integers: ==> bytes (bytestr (0x01,1)); bytes (bytestr (0x0102,2)) [0x1] [0x2,0x1] The host system's byte sizes of various atomic C types can be determined with symbolic values declared at the beginning of `clib.q', such as `SIZEOF_CHAR', `SIZEOF_SHORT', `SIZEOF_LONG', `SIZEOF_FLOAT' and `SIZEOF_DOUBLE'. Another fact worth mentioning is that even on big-endian systems, integers are always encoded with the "least significant limb" first. So, for instance, given that the limb size is 4, as in the above examples, the 2-limb integer `0x0102030405060708' consists of bytes `0x8 0x7 0x6 0x5 0x4 0x3 0x2 0x1' on a little-endian system, in that order, whereas the byte order on a big-endian system is `0x5 0x6 0x7 0x8 0x1 0x2 0x3 0x4'. Here is how we can quickly check the byte order of the host system: ==> hd (bytes (bytestr 1)) This expression returns 1 on a little-endian system and zero otherwise. As long as an integer does not exceed the machine's word size (which usually matches the limb size), we can simply convert between big-endian and little-endian representation by reversing the byte list: ==> bytestr (reverse (bytes B)) Floating point values can be encoded either in double or single precision, depending on the `SIZE' argument. The default size is double precision (usually 8 bytes). ==> bfloat (bytestr (1/3)); bfloat (bytestr (1/3,SIZEOF_FLOAT)) 0.333333333333333 0.333333343267441 The default size of the encoding of a character string is the byte size of the string in the target encoding (the system encoding by default). If an explicit size is given, the string is zero-padded or truncated if necessary. The following example will work with any system encoding based on 7 bit ASCII (like Latin1, UTF-8 or ASCII itself): ==> dec ==> def S1 = bytestr "ABC", S2 = bytestr ("ABC",2), S3 = bytestr ("ABC",5) ==> bytes S1; bytes S2; bytes S3 [65,66,67] [65,66] [65,66,67,0,0] ==> bstr S1; bstr S2; bstr S3 "ABC" "AB" "ABC" By combining elements like the ones above, and including appropriate "tagging" information, more complex data structures can be represented as binary data as well. For this purpose, the byte strings of the tags and the data elements can be concatenated with `bcat' or the `++' operator. This is useful, in particular, for compact storage of objects in files. Moreover, some system functions involve binary data which might represent C structures and/or vectors. Such data can be assembled from the constituent parts by simply concatenating them. For instance, consider the following C struct: struct { char foo[108]; short bar; int baz; }; A value of this type, say `{"Hello, world.", 4711, 123456}', can then be encoded as follows: ==> bytestr ("Hello, world.",108) ++ bytestr (4711,SIZEOF_SHORT) ++ \ bytestr (123456,SIZEOF_INT) Similarly, a list of integers can be converted to a corresponding C vector as follows: ==> bcat (map bytestr [1..100]) When encoding such C structures you must also consider alignment issues. For instance, most C compilers will align non-byte data at even addresses. In order to facilitate the handling of C vectors of integers and floating point values, as of Q 7.11 `clib' offers a number of specialized operations which provide direct read/write access to elements and slices of numeric vectors, and allow you to convert between C vectors and Q lists of integer or floating point values. These operations are all implemented directly in C and will usually be much more efficient for manipulating numeric C vectors than the basic byte-oriented functions. Moreover, they allow you to modify the elements of a C vector in a direct fashion, turning byte strings into a mutable data structure. Different operations are provided to handle vectors of signed or unsigned 8/16/32 bit (machine) integers, as well as single (32 bit) and double precision (64 bit) floating point numbers. For instance: ==> def B = uint32_vect [100..110] ==> uint32_list B [100,101,102,103,104,105,106,107,108,109,110] ==> get_uint32 B 1 101 ==> put_uint32 B 1 0xffffffff () ==> uint32_list B [100,4294967295,102,103,104,105,106,107,108,109,110] Note that, because these C vectors are just normal byte strings, you can freely convert between different representations of the numeric data. E.g.: ==> take 12 $ int8_list B [100,0,0,0,-1,-1,-1,-1,102,0,0,0] Entire slices of byte strings can be retrieved and overwritten as well. Note that, as with `sub', the indices are adjusted automatically to stay within the bounds of the target vector. ==> put_uint32 B (-2) (uint32_vect [90..94]) () ==> uint32_list B [92,93,94,103,104,105,106,107,108,109,110] ==> uint32_list $ get_uint32 B (-2,3) [92,93,94,103] ==> uint32_list $ get_uint32 B (8,100) [108,109,110]  File: qdoc.info, Node: Extended File Functions, Next: C-Style Formatted I/O, Prev: Byte Strings, Up: Clib 12.4 Extended File Functions ============================ `Clib' provides the following enhanced and additional file functions: public extern ::fopen NAME MODE, fdopen FD MODE, freopen NAME MODE F; public extern fileno F; public extern setvbuf F MODE; public extern fconv F CODESET; public extern tmpnam, tmpfile; public extern ftell F, fseek F POS WHENCE; public rewind F; public extern gets, fgets F; public extern fget F; public extern ungetc C, fungetc F C; These are all defined in `clib.q' and thus included in the prelude. The `fopen' version of `clib' handles the `+' flag in mode strings, thus enabling you to open files for both reading and writing. The mode `"r+"' opens an existing file for both reading and writing; the initial file contents are unchanged, and both the input and output file pointers are positioned at the beginning of the file. The `"w+"' mode creates a new file, or truncates it to zero size if it already exists, and positions the file pointers at the beginning of the file. The `"a+"' mode appends to an existing file (or creates a new one); the initial file pointer is set at the beginning of the file for reading, and at the end of the file for writing. All these modes also work in combination with the `b' (binary file) flag. The `freopen' function is like `fopen', but reopens an existing file object on another file. Just as in C programming, the main purpose of this operation is to enable the user to redirect the standard I/O streams associated with the interpreter process (available in the interpreter by means of the `INPUT', `OUTPUT' and `ERROR' variables). The `fdopen' function opens a new file object on a given file descriptor, given that the mode is compatible. Conversely, the `fileno' function returns the file descriptor of a file object. (See also the functions for direct file descriptor manipulation in *Note Low-Level I/O::.) The `setvbuf' function sets the buffering mode for a file (`IONBF' = no buffering, `IOLBF' = line buffering, `IOFBF' = full buffering). This operation should be invoked right after the file has been opened, _before_ any I/O operations are performed. The `fconv' function sets the encoding of a file. By default, Q's built-in I/O operations as well as `clib''s string I/O functions assume the system encoding, and convert between this encoding and the internal UTF-8 string representation as needed. If a text file uses an encoding different from the system encoding, you can use the `fconv' function to set the desired encoding. `CODESET' must be a string denoting a valid encoding name for the `iconv' function (see also *Note Internationalization::, below). This affects all subsequent text read/write operations on the file. (This operation only works for Unicode-capable systems which have `iconv' installed. Also note that this function is only available for Q 7.0 and later.) The `tmpnam' and `tmpfile' functions work just like the corresponding C routines: `tmpnam' returns a unique name for a temporary file, and `tmpfile' constructs a temporary file opened in `"w+b"' mode, which will be deleted automatically when it is closed. See the `tmpnam'(3) and `tmpfile'(3) manual pages for details. The `ftell'/`fseek' functions are used for file positioning. The `ftell' function returns the current file position, while `fseek' function positions the file at the given position. The `rewind' function provides a convenient shorthand for repositioning the file at the beginning. These operations work just like the corresponding C functions. The `WHENCE' argument of `fseek' determines how the `POS' argument is to be interpreted; it can be either `SEEK_SET' (`POS' is relative to the beginning of the file, i.e., an absolute position), `SEEK_CUR' (`POS' is relative to the current position) or `SEEK_END' (`POS' is relative to the end of the file). In the latter two cases `POS' can also be negative. Portability Notes: * According to the ISO C specification, you should use `fflush' or `fseek' before switching between reading and writing on a file opened with the `+' flag. * On non-UNIX systems, `ftell' and `fseek' might only work reliably if the file is opened in binary mode (`b' flag). The `gets'/`fgets' functions work like the C `fgets' function, i.e., they read a line from standard input or the given file _including_ the trailing newline, if any. The `fget' function reads an entire file at once and returns it as a string. The `ungetc'/`fungetc' functions push back a single character on standard input or the given input file, like the C `ungetc' function. The C library only guarantees that pushing back a single ASCII character will work, so the result of pushing back multiple or multibyte characters is implementation-dependent. Moreover, the following additional aliases are provided for C aficionados: public ::readc as getc, ::freadc F as fgetc; public ::writes S as puts, ::fwrites F S as fputs; public ::writec C as putc, ::fwritec F C as fputc; Examples -------- Open a new file for both reading and writing: ==> def F = fopen "test" "w+" Write a string to the file: ==> fwrites F "The little brown fox.\n" () Current position is behind written string (at end-of-file): ==> ftell F 22 Rewind (go to the beginning of the file): ==> rewind F () Read back the string we've written before: ==> fgets F "The little brown fox.\n" Check that we're again at end-of-file: ==> feof F true Output another string: ==> fwrites F "The second line.\n" () Position behind the first string: ==> fseek F 22 SEEK_SET () Reread the second string: ==> fgets F "The second line.\n" And here's how to read an entire text file at once: ==> def T = fget (fopen "clib.q" "r") To quickly compute a 32 bit checksum of the file: ==> sum (bytes (bytestr T)) mod 0x100000000 3937166 Finally, let's split the text into lines and add line numbers using `sprintf' (see *Note C-Style Formatted I/O::): ==> def L = split "\n" T ==> def L = map (sprintf "%3d: %s\n") (zip [1..#L] L) ==> do writes (take 5 L) 1: 2: /* clib.q: Q's system module */ 3: 4: /* This file is part of the Q programming system. 5: ()  File: qdoc.info, Node: C-Style Formatted I/O, Next: File and Directory Functions, Prev: Extended File Functions, Up: Clib 12.5 C-Style Formatted I/O ========================== These functions provide an interface to the C `printf' and `scanf' routines. They are all defined in `clib.q' and thus included in the prelude. public extern printf FORMAT ARGS, fprintf F FORMAT ARGS, sprintf FORMAT ARGS; public extern scanf FORMAT, fscanf F FORMAT, sscanf S FORMAT; Arguments to the `printf' routines and the results of the `scanf' routines are generally encoded as tuples or single non-tuple values (if only one item is read/written). All the usual conversions and flags of C `printf'/`scanf' are supported, except `%p' (pointer conversion). The basic `h' and `l' length modifiers are also understood, but not the fancy ISO C99 extensions like `ll' or `hh', or `l' modifiers on characters and strings. Two further unsupported features of the `printf' functions are the `%n' (number of written characters) conversion and explicit argument indexing (`m$'); thus all arguments have to be in the same order as specified in the `printf' format string. The `%n' conversion _is_ implemented for the `scanf' functions, though. As these functions are simply wrappers for the corresponding C functions, integer conversions are generally limited to values which fit into machine integers. To handle integers of arbitrary sizes, you might treat them as strings (`%s') in the format string and do the actual conversion manually with `val' or `str'. Seasoned C programmers will appreciate that the wrapper functions provided here are _safe_ in that they check their arguments and prevent buffer overflows, so they should never crash your program with a segfault. To these ends, if a `%s' or `%[...]' conversion without maximum field width is used with `scanf', the field width will effectively be limited to some (large) value chosen by the implementation. Examples -------- See the `printf'(3) and `scanf'(3) manual pages for a description of the format string syntax. Some basic examples follow (`' indicates that you hit the carriage return key to terminate a line): ==> printf "%d\n" 99 99 () ==> printf "%d\n" (99) 99 () ==> printf "%s %s %d\n" ("foo","bar",99) foo bar 99 () ==> scanf "%d" 99 99 ==> scanf "%s %s %d" foo bar 99 ("foo","bar",99) As indicated, multiple values are denoted as tuples, and the `printf' function accepts both a single value or a one-tuple for a single conversion. The `scanf' function always returns a single, non-tuple value if only a single conversion is specified. Zero items are represented using the empty tuple. Note that you always have to supply the `ARGS' argument of `printf', thus you specify an empty tuple if there are no output conversions: ==> printf "foo\n" () foo () The `scanf' function also returns an empty tuple if no input items are converted. For instance (as usual, using the `*' flag with a `scanf' conversion suppresses the corresponding input item): ==> scanf "%*s" foo () Note that while `scanf' for most conversions skips an arbitrary amount of leading whitespace, the trailing whitespace character at which a conversion stops is _not_ discarded by `scanf'. You can notice this if you invoke, e.g., `readc' afterwards: ==> scanf "%s %d"; readc foo 99 ("foo",99) "\n" If you really have to skip the trailing whitespace character, you can do this with a suppressed character conversion, e.g.: ==> scanf "%s %d%*c"; writes "input: "||reads foo 99 ("foo",99) input: The `fprintf'/`fscanf' functions work analogously, but are used when writing or reading an arbitrary file instead of standard output or input. For instance: ==> var msg = "You're not supposed to do that!" ==> fprintf ERROR "Error: %s\n" msg Error: You're not supposed to do that! () The `sprintf' function returns the formatted text as a string instead of writing it to a file: ==> sprintf "%s %s %d\n" ("foo","bar",99) "foo bar 99\n" Likewise, `sscanf' takes its input from a string: ==> sscanf "foo bar 99\n" "%s %s %d" ("foo","bar",99) The `%n' conversion is especially useful with `sscanf', since it allows you to determine the number of characters which were actually consumed: ==> sscanf "foo bar 99 *** extra text here ***\n" "%s %s %d%n" ("foo","bar",99,10) You might then use the character count, e.g., to check whether the input format matched the entire string, or whether there remains some text to be processed. Some remarks about the role of the length modifiers `h' and `l' are in order. Just as with the C `scanf' routines, you need the `l' modifier to read a double precision value; a simple `%f' will only read single precision number: ==> scanf "%f" 1e100 inf ==> scanf "%lf" 1e100 1e+100 The `printf' functions, however, always print double precision numbers, so the `l' modifier is not needed: ==> sprintf "%g" 1e100 "1e+100" For the integer conversions, the `h' and `l' modifiers denote short (usually 2 byte) and long (usually 4 byte) integer values. If the modifier is omitted, the default integer type is used (this usually is the same as `long', but your mileage may vary). As already indicated, the `printf' and `scanf' routines are limited to machine integer sizes. Thus a `scanf' integer conversion will always return a short or long integer value, depending on the length modifier used. If a `printf' integer conversion is applied to a "big" integer value, only the least significant bytes of the value are printed, as if the printed number (represented in 2's complement if negative) had been cast to the corresponding integer type in C. Thus the printed result will be consistent with C `printf' output under all circumstances. For instance: ==> def N = 0xffff70008000 // big number ==> printf "%hu %lu\n" (N,N) 32768 1879080960 () ==> printf "%hd %ld\n" (N,N) -32768 1879080960 () To correctly print a big integer value, you can convert it manually with Q's built-in `str' function, then print the value using a `%s' conversion: ==> printf "%s\n" (str 1234567812345678) 1234567812345678 () Similarly, you can read a big integer value by converting it as a string, and then apply the val builtin. ==> val (scanf "%s") 1234567812345678 1234567812345678 Here you might use the `%[...]' conversion to ensure that the number is in proper format (the initial blank is needed here to skip any leading whitespace): ==> val (scanf " %[0-9-]") -1234567812345678 -1234567812345678 On output, the integer and floating point conversions can all be used with either integer or floating point arguments; integers will be converted to floating point values and vice versa if necessary: ==> printf "An integer: %d\n" 99.9 An integer: 99 () ==> printf "A floating point value: %e\n" 99 A floating point value: 9.900000e+01 ()  File: qdoc.info, Node: File and Directory Functions, Next: Process Control, Prev: C-Style Formatted I/O, Up: Clib 12.6 File and Directory Functions ================================= These functions provide the same functionality as their C counterparts. They are all to be found in `system.q' and thus you have to explicitly import the `system' module to use them. public extern rename OLD NEW; // rename a file public extern unlink NAME; // delete a file public extern truncate NAME LEN; // truncate a file (U) public extern getcwd, chdir NAME; // get/set the working directory public extern mkdir NAME MODE; // create a new directory public extern rmdir NAME; // remove a directory public extern readdir NAME; // list the files in a directory public extern link OLD NEW; // create a hard link (U) public extern symlink OLD NEW; // create a symbolic link (U) public extern readlink NAME; // read a symbolic link public extern mkfifo NAME MODE; // create a named pipe (U) public extern access NAME MODE; // test access mode public extern chmod NAME MODE; // set the file mode public extern chown NAME MODE UID GID; // set file ownership (U) public extern lchown NAME MODE UID GID; // set link ownership (U) public extern utime NAME TIMES; // set the file times public extern umask N; // set/get file creation mask public extern stat NAME, lstat NAME; // file and link information The `stat'/`lstat' functions return a tuple consisting of the commonly available fields of the C `stat' struct, see `stat'(2). For your convenience, the following mnemonic functions are provided for accessing the different components: public st_dev STAT, st_ino STAT, st_mode STAT, st_nlink STAT, st_uid STAT, st_gid STAT, st_rdev STAT, st_size STAT, st_atime STAT, st_mtime STAT, st_ctime STAT; Examples -------- These all need the `system' module, so you have to import it in the interpreter to make the following examples work. E.g.: ==> import system With that out of the way, let's play around with some of these functions: ==> mkdir "tmp" 0777||chdir "tmp"||mkfifo "foo" 0666||\ rename "foo" "bar"||unlink "bar"||chdir ".."||rmdir "tmp" () (Create a `tmp' subdirectory, change to it, create a new FIFO special file, rename that file, delete it, change back to the original directory, and remove the `tmp' directory. All with a single expression which realizes identity.) Now for something more useful. We can retrieve the current umask while setting it to zero, and then reset it to the original value as follows: ==> def U = umask 0; oct; umask U || U; dec 022 List the files in the current directory: ==> readdir "." [".","..","Makefile","givertcap","clib.c","clib.q","Makefile.am", "Makefile.in","README-Clib","examples","Makefile.mingw"] Get the size of a file: ==> st_size (stat "README-Clib") 355  File: qdoc.info, Node: Process Control, Next: Low-Level I/O, Prev: File and Directory Functions, Up: Clib 12.7 Process Control ==================== With the notable exception of the `exit' function which is included in `clib' (and thus in the prelude), all the following functions need an explicit import of the `system' module. The `system' function returns the status code of the command if execution was successful, and fails otherwise: public extern system CMD; // exec command using the shell `Clib' also provides the usual UNIX process creation and management routines. Most of these really require a UNIX system; no attempt is made to emulate operations like `fork' on systems where they are not implemented. Thus the only process operations which currently work under Windows are `system', `exec', `spawn', `_spawn', `exit' and `getpid'. public extern fork; // fork a child process (U) public extern exec PROG ARGS; // execute program public extern spawn PROG ARGS; // execute program in child process public extern _spawn MODE PROG ARGS; // execute child with options public extern nice INC; // change nice value (U) public extern exit N; // exit process with given exit code public extern pause; // pause until a signal occurs (U) public extern raise SIG; // raise signal in current process public extern kill SIG PID; // send signal to given process (U) public extern getpid; // current process id public extern getppid; // parent's process id (U) public extern wait; // wait for any child process (U) public extern waitpid PID OPTIONS; // wait for given child process (U) All these operations are simply wrappers for the corresponding C library routines. Note, however, that the `kill' function takes the signal to send as its _first_ argument, which makes it easier to use partial applications of the function, e.g., to iterate a kill operation over a list of process numbers (as in `do (kill SIGTERM) PIDs'). The `exec' function performs a path search like the C `execlp'/`execvp' function; the parameters for the program are given as a string list `ARGS', and as usual the first argument should repeat the program file name. This function never returns unless it fails. The `spawn' and `_spawn' operations are provided to accommodate Windows' lack of `fork' and `wait'; these functions work on both UNIX and Windows. The `spawn' function works like `exec', but runs the program in a new child process. It returns the new process id (actually the process handle under Windows). The `_spawn' function is like `spawn', but accepts an additional `MODE' parameter which determines how the child is to be executed, either `P_WAIT' (wait for the child, return its exit status), `P_NOWAIT' (do not wait for the child, same as `spawn'), `P_OVERLAY' (replace the current image with the new process, same as `exec') and `P_DETACH' (run the new process in the background). (Note that the `P_DETACH' option is ignored on UNIX systems; the correct way to code a "daemon" on UNIX is shown in the examples section below.) On UNIX, the following routines are provided to interpret the status code returned by the `system', `_spawn', `wait' and `waitpid' functions: public extern isactive STATUS; // process is active public extern isexited STATUS, exitstatus STATUS; // process has exited normally, get its exit code public extern issignaled STATUS, termsig STATUS; // process was terminated by signal, get the signal number public extern isstopped STATUS, stopsig STATUS; // process was stopped by signal, get the signal number For more information about the process functions, we refer the reader to the corresponding UNIX manual pages. Operations to access the process environment are also implemented. The `getenv' function fails if the given variable is not set in the environment; this lets you distinguish this error condition from a defined variable with empty value. The `setenv' function overwrites an existing definition of the given variable: public extern getenv NAME, setenv NAME VAL; // get/set environment variables On UNIX, the following operations provide access to process user and group information, as well as process groups and sessions. Not all operations may be implemented on all UNIX flavours. Please see the UNIX manual for a description of these functions. /* User/group-related functions (U). */ public extern setuid UID, setgid GID; // set user/group id of process public extern seteuid UID, setegid GID; // set effective user/group id public extern setreuid RUID EUID, setregid RGID EGID; // set real and effective ids public extern getuid, geteuid; // get real/effective user id public extern getgid, getegid; // get real/effective group id public extern getlogin; // get real login name // get/set supplementary group ids of current process public extern getgroups, setgroups GIDS; /* Session-related routines (U). */ public extern getpgid PID, setpgid PID PGID; // get and set process group public extern getpgrp, setpgrp; // dito, for calling process public extern getsid PID; // get session id of process public extern setsid; // create a new session Examples -------- Invoke the `system' function to execute a shell command: ==> import system ==> system "ls -l" You can also run the program directly with the `spawn' function: ==> spawn "ls" ["ls","-l"] Get and set an environment variable: ==> getenv "HOME" "/home/ag" ==> getenv "MYVAR" // variable is undefined getenv "MYVAR" ==> setenv "MYVAR" "foo bar" () ==> getenv "MYVAR" "foo bar" Here are some examples demonstrating the use of named pipes and the process functions on UNIX systems. The `mkfifo' function allows the creation of so-called "FIFO special files" a.k.a. "named pipes", which provide a simple inter-process communication facility. For instance, create a named pipe as follows: ==> mkfifo "pipe" 0666 () You can then open the writeable end of the pi