HomeDownloadRelease Notes

Pike 7.8.700 Release Notes

Changes since Pike 7.8.352 (third 7.8 release):

Extensions and New Functions

  • Added module ZXID

    This module provides an interface to the ZXID Identity Management toolkit, which implements (among other tasks) the SAML 2.0, Liberty ID-WSF 2.0, and XACML 2.0 protocols.

  • Added module Search.

    This module implements a free text search database. It needs some indexed storage, and there is a storage backend for use with MySQL. There is support for indexers with filters for different document types. The query support includes metadata and date range searches.

    It is the core of the search functionality that is used in the commercial Roxen products, so it has been in production use for quite some time already.

  • Typed query result support in Sql.Sql and in the MySQL module.

    Traditionally the SQL glue has returned all types of data as formatted strings with their display representations. Now there are new so-called "typed" variants of the query functions (typed_query, big_typed_query, and streaming_typed_query in the Sql.Sql class) that instead return the data as the appropriate pike native types, e.g. pike integers for SQL integers, pike floats for SQL floats, etc.

    This requires support by the DB server specific backends, which is currently limited to the MySQL glue.

  • has_prefix now works on objects.
  • Standards.JSON

    New module for encoding and decoding JSON data. Implemented in C.

  • Val

    New module to contain some global special objects that are shared between other modules. There is e.g. a Val.null object which is used both by the Sql module to represent SQL NULLs in typed mode, and by Standards.JSON to represent the null value.

  • Protocols.HTTP
    • Protocols.HTTP.Session will no longer override an existing content-type header in async_do_method_url.
    • Allow directly specifying the body of a POST request (needed for the Google Apps API).
    • A more complete list of response codes and descriptions is included and used.
    • Query includes proxy support for GET (http/https) and POST (https) in both sync and async mode.
    • Query exposes close() so you do not have to wait for garbage collection.
    • Server adds Basic Range support.
  • Regexp.PCRE.replace() adds support for capture groups.
  • Updated timezone data to tzdata2010o.
  • String.normalize_space

    New and efficient C implementation; sanitises whitespace, even in wide strings.

  • String.trim_all_whites

    Extended to the entire range of Unicode white spaces. (And is faster.)

  • Tools.PV and pike -x pv now work with GTK2 as well as GTK.
  • Integer constants exported by modules can now be used in cpp expressions.
  • Modules/programs that have the constant "dont_dump_program" will not be dumped on installation. This is used to prevent dumping of programs that check for external dependencies at compile time such as the MySQL module.
  • Added Thread.Queue.peek_array.
  • Fixed stack overrun bug in Nettle.CBC and Nettle.Proxy. This is not believed to have been a security issue.
  • System
    • Added getloadavg() on platforms that support the getloadavg() call.
    • Added cross-platform get_home() and get_user() wrappers.
    • Added GetUserName() for Windows systems.
  • SSL
    • Added support for AES to SSL3, and fixed bug that prevented some handshake failure alerts to be sent to the client.
    • Implemented RFC 5746 "TLS Renegotiation Indication Extension".
    • Support parsing of Hello Extensions (RFC 5246 7.4.1.4).
    • Added SSL3.sslfile.{set,query}_callbacks() as per Stdio.File.
  • Stdio
    • New methods in Stdio.File:
      • set_callbacks() and query_callbacks() for handling several callbacks at once.
      • send_fd() and receive_fd() allow file descriptors to be sent and received over UNIX-domain sockets.
    • The type for Stdio.Fd.create() is now as documented.
    • Stdio.Terminfo contains a new class, MetaTerminfoDB, that merges several terminfo directories.
    • Stdio.File()->query_address() now knows about IPv6-mapped IPv4 addresses. Added some extra magic to Stdio.File()->connect() when mixing IPv4 and IPv6 addresses.
  • ADT.Heap->values and ->num_values can now be overloaded.
  • Added support for binding a Protocols.DNS server to multiple port/interface combinations.
  • Support for asynchronous DNS query replies in Protocols.DNS.
  • Calendar
    • dwim_time() should know how to parse ISO 8601 timestamps.
    • Added Calendar.Greek.
  • Process
    • Process.spawn(), popen() and system() et al now use Process.Process.
    • Process.Process adds support for spawning processes via forkd.
    • Process.spawn_pike() can now optionally pass predefines as well as program and include path arguments.
    • Process.spawn_pike() should also be more reliable on Windows.
  • Added functions to access the numerator and the denominator of a Gmp.mpq.
  • Added lower level functions Tools.X509.[dr]sa_sign_key(). This makes it easier to create hierarchial certificates.
  • makeselfsigned[dr]sa_certificate() now use the corresponding [dr]sa_sign_key().
  • Added the !-modifier for sscanf().
  • Added Int.reflect().
  • Image.Dims adds support for Photoshop files.
  • Added Gmp.encode_json().
  • Added Thread.Queue.peek_array.
  • Add System.getloadavg() on platforms that support the getloadavg() call.
  • Pike.Backend has reinstated call_out memory usage, and a new function, Pike.DefaultBackend.get_stats() for viewing this data.
  • Locale.Charset now supports encoding and decoding UTF32.
  • Added Sql.Sql()->is_open() and Sql.Sql()->ping(). Note that your chosen database module must support these (Mysql and pgsql currently do).
  • MIME.ext_to_type() now knows about most all popular mime types (including ~700 new entries).

Optimizations

  • MIME should now use 1/3 less memory when decoding big messages.
  • Nettle.hash

    Avoid allowing threads during hashing of small strings (current threshold is 1 MB). This change improves ApacheBench numbers ~20% for a Roxen benchmark on a quad-core Core i5.

  • String.trimallwhites is 10% faster and includes all Unicode.
  • Improved performance of Process.run when threads aren't supported.
  • Locale.Charset improves performance of encoders when replacement is active by a few magnitudes.
  • pike -x pmar_install can install from an http/s url, supports gzipped archives, md5 hash checking and better uninstall support on Windows.
  • Monger
    • Monger (pike -x monger) can now install pmar (pre-built module archives) from the module repository, when available (particularly useful on Windows).
    • Monger can also download and build from a module's source repository (svn, hg, git) when specified in the Pike module repository (modules.gotpike.org).

Deprecations

  • The libpq based old Postgres driver is now deprecated. The new pgsql driver is faster, better tested, more stable, maintained, has more features, does not depend on external libraries and is less filling.
  • Stdio.File: The << operator on files is deprecated.

Incompatible changes

  • Signal handlers in GTK2 do no longer get their arguments in the form of an array. The prototype for a signal handler which was previously e.g.

    int query_tooltip(GTK2.Widget w, array(mixed) params, mixed callback_arg)

    now becomes

    int query_tooltip(GTK2.Widget w, int x, int y, int keyboard_tooltip, GTK2.GObject tooltip, mixed callback_arg)

Bug fixes

  • Architecture/OS specific
    • Fixed segfault in combine_path_nt on windows when the first char of an appended path is wide.
    • Stdio.cp should now work on directory trees on Windows.
    • Fixed bug that caused file_stat on Windows NT to not return a result on certain non-FAT filesystems. Fixes bug 6432.
  • Calendar module
    • Fixed backward compatibility for Calendar.iso_name() and Calendar.iso_short_name().
    • Fix evaluation order for Calendar.YMD.`+().
    • Fixed problem with Calendar.month_from_yday() when applied on week
    • [bug 5303]
  • Compiler
    • Fixed issue where a stale pointer into the identifier table could sometimes be dereferenced when defining getters and setters.
    • Fixed NULL-deref for compilation errors in range expressions.
    • Fixed bug that prevented the literal zero from being considered a constant.
    • Fixed fatal error in the constant expression evaluator of the compiler for certain code.
  • Documentation
    • Added Autodoc support for new-style getter and setter syntax.
    • Fixed support for documenting arguments to implicit create().
  • Graphics
    • Fixed leak in the PNG decoder for images with tRNS chunk.
    • Fixed several issues related to lables on the X and Y axises and the calculation of xmaxvalue for the bars/line/linear case in Graphics.Graph.
    • Fix integer underflow in Image.JPEG. Fixes [bug 6413].
  • I/O
    • Support undocumented OS behaviour on out-of-band data returning EOPNOTSUPP and ECONNRESET when a stream is half closed. The undocumented behaviour is found on (at least) 2.6.x Linux and FreeBSD 7.x and 8.x.
    • Stdio.{write,append}_file() now throw errors in ALL failure modes.
    • Stdio.File()->query_address() should return a correct errno when given a socket with an invalid protocol family.
    • Stdio.File()->file_peek() now refrains from releasing the interpreter lock if there's no timeout.
  • Protocol modules
    • Fixed backtraces when Protocols.HTTP.Query objects where garbage collected. [bug 5730]
    • Multiple multipart file submissions in Protocols.HTTP.Server will be preserved as suggested by HTML5.
    • Protocols.HTTP.Query fixes broken state when open_socket() has thrown an error.
    • Tools.Standalone.httpserver has fixed entity encoding. Makes files with spaces in them accessible.
    • Protocols.HTTP Sets the content-length header for zero-length data. Fixes [bug 5936].
    • Protocols.HTTP.Query uses content-length instead of content_length. The latter will be corrected in the encode method, but is problematic when that function is overridden.
    • Protocols.HTTP.Query filters weak SSL-ciphers rather than explicitly listing strong ones.
    • Protocols.HTTP includes some close, state reset and keep-alive handling fixes.
    • Fixed so than an existing content-type is not overriden in asyncdomethod_url.
  • Runtime
    • Use nanosleep() when available instead of poll or select hacks for the Pike-level sleep() function.
    • Fixed a segfault occuring if exit() is used from a thread when the Pike runtime has been compiled with cleanup-on-exit enabled.
    • Fixed bug in new_check_call(). Fixes [bug 6442].
  • sprintf
    • Fix a buffer overrun when formatting Gmp.mpz as floats if the precision exceeds the number of digits.
    • Ensure that decimal points are used when floats are formatted in casts etc. Previously the float formatting in casts and in sprintf("%O",...) were susceptible to locale settings and could therefore use something other than a '.' as decimal separator. Note that sprintf("%f",...) is still locale sensitive.
  • Sql database modules
    • Fixed memory leak on out of memory in Mysql.fetch_json_result().
    • Fixed locking problem in Mysql.
    • Fixed dangling cache reference in prepared statements when using CREATE statements via Sql.pgsql.
    • Fixed broken (auto-re-) connect logic in Sql.pgsql.
    • Support all text queries in Sql.pgsql.
    • Fixed rare string corruption in Sql.pgsql.
  • SSL/TLS
    • Fixed bugs in handling of abrupt SSL3 remote close. sslfile now also triggers EPIPE errors if the user attempts to read or write to a connection that has been abruptly closed (to detect truncation attacks better).
    • Fixed issue where SSL3 async_client objects became garbage even after close().
    • Tools.X509 uses a unique serial number for each selfsigned certificate. Firefox otherwise complains about serial number clashes.
  • Misc
    • System.normalize_path no long fails for files with 8-bit chars in their names.
    • Improved widestring support for Parser.Tabular.
    • Fixed a problem in "pike -x module" that prevented auto tools from running in the correct directory.
    • Fixed issue where GTK objects weren't destroyed properly. http://pike.ida.liu.se/docs/tutorial/hello/window.xml now works again.
    • Fixed encoding/decoding bug in "pike -x httpserver" that prevented files with space in the name from being accessable.
    • Parser.XML.Tree contains fixes for add_child_before() and add_child_after().
    • Parser.Tabular fixes support for Stdio.File objects. For example:

      • Pike v7.8 release 468 running Hilfe v3.5 (Incremental Pike Frontend)

      > Parser.Tabular(Stdio.File("data.csv"), Stdio.File("format.csv"));

    • Fixed return value from Regexp.PCRE.exec to always have entries for all submatches.
    • Fuse has fixed support for readlink().
    • Fixed infinite loop in Audio.Format when reading MP3 frames.
    • Added path to pike binary in pike -x module used by verify targets. Should also fix problems installing modules using pike -x monger.
    • Reduce optimization in gethostby{addr,name} that could cause overzealous caching or results.

Building, installing and developer tools

  • Fixes to allow Regexp.PCRE to compile against recent versions of PCRE lib.
  • Fix Gz module build on recent Gentoo Linux.
  • Fixed some library searchpath issues.
  • Some potential fixes for --enable-pikelib mode on platforms had problems with the final linking.
  • Some issues with the support for building modules needing a C++ compiler have been fixed.
  • Made it possible to override $(OS) with an environment variable $PIKE_BUILD_OS.
  • Check for configure scripts built with pre-propagated_variables pike.
  • Added Nettle/config.guess that knows about MaxOS X and x86_64.
  • Fix broken detection of libjpeg lossless transformation support.
  • Installer will no longer dump sql driver files that depend on system libraries.
  • Fixed support for --without-threads.
  • Fix to make dump.pike dump itself just like any other module class.
  • Improved support for LIBPIKE.
  • Added PIKE_CORE define to detect when not compiling a module.
  • Improved support for autoconf 2.6x.
  • Ensure that --with-rtldebug enables assertions.
  • Configure script now attempts to set up the PKG_CONFIG_PATH.
  • Updated the ABI selector for MacOS X. Default to attempting the compiler default ABI (instead of forcing 32-bit ABI).
  • CMOD Enhancements
    • Tools.Standalone.precompile supports ranges for strings and negative int(low..high) bounds.
    • Fixed bug in generate_overload_func_for() when there are alternatives accepting zero arguments.
  • Tools.Testsuite
    • Added some functions to make pretty logging in tests easier.
    • Tools.Standalone.test_pike passes the verbosity level in an environment variable TEST_VERBOSITY to subtests.
    • Tools.Testsuite accumulates multiple result reports.