Pike 8.0.1116 Release notes

Changes since Pike 8.0.702 (release 13)

New Features

• Core

  • Added “->?” as the safe indexing variant of “->”.

  • Added int(7,8,16bit).

• ADT.History

  Support encode/decode_value().

• Pike.FakeObject

  This is used as a placeholder used in place of other objects in backtraces.

• Pike.DestructImmediate

  An empty class that can be inherited to get the PROGRAM_DESTRUCT_IMMEDIATE flag set.

• Image.Dims.exif_get and Image.exif_decode

  New convenience functions to get the functionality of Image.Dims.exif_get_JPEG and Image.JPEG._decode (rotation of the image according to the EXIF orientation information) without needing to check if the image is JPEG first.

• Concurrent

  • Add delay() to postpone a future.

  • Rewrite timeout() to reduce the memory footprint of a future in the common case.

  • Added query_{success,failure}_callbacks() and try_get() to Concurrent.Future.

• Process.Process

  • On NT it is now possible to specify the preferred console handler via the “conpty” option. The default is to take it from the first pty of “stdin”, “stdout”, and “stderr”. If none of them is a pty the console handler is inherited from the current process.

• SSL

  The SSL module now supports session ticket both client and server side. This functionality has been backported from 8.1/master.

• Standards.X509

  Standards.X509 now allows the user to select acceptable signature algorithms to be used in the verification of certificates.

• Stdio.Buffer

  Added truncate() and the ability to search for substrings.

• Stdio.File

  Support openat() with a single argument().

• Stdio.PROP_TTY

  Stdio.File()->pipe() can now be used to create tty-capable pipes (aka pseudo-ttys, aka ptys).

Bug fixes

• ADT.CritBit.FloatTree

  Fix FloatTree when using 128bit floats.

• ADT.CritBit.IntTree

  Fixed the signature of ADT.CritBit.IntTree()->_values().

  values(ADT.CritBit.IntTree()) claimed to return an array(int), whereas IntTrees can hold any type as value.

  Hide mutex keys and crypto contexts from backtraces.

  Having unexpected references to mutex keys may cause hangs and complaints about attempting to lock mutexes recursively.

• combine_path()

  Fixed corner case where combine_path() on strings starting with “./../“ used to gain one directory level. Eg:

  Old behavior:

  combine_path(“.”, “../foo”); (1) Result: “../foo” combine_path(“./..”, “foo”); (2) Result: “foo” combine_path(“./../foo”); (3) Result: “foo”

  New (fixed) behavior:

  combine_path(“.”, “../foo”); (1) Result: “../foo” combine_path(“./..”, “foo”); (2) Result: “../foo” combine_path(“./../foo”); (3) Result: “../foo”

• describe_backtrace()

  • Do not use value identifiers that have not been defined in the visible backtrace.

  • Fix clipped/canclip propagation.

• mktime()

  Fix zone handling when localtime is in daylight savings time.

• strftime()

  Remove non-standard conversions %k and %l.

• types()

  Fixed reference leak.

• crypt()

  Do not throw errorss on validation failure. Primarily ocurrs on RHEL 8, return a failure code instead, consistent with historical use.

• Backend.PollBackend

  Signal read events on POLLHUP.

• Calendar

  • Fixed typo in the Catalan word for the month December.

  • Updated tzdata to 2019c.

• Compiler

  • Adhere to the calling convention more in the machine code generator on amd64, namely by keeping the stack pointer aligned before calling into C code. GCC 8 started to emit instructions relying on the correct, alignment, causing General Protection Faults.

  • Fixed some corner cases where file names where missing from backtraces.

  • Check the proper flag to determine if a local variable may be used before optimizing away certain assignments if the assignment is immediately followed by a return of the variable in question.

• Concurrent.Promise

  • Fixed race condition where success- and/or failure- callbacks could get lost.

  • Convert errors thrown by the executor passed to Concurrent.Promise() on create into promise rejection.

  • Fixes for some recursive mutex lock errors.

• Crypto.ECC

  Now works against Nettle 3.5.

• Filesystem.Tar

  Fix extraction of S_ISUID and S_ISGID bits.

  POSIX mandates that after a chown() (by unprivileged users), the S_ISUID and S_ISGID bits are cleared, Linux 2.2.13 removed the special case for root. The order of chmod() and chown() have been reordered trying to fix an issue with applying utime() under Windows. With this change, utime() is applied first, then chown() and then chmod(), restoring the historical order for chown() and chmod().

• Gettext

  Prevent argument of textdomain from being empty.

• Image.Dims

  • Fixed the inconsistency that Image.Dims.get_JPEG (and thus Image.Dims.get) would flip the dimensions on JPEGs with EXIF orientation information, even though Image.JPEG.decode does not actually perform the rotations.

• Inotify

  Improved handling of invalid event callbacks. [bug 8042]

• Odbc

  The module’s configure script now respects ABI suffixes when searching for the ODBC library dir.

• Parser.HTML

  Fixed a Pike stack overflow occuring with certain inputs to the HTML parser.

• Pike.count_memory

  Prevents a loop with ADT.List objects.

• Process.run

  • Don’t error if stdin modifier is supplied as the empty string. [LysLysKOM 23099651]

• Protoocls.HTTP

  • Protocols.HTTP.do_async_proxied_method() did not pass on request headers unless credentials were given to the proxy or the protocol to be used was HTTPS.

  • Add Host header to proxied requests.

• Protocols.HTTP.Server

  Fixed bug in Request()->update_mime_var(). [bug 10033]

• Protocols.HTTP.Session

  • Fix race-condition when multiple threads call give_me_connection() with the same url at the same time.

  • Don’t attempt to reuse SessionQuery objects.

    give_me_connection() now always returns a new SessionQuery object (albeit sometimes with a reused connection).

    This should alleviate issues with stuff being left around from previous queries.

• Protocols.DNS

  Improved error handling when calling async callbacks.

  Survive Stdio.UDP()->bind() failing with EACCES.

• Protocols.HTTP.Query

  On Solaris 11 Stdio.File()->connect() often fails with EADDRINUSE. If this happens, retry the connection.

• Protocols.Websocket

  Tentative fix for error ’indexing the NULL value with “set_nonblocking’.

• Runtime

  • Certain operations on arrays of only objects could have triggered a type confusion under some circumstances, causing integers to be freed as objects, and thus a NULL derefence (SIGSEGV).

  • Fix some gratuitous rounding to double with long-double-precision.

  • Improved Fix some gratuitous rounding to double with long-double-precision.

• Search.Filter.HTML

  Fix title always being set to “”. [bug 10020]

• Sql.pgsql

  • Ensure that the db connection is closed when the portal is closed.

  • Small speed boost.

  • Fix the only known deadlock-cause logged in the last year; happened (rarely) on multiple parallel queries on a single connection.

  • Fix the only known race condition logged in the last year; happened on some queries that store >32KB data per query.

  • Cast integers to floats, when floats are desired.

  • Simplify error handling.

  • Toggle cache_autoprepared_statements default to off; turning it on triggers a bug in PostgreSQL sometimes that causes spikes in CPU usage of the database.

  • Fix support for text-multiqueries (separated by semicolons).

  • resync() is synchronous now, and prevents race conditions if followed back-to-back with new queries.

  • disregard pending statements on connection destruction.

• SSL

  • When verifying the hostname against the certificate, only accept (wildcard) for one level, i.e. .example.com matches foo.example.com, but not www.foo.example.com. Previously, all levels were matched.

  • No longer accept MD2, MD5, and SHA1 as signature algorithms when verification is requested. MD2 and MD5 are outright dangerous at this point, and SHA1 should no longer be very relevant as web browsers already force CAs for several years to no longer issue any SHA1 certs anymore.

    Acceptable signature algorithms are however configurable via SSL.Context now, in case support of older/insecure algorithms is vehemently required by an application.

• SSL.File

  • Do not close automatically on write error.

    The internal ssl_write_callback() would call shutdown() on write error, which would cause the object to enter one of the closed states, without the user actually having closed the file.

    This in turn caused errors like “Not open” from functions like read() and set_nonblocking().

  • Fix a situation where setting callbacks before the TLS handshake had completed would cause the connection to hang if an alert was generated (for instance when the peers are unable to agree on a cipher suite.)

  • Permit data received to be read() after the connection is closed by the peer. Previously, read would throw a “Not open” error rendering the data irretrievable.

• Standards.JSON

  • encode() now allows other threads to run every now and then.

  • Fix floating point literals with long-double-precision

• Standards.EXIF

  Improved detection of EXIF section in get_properties().

• Standards.URI

  • Updated list of unsafe characters to RFC 3986. [bug 8004]

  • String representations of the URI that are generated by the object are now canonical again. This also fixes an issue where Standards.URI instances compared identical under the == comparison, but would hash differently.

• Standards.X509

  Use old syntax for compatibility.

• sybase

  The module’s configure script now respects ABI suffixes when searching for the sybase library dir.

• System

  • System.openlog() now keeps a reference to the ident passed, since system level syslog() may access the string pointed to by the ident passed to openlog. The reference will be released after a subsequent call to openlog() or closelog().

• Thread.MutexKey

  When simulated, mark for immediate destruction.

• Thread.Queue

  Attempt to work around glibc bugs.

  Glibc 2.24 and earlier have a broken implementation of condition variables (cf https://sourceware.org/bugzilla/show_bug.cgi?id=13165).

  Attempt to work around the issue by having the signalling thread release the associated mutex before signalling.

• Thread.ResourceCount

  Fixed mutex handling.

• Tools.Standalone.httpserver

  The builtin webserver tool shipped with Pike (pike -x httpserver) was previously susceptible to a directory traversal attack via URL encoding.

Building & Tools

• Add –with-exclude-site-prefixes configure option.

  This will cause any directory specified in this option to be excluded from the search path for binaries/includes/libraries. This option will override any paths found by other means, allowing “system” directories like /usr/local to be exluded for making binary distributions.

• Drop support for Postgres 7.1.

• Fix compilation with Nettle 3.1.1.

• Support recent versions of libfreetype.

• Drop use of the obsolete (~20 years) Solaris ioctl /proc API.

  The ioctl-based /proc API was obsoleted in Solaris 2.6 (July 1997), but to be supported “until the next major release”. In Solaris 11.4 (August 2018) the support has now finally been removed.

  Most of the removed code was from 1995, and was thus older than Pike…

• Improved pkg-config search path detection on Solaris.

  Solaris 11.4 provides a single pkg-config binary, which by default searches in /usr/lib/pkgconfig/ and /usr/share/pkgconfig/. Unfortunately, /usr/lib/pkgconfig/ contains information for the 32-bit packages (64-bit d:o is in /usr/lib/64/pkgconfig/).

  This issue caused eg 32-bit headers for libffi to be used, causing a SIGSEGV in ffi_closure_unix64_inner() due to the differing sizes of ffi_closure in 32-bit and 64-bit abi.

• Check for actual availability of YP functions, not just headers.

• Attempt to use paxctl to alter the PaX flags for the pike binary to allow for generation of machine code on NetBSD.

• Silence GL deprecation warning and fix a missing definition error on Darwin for newer versions of MacOS.

o Fix check for my_bool and compilation errors related to MySQL 8.