2016-02-08

Racket Web Server Security Vulnerability

We recently discovered a serious security vulnerability in the Racket web server, which can lead to unintended disclosure of files on the machine running the web server. This vulnerability is fixed in Racket version 6.4, just released, and we encourage people to upgrade to that version.

The vulnerability affects web servers that serve static files using the #:extra-files-paths option, including the default value of this option. If you do not use the Racket web serve to serve static files, or you do so via a mechanism that does not use the make-url->path function, then you are likely not vulnerable. Affected web serves will allow specially-crafted URLs to access files outside of the specified paths, potentially exposing any file that the web server process is able to read.

If you cannot immediately upgrade to version 6.4, we have provided a package catalog with updated versions of the "web-server-lib" package for versions of Racket back to 6.0. That catalog is located at

 http://download.racket-lang.org/patches/web-server-1/

To use it to upgrade your Racket installation, add it as a catalog using raco pkg config. To make this process easier, you can download the Racket script available here. Then run:

  $ racket add-catalog.rkt
  $ raco pkg update -i web-server-lib

If you need advice on updating Racket installations older than version 6.0, please let us know and we will provide them.

To test that your Racket installation is fixed, you can run the program here. It will print whether your installation is out of date.

Racket v6.4

Racket version 6.4 is now available from http://racket-lang.org/

  • We fixed a security vulnerability in the web server. The existing web server is vulnerable to a navigation attack if it is also enabled to serve files statically; that is, any file readable by the web server is accessible remotely. For more information, see this post.
  • DrRacket's scrolling is faster.
  • Incremental garbage-collection mode can eliminate long pauses in a program. For example, incremental mode is useful for avoiding pauses in games and animations.
    Programs must specifically request incremental mode with (collect-garbage 'incremental), but libraries such as 2htdp/universe include the request as part of the library's implementation.
  • The default package catalog is an HTTPS address instead of HTTP, and package operations properly validate server certificates when using HTTPS.
  • Documentation may define their own categories for the manual top-level page by using strings, rather than only symbols that name pre-defined categories.
  • The Racket cheat sheet is included in the main distribution.
  • DrRacket is available in Bulgarian, thanks to Alexander Shopov.
  • The contract Typed Racket generates for the Any type is more permissive, allowing more typed/untyped programs to work without contract errors.
  • Redex supports binding specifications; describe which variables bind in which expressions and your metafunctions and reduction relations automatically become scope-sensitive. Thanks to Paul Stansifer for this improvement.
  • All pict functions accept pict-convertibles. This allows for transparent interoperability between pict and libraries like 2htdp/image.
  • The raco profile and raco contract-profile commands provide easy access to profiling tools, without requiring program modifications.
Feedback Welcome

2015-11-23

Racket v6.3

Racket version 6.3 is now available from http://racket-lang.org/

  • Racket's macro expander uses a new representation of binding called "set of scopes". The new binding model provides a simpler explanation of how macros preserve binding, especially across module boundaries and in hygiene-bending expansions. The new expander is mostly compatible with existing Racket macros, but there are some incompatibilities. For the formally inclined, a research paper on this macro system will appear at POPL next year: http://www.cs.utah.edu/plt/scope-sets/
  • Racket's GUI library now uses Gtk+ 3 when available, instead of Gtk+ 2. Set the PLT_GTK2 environment variable to select Gtk+ 2.
  • Added a new Redex tutorial based on a week-long workshop in SLC.
  • Better syntax error checking for Redex patterns that do not use holes correctly.
  • The blueboxes are more agressive about finding names to look up in the docs, meaning they are useful much more often.
  • Submodules are now fully supported in Typed Racket. Previously, some uses of submodules would produce internal errors, making it hard to module+ test and module+ main effectively in Typed Racket. The switch to the set-of-scopes expander fixed these problems, and submodules are now happily at home in Typed Racket.
  • The typed/racket/unsafe library provides import and export forms that circumvent contract generation. This improves performance for typed-untyped interaction at the cost of safety and debuggability.
  • Typed Racket provides experimental support for units (from racket/unit).
  • The experimental define-new-subtype form allows overlaying finer distinctions between otherwise identical types, similar to Haskell's new type.
  • The Promise type constructor changes in a backwards-incompatible way to exclude promises created with promise/name.
  • The unstable-* packages are out of the main distribution. Most of their contents have been either merged with established Racket libraries or spun off as their own packages. This change is backwards compatible for packages that properly list their dependencies. Full details: http://blog.racket-lang.org/2015/10/retiring-unstable.html
  • edu: big-bang supports a display-mode clause so that world programs can take over the entire screen.
Feedback welcome

2015-10-30

Retiring unstable

Some of you may be familiar with the unstable collection, whose
purpose was to serve as a staging ground for new APIs that hadn't yet
found a more permanent home. With the advent of the package system,
packages can serve that same purpose, which removes the need for a
dedicated unstable collection provided by the main distribution.

For this reason we are moving unstable-* packages out of the main
distribution.

For backwards compatibility, the packages remain available from the
package catalog. Packages that properly list their dependencies (as they
should! it's an error not to!) are unaffected by this change. Packages
that are missing dependencies may need to be adjusted to include the
appropriate unstable dependencies.

The unstable packages contained many useful functions and APIs, and we
merged many of them into established Racket libraries. Others were spun
off as their own packages. The remaining APIs, which we judged too
narrow or too immature, we left in unstable packages, where they are
still available in their original form. In all cases, the original
unstable libraries continue to export the same bindings they always
did, to ensure backwards compatibility.

For completeness, here is a list of the fate of each unstable library
that used to be part of the main distribution.

  • unstable/2d

    • Moved to the 2d package.
  • unstable/arrow

    • Left in unstable-lib.
  • unstable/automata

    • Moved to the automata package.
  • unstable/bytes

    • Left in unstable-lib.
  • unstable/class-iop

    • Moved to the class-iop package.
  • unstable/contract

    • Moved non-empty-string? to racket/string.
    • Moved port-number? and tcp-listen-port? to racket/tcp, the
      latter renamed to listen-port-number?.
    • Moved if/c, failure-result/c, predicate/c and
      rename-contract to racket/contract.
    • Moved treeof to plot/utils.
    • Moved sequence/c to racket/sequence.
    • Left path-piece?, maybe/c, truth/c in unstable-contract-lib.
  • unstable/custom-write

    • Moved make-constructor-style-printer to racket/struct.
    • Left prop:auto-custom-write in unstable-lib.
  • unstable/debug

    • Left in unstable-debug-lib.
  • unstable/define

    • Left in unstable-lib.
  • unstable/error

    • Left in unstable-lib.
  • unstable/find

    • Left in unstable-lib.
  • unstable/flonum

    • Superceded by math/flonum. Left in unstable-flonum-lib.
  • unstable/function

    • Merged with racket/function.
  • unstable/future

    • Merged with racket/future.
  • unstable/gui/notify

    • Moved to framework/notify, with naming changes.
  • unstable/gui/pict

    • Moved color/c, light, dark, red, orange, yellow, green,
      blue, purple, black, brown, gray, white, cyan, and
      magenta to pict/color.
    • Moved show, hide, pict-if, pict-cond, and pict-case to
      pict/conditional.
    • Merged scale-to with pict's scale-to-fit.
    • Merged ellipse/border, circle/border, rectangle/border,
      rounded-rectangle/border with pict's ellipse, circle,
      rectangle, and rounded-rectangle, respectively.
    • Merged pin-label-line, pin-arrow-label-line, and
      pin-arrows-label-line with pict's pin-line, pin-arrow-line,
      and pin-arrows-line, respectively.
    • Moved blur, shadow, and shadow-frame to pict/shadow.
    • Moved unstable/gui/pict/align to ppict/align, in the ppict
      package.
    • Left color, pict-match, pict-combine, with-pict-combine,
      fill, strike, shade, blur-bitmap!, arch,
      draw-pict-centered, backdrop, cross-out, and
      make-plt-title-background in unstable-lib.
  • unstable/gui/ppict

    • Moved to the ppict package.
  • unstable/gui/prefs

    • Moved to framework/preferences, with naming changes.
  • unstable/gui/redex

    • Left in unstable-redex.
  • unstable/gui/scribble

    • Left in unstable-lib.
  • unstable/gui/slideshow

    • Moved with-size, with-scale, big, small, with-font,
      with-style, bold, italic, subscript, superscript, caps,
      and blank-line to slideshow/text.
    • Moved slide/staged, staged, stage, stage-name, at,
      before, after, before/at, after/at to the staged-slide
      package.
    • Left column, columns, column-size, two-columns,
      mini-slide, tabular, reveal, revealing-slide, and
      items-slide in unstable-lib.
  • unstable/gui/snip

    • Left in unstable-lib.
  • unstable/hash

    • Merged with racket/hash.
  • unstable/latent-contract

    • Left in unstable-latent-contract-lib.
  • unstable/lazy-require

    • lazy-require has been in racket/lazy-require for some time.
    • Left begin-on-demand in unstable-lib.
  • unstable/list

    • Moved check-duplicates, remf, remf*, group-by,
      cartesian-product, list-update, and list-set to racket/list.
    • Moved list-prefix?, take-common-prefix, drop-common-prefix,
      and split-common-prefix to racket/list, with slight API changes
      to harmonize with Racket's list API.
    • Left filter-multiple, extend, map/values, and map2 in
      unstable-list-lib.
  • unstable/logging

    • Moved with-intercepted-logging and with-logging-to-port to
      racket/logging.
    • Left start-recording and stop-recording in unstable-lib.
  • unstable/macro-testing

    • Moved to syntax/macro-testing.
  • unstable/markparam

    • Moved to the markparam package.
  • unstable/open-place

    • Moved open-place to racket/place, and renamed it place/context.
  • unstable/options

    • Moved to the option-contract package.
  • unstable/parameter-group

    • Moved to the parameter-group package.
  • unstable/pretty

    • Merged pretty-format/write, pretty-format/display, and
      pretty-format/print with racket/pretty's pretty-format.
    • Left break-lines in unstable-pretty-lib.
  • unstable/recontract

    • Merged with racket/contract some time ago.
  • unstable/sandbox

    • Merged with scribble/eval.
  • unstable/sequence

    • Moved in-syntax and in-slice to racket/sequence.
    • Left in-pairs, in-sequence-forever, and sequence-lift in
      unstable-lib.
  • unstable/socket

    • Moved to the unix-socket package.
  • unstable/string

    • Left in unstable-lib.
  • unstable/struct

    • Moved struct->list to racket/struct.
    • Left make in unstable-lib.
  • unstable/syntax

    • Moved make-variable-like-transformer to syntax/transformer.
    • Moved syntax-source-directory and syntax-source-file-name to
      syntax/location.
    • Left explode-module-path-index, phase-of-enclosing-module,
      format-unique-id, syntax-length, and syntax-within? in
      unstable/syntax.
  • unstable/temp-c

    • Moved to the temp-c package.
  • unstable/time

    • Left in unstable-lib.
  • unstable/wrapc

    • Moved to syntax/contract.

2015-09-22

Racket Package Server Security Vulnerabilities

Recently, we discovered several security vulnerabilities with how both the Racket package catalog server and the Racket package client work. The vulnerabilities have now all been fixed, and we do not know of any exploitation of them. However, we encourage you to take the following steps:
  • Change your password on the http://pkgs.racket-lang.org site.
  • Check any packages you have uploaded to the site, to ensure that no unexpected changes have been made to them.
  • Do not use the released versions of the raco pkg catalog-archive command, or the file/untar and file/unzip libraries, on untrusted inputs. If you use these tools or libraries, use a snapshot build available from http://pre.racket-lang.org/.

The errors, and how they were fixed

A total of 5 errors related to package handling were reported to us by Tony Garnock-Jones and Asumu Takikawa. Two were XSS vulnerabilities relating to handling user input in the package administration dialog. One was an error where unsanitized email addresses with path name components in them could allow a malicious user to impersonate someone else, whom they shared an email suffix with (such addresses are illegal on most mail servers, like Outlook and GMail, but not illegal in SMTP itself.) Two were errors in handling MANIFEST files and tar/zip archives, which allowed decompression to write to arbitrary locations on the file system. These last errors affected not only the server, which decompresses packages to analyze them, but also clients using the commands described above.

The relevant server-side code was fixed to appropriately sanitize user input. The package handling libraries now reject any attempts to navigate up the filesystem hierarchy, meaning that these attacks are no longer possible.

Unfortunately, due to the nature of these attacks, we cannot be sure that they were not exploited, but we have no evidence that they were. Therefore, we encourage anyone with an account to change their password, and to treat the password as compromised. Please also check your existing packages to make sure they are as you left them.

Furthermore, using the file/untar and file/unzip libraries, the raco pkg catalog-archive command, and the internal functions that manipulate packages is not safe on untrusted inputs in released versions of Racket. Since raco pkg install executes code, it is already unsafe to use on untrusted packages, but simply extracting malicious packages is also unsafe.

We have not released a new version of Racket, but encourage anyone who needs to perform these commands to use a snapshot build. The next version of Racket will be released on-schedule in October. If, however, you would benefit from a patched version of Racket 6.2.1, please let us know.

2015-08-11

Modules, Packages and Collections

Racket, the Racket docs and Racketeers use a number of terms to refer to various units of Racket code. Of those, module, package and collection refer to related but distinct concepts. Their exact  relations and distinctions can be confusing for new users. This is an attempt at explaining those concepts, what they are for, and how they relate to each other.

To begin with the smallest of the three, a file that begins with #lang and the name of a language is a module. There are also other ways to construct modules, but let's not worry about those.

A module is the basic unit of functionality for Racket code.

Once your Racket programs get larger, though, you'll want to split them over multiple modules. This allows you to organize your source better, enables separate compilation, and makes it possible for you to mix and match modules written in different Racket languages (Racket, Typed Racket, Datalog, Scribble, etc.).

That's where packages and collections come in. They help you organize your modules.

A package is an group of modules that you can install together, and that usually provide one piece of functionality. To pick a random example, take the pict3d package from pkgs.racket-lang.org. That package is a collection of modules which together implement a functional 3D engine. You can install it using raco pkg install pict3d, or via the graphical package manager in DrRacket.

So, to sum up, packages are units of code distribution.

A collection is a group of modules whose functionality is related to the same topic, for example data structures (the data collection), or wrapper libraries for use with Typed Racket (the typed collection). Modules are referred to and required using collection paths. For example, when you require racket/class, you're requiring the class module from the racket collection.

Modules within a collection do not necessarily come from the same package, and may not be developed together. For example, some data structures in the data collection are provided as part of the core of Racket, such as the integer sets in data/integer-set. Other data structures are provided by additional packages which you may need to install separately, such as the hash-array-mapped tries in data/hamt, which are provided by the hamt package. Having both of those in the data collection signals that they both provide data structures. If you develop your own data structures, putting them in the data collection is probably the right thing to do.

Many packages, however, provide functionality that does not fall under existing categories, and provide their own, new collection. For example, the pict3d package we discussed above puts its modules in the pict3d collection. For that reason, the distinction between package and collection is sometimes a bit blurred.

So, to sum up, collections are units of code classification.

The term library does not have a technical meaning in Racket. We usually use it to refer to a package, or to a set of packages that are developed together. For example, the Rackunit library is split across multiple packages: rackunit, rackunit-lib, rackunit-gui, rackunit-plugin-lib, rackunit-doc and rackunit-test. This allows packages to only depend on part of Rackunit. For example, a package for a string-processing library probably should not depend on the Racket GUI library (to be deployed on headless servers, for example), and so should depend on the rackunit-lib package for its testing, instead of on the full rackunit package, which brings in GUI support via the rackunit-gui package, and would introduce a dependency to Racket's GUI library.

Hopefully, this clarifies the Racket code organization terminology a bit.

2015-08-10

Racket v6.2.1

Racket v6.2.1 is now available from http://racket-lang.org/

Version 6.2.1 patches the recent v6.2 release in three small ways:

  • For the How to Design Programs teaching languages, DrRacket offers an option to use the old style for printing the constants true, false, and empty instead of #true, #false, and '().
  • The teaching languages come with some additional functions to match the August 2015 stable release of HtDP 2nd edition.
  • A repair to the compiler avoids an infinite loop at compile time for certain expressions that should loop forever at run time.
Feedback Welcome