The Racket Blog

Scribble Your Blogs

2012-04-02T18:46:00.000-04:00

Scribble is a great language for writing documentation. Now it’s a great language for writing blog posts, too. I’ve just released a tool called Scriblogify that compiles Scribble documents and posts them as blog entries. Scriblogify is a more polished and automated version of the scripts I’ve been using for several months to prepare posts for my own blog.

To get Scriblogify, just download it from PLaneT:

(require (planet ryanc/scriblogify:1))

raco planet install ryanc scriblogify.plt 1 0

The package automatically installs a raco subcommand (raco scriblogify) that can be used to configure Scriblogify and process and upload blog posts.

Configure Scriblogify by running

raco scriblogify --setup

That will open a browser window with the Scriblogify configuration servlet. The servlet will prompt you to authorize Scriblogify to access your Blogger and Picasa Web Albums accounts (only the Blogger/Picasa combination is currently supported) and then create one or more profiles—named combinations of blogs and web albums to upload to.

Scriblogify automatically handles images computed in your Scribble documents by uploading them to a web album. For example, here are some images computed with the slideshow/pict library:

> (require slideshow/pict)

> (define rainbow-colors

    '("red" "orange" "yellow" "green" "blue" "purple"))

> (for/list ([c rainbow-colors])

    (colorize (filled-rounded-rectangle 20 20) c))

'(     )

> (for/list ([c rainbow-colors]

             [dir (in-cycle '(right left))])

    (standard-fish 25 25 #:color c #:direction dir))

'(     )

> (cc-superimpose

   (cc-superimpose (cloud 100 80 "lightblue")

                   (cloud 90 70 "white"))

   (hc-append 10

    (standard-fish 30 30 #:color "red" #:direction 'right)

    (standard-fish 25 20 #:color "blue" #:direction 'left)))

By Scribbling your blog entries, you get Scribble’s nice code formatting, colorizing, and documentation links for free—well, once you’ve updated your blog’s CSS (see below). If you’re blogging about bleeding-edge work, there’s an option to make Scriblogify link to the nightly build docs (updated daily) instead of the release docs (updated every 3 months).

Scriblogify’s documentation has more details, including how to update your blog’s CSS for Scribbled content and what bloggable Scribble documents look like.

You can see the source for this blog post here. This blog entry was created with the following command:

raco scriblogify -p the-racket-blog scribble-your-blogs.scrbl

Now go forth and Scribble your blogs.

Fixed Racket v5.2.1 Installers

2012-03-26T19:55:00.001-04:00

Dear Racketeers,

We have just released a DrRacket version 5.2.1 that starts fine today. The fixed version has replaced the 5.2.1 installers. This version and the original 5.2.1 differ only in this one fix.

Racket v5.2.1

2012-02-02T15:30:00.001-05:00

Racket version 5.2.1 is now available from

http://racket-lang.org/

Release Highlights:

Performance improvements include the use of epoll()/kqueue() instead of select() for the Racket thread scheduler, cross-module inlining of small functions, and the use of SSE instead of x87 for JIT-compiled floating-point operations on platforms where SSE is always available (including x86_64 platforms). A related change is the interning of literal numbers, strings, byte strings, characters, and regexps that appear in code and syntax objects.
DrRacket uses a set of composable ray-traced icons available from the new images library collection.
Typed Racket's typecheck-fail form allows macro creators to customize the error messages that Typed Racket produces. This is especially useful when creating pattern matching macros.
The performance of Redex's matcher has been substantially improved; depending on the model you should see improvements between 2x and 50x in the time it takes to reduce terms.
Plots look nicer and are more correct at very small and very large scales. New features include customizable dual axis ticks and transforms (e.g., log axes, date and currency ticks, axis interval collapse and stretch), stacked histograms, and 3D vector fields. The legacy fit function and libfit have been removed.
The 2htdp/universe library's big-bang form supports an experimental game pad key handler.
The db library now supports nested transactions and PostgreSQL arrays. Bugs involving MySQL authentication and memory corruption in the SQLite bindings have been fixed.
The Macro Stepper tool in DrRacket no longer executes a program after expanding it.
In the DMdA teaching languages, infinite recursive signatures ("streams", for example) with no intervening mixed are now supported, and the signatures of record definitions without fields now have generators for use with property.
MysterX's ActiveX support is deprecated and will be removed in the next release. MysterX's core COM functionality will become deprecated in the next release, but COM functionality will be supported for the foreseeable future as a compatibility layer over a forthcoming ffi/com library.

Zack Galler's Experience with Stateful vs Stateless Web Apps

2012-02-01T19:25:00.000-05:00

Communication using HTTP between client and server is a simple problem of halted computation.

A client computes a request, transmits and halts, waiting for a server response. On receipt, the server computes a response, transmits and halts, waiting for the next client request.

This much is well known.

Racket's magnificent stateful Web server does three things on the server side:

it reifies a Racket continuation, capturing where the server computation has halted.
it externalizes the continuation, creating a URL-representation that uniquely maps to the Racket continuation
it disseminates the externalized continuation to interested clients, typically via HTTP response, but alternately via SMTP or any other protocol.

Then, it waits.

Later, when presented with an externalized continuation, a quick inverse mapping occurs, the underlying Racket continuation is invoked, and the server processes the new client request.

Rinse and repeat.

The problem with this approach is twofold

the reified Racket continuations live in server memory. And there's no safe way to garbage collect, as the continuations could be invoked at any time. There are strategies to reclaim memory, but some load level will noticeably decrease the performance of your application. And its not possible to figure out what that load level is prior to finishing your application. This is a problem.
Again, the reified Racket continuations live in server memory and cannot be moved. So there's no way to scale an application to more than one server. It's a necessarily one machine system. This makes problem #1 worse.

Racket's yet more magnificent stateless Web server does exactly the same three things:

to reify, it rewrites the entire call stack into a format known as A-Normal Form (ANF).
to externalize, the ANF'd stack is encoded for transmission over HTTP.
and then it's sent over to the client (dissemination).

Later, when presented with encoded stack, the stateless server performs an inverse transform to reconstruct the call stack, at which point the server keeps going.

So we've lost the invocation step and substituted a reconstruction.

But in exchange, we've eliminated continuations from server memory, and solved both enumerated problems above. Neat trick.

I provide a few lessons learned for the archives for the next person to attempt porting #lang racket to #lang web-server code.

First, the predicate serializable? from racket/serialize is invaluable. The #lang web-server code will not transform if there are non-serializable constructs in the dynamic extent of the invocation of send/suspend, such as a local binding or argument.

Second, invocations of native continuations reified with call/cc frequently throw errors related to continuation prompts, such as “attempt to cross a continuation barrier” or “no corresponding prompt tag in continuation”. In all cases, I was able to remedy the situation by enclosing the invocation in call-with-continuation-prompt. This may be an error in the system, but it is unclear at this time.

Third, the transformation does not allow parameters or dynamic-wind, because the internal data-structures representing them are not serializable, but continuation-marks can be used to reimplement the piece of the functionality you need.

Finally, thank you to the Racket team. I think the stateless Web language is important technology and must have required an enormous amount of work to implement.

Anecdotally, application speed seems at or better than the stateful code.

To learn more about the stateless Web application infrastructure, consult the manual or post to the mailing list.

(This post was written by Zack Galler with minor edits before posting by Jay McCarthy.)

Racket v5.2

2011-11-09T22:49:00.000-05:00

Racket version 5.2 is now available from

http://racket-lang.org/

Release Highlights:

DrRacket comes with an experimental, on-line check syntax tool, although this new tool is disabled default. See below for more information.
The new db library offers a high-level, functional interface to popular relational database systems, including PostgreSQL, MySQL, and SQLite, as well as other systems via ODBC.
A new XREPL collection provides convenient commands for a plain racket REPL. It is particularly convenient for people who prefer console-based work and alternative editors. See also the new chapter on command-line tools and other editors at the end of the Racket Guide.
The plot collection has been reimplemented in Racket. It now offers PDF output, log axes, histograms, and more. Some code that uses plot will still work, and some will need light porting. The plot/compat module offers expedient backward compatibility.
DrRacket uses more conventional key bindings: C-t creates a new tab, C-w closes the current one, and C-r runs the definitions. On Mac OS X, the Command key is used. See “Defining Custom Shortcuts” in the DrRacket manual for an example that uses the old key bindings.
The new raco link command registers a directory as a collection, which allows the collection directory to reside outside the “collects” tree and without changing the PLTCOLLECTS environment variable.
Typed Racket:
- Typed Racket provides static performance debugging support to show which code gets optimized and point out code that does not. Use the “Performance Report” button in DrRacket.
- More intuitive types in printouts in the REPL and in error messages. Use :query-result-type to explore types, or :print-type for a full printout.
- Typed Racket now supports defining function with optional arguments using the same syntax as Racket.
Redex now supports specifying (and testing and automatically typesetting) judgment forms including type systems and SOS-style operational semantics.
Fixed several GUI problems, including problems on Ubuntu 11.10 (GTK+ 3) and 64-bit Mac OS X.
Internal-definition expansion has changed to use let* semantics for sequences that contain no back references. This change removes a performance penalty for using internal definitions instead of let in common cases, and it only changes the meaning of programs that capture continuations in internal definitions. Internal definitions are now considered preferable in style to let.
Support for begin-for-syntax has been generalized; modules may now define and export both value bindings and syntax bindings (macros) at phase 1 and higher.
Due to a bug, phase 1 syntax (or higher) is not available in DrRacket's #lang-based REPL. A simple workaround is to disable debugging in DrRacket (see “no debugging” radio button in detailed language dialog).

Additional Items:

The racket/gui library (and Slideshow) provides more support for multiple-screen displays.
DrRacket remembers whether an opened file used LF or CRLF line endings, and will continue using the same. When creating a new file, a preference determines how it is saved.
net/url can now follow HTTP redirections.
The LNCS and JFP class files are no longer distributed with Racket. Instead, they are downloaded on demand.
The Algol language implementation is now available as a plain language using #lang algol60.
The Racket-to-C compiler (as accessed via raco ctool or mzc) has been removed; Racket's JIT has long provided better performance, and the FFI provides better access to C libraries.
Contracts can be applied to exports with the new contract-out form within provide, instead of a separate provide/contract form. (The new contract-out form is implemented as a new kind of “provide pre-transformer”.)
The date* structure type is an extension of date with nanosecond and time-zone-name fields.
New looping constructs: for/sum and for/product.
Direct calls to keyword-accepting functions are now optimized to eliminate the overhead of keywords. In addition, the compiler detects and logs warnings for keyword-argument mismatches.
The libfit interface is available from plot/deprecated/fit, and will be removed in the near future.
The Unix installer has been re-done, and it is now more robust.
The built-in reader and printer support for Honu is removed. (This functionality is re-implemented in Racket.)

On-line Check Syntax:

DrRacket now provides an on-line version of the syntax check tool, which means that syntax checking runs automatically while you continue to edit a program. With this tool enabled, its annotations (e.g., binding arrows) and actions (e.g., the renaming refactoring and direct documentation links) are almost always available.

We have noticed that on-line syntax checking renders DrRacket unstable on occasion, perhaps because it relies on relatively new support for parallelism. Occurrences of the problem are rare, but they are not rare enough, which is why we have disabled the tool by default. At the same time, current users of the tool find it so valuable that we felt it should be included in the release. We expect to track down the remaining problems and enable the tool by default in near-future release.

To enable on-line syntax checking (for #lang-based programs only), click on the red dot in the bottom right of DrRacket's window. To turn it off, click there again.

On eval in dynamic languages generally and in Racket specifically

2011-10-18T14:28:00.004-04:00

The eval function is at the heart of a dynamic language, and it strikes many newcomers as an amazingly powerful tool. At the same time, experienced programmers avoid eval, because unnecessary use creates trouble. It’s not easy to explain why eval should be avoided or when it”s appropriate to use eval, but I’ll take another stab at it here.

What is `eval`?

Consider the following “program” in English prose:

Assume that your favorite color is red. Now imagine a balloon that is your favorite color. Paint a canvas the same color as the balloon.

As English goes, that’s a fairly clear program with a fairly well-defined result. When I follow those instructions, at least, I will always produce a red canvas (assuming that I have a canvas and some red paint, but a potential lack of art supplies is not the point here).

I would come up with a red canvas even if I read the instructions when surrounded by people who speak only Chinese, obviously, since I’m the one reading the instructions. Furthermore, it would be straightforward to translate the program to Chinese, and then a person who reads Chinese would produce a red canvas.

A translator might even take the liberty of simplifying the program to just

Paint a canvas red.

The translation loses some of the poetry of the original, but the result is the same.

In Racket terms, the paragraph corresponds to a module. It can be compiled (i.e., translated) and optimized (i.e., simplified). A program can be made up of multiple modules that are written in different languages, but since each module can be reliably translated, they can all be compiled into some common language to run the program.

Here’s a different program:

Tell the person next to you “Assume that your favorite color is red.” Tell the person “Now, imagine a balloon that is your favorite color.” Tell the person “Paint canvas the same color as the balloon.”

Getting a red canvas back may be a little trickier in this case. If the person next to me speaks only Chinese, then my program may fail with a message-not-understood error.

If I want to translate the program to Chinese, then it’s not clear whether the parts in quotes should be translated. Maybe I mean for a person who can read Chinese but only sound out English to run the program when surrounded by English speakers, or maybe I mean for a Chinese person to run the program when surrounded by Chinese people. Either way, I have to be a lot more specific to a translator. For more complex programs, the instructions to the translator can become complex and fragile.

Finally, a translator probably won’t feel comfortable simplifying the program to

Tell the person next to you “Paint a canvas red.”

because there could be all sorts of environmental conditions that make the result different—such as people who are willing to paint but unwilling to accept assumptions about their favorite colors.

The paragraph with “tell the person...” is a program that uses eval. It can’t be compiled and optimized as well as the earlier paragraph, and the language context in which it is run may change the result. The quotes around sentences correspond to the quote in front of an expression passed to eval in Racket; there’s no particular reason that the language for eval will match the language of the program that has the quoted text. The issues become even more complex if you try to implement different parts of the program in different languages.

If the analogy to multiple spoken languages seems strange—maybe your language is Javascript, period—the problem of translation to another language is really a proxy for program understanding. There’s a direct connection to performance and optimization (i.e., translation to efficient machine code), but using eval also makes a program more difficult to understand for the same reasons that it makes the program more difficult to translate. For example, a reader of your program may not be able to tell whether “assume your favorite color is red” is just a rhetorical device to get to a red canvas or whether some new instructions will arrive that will ask for your favorite color.

When is `eval` Good?

The program with “tell the person next to you” above uses eval in a bad way. The task could just as well be performed by the person reading the instructions, instead of getting another nearby person involved.

Some other uses eval are both good and necessary. For example, consider the following program:

Ask the construction manager for instructions. Walk to the building site and convey those instructions to the construction crew.

This program uses eval when it conveys instructions to the construction crew, but no quoted forms appear in the program. The absence of quoted code is one sign that eval may be appropriate. Note that the program could work no matter what language the manager and crew speak, although there is an implicit (and sometimes non-trivial) assumption that the manager and crew speak the same language.

Here’s another example:

Go outside, and tell each member of the construction crew “take a lunch break, now.”

There’s a quoted program in this case, but it’s crucial to ask other people to run the quoted program, instead of just taking the lunch break yourself. That is, eval is really necessary. The implementor of this program takes on the burden of making sure that the instructions are in a suitable language, however, and may need to parameterize the quoted program by an explicit action to translate it to a language understood by the construction crew.

Here’s one more reasonable example:

Ask the construction manager for instructions. Follow them.

In this case, it’s the construction manager’s problem to give you instructions in a language that you understand.

Here’s a questionable example:

Decide how long to work before lunch, say N hours, and write a note to yourself to work N hours. Add to the note by telling yourself to take a lunch break afterward.

If you could really write that program without quotes, then it’s probably ok. The example is misleading, though, because languages don’t usually support

write a note to yourself to work N hours

You’d have to write instead

write a note to yourself that says “work” followed by the number N and then “hours”

and the quote marks are where the problem comes in. If you translate the program to Chinese, then you have to be careful to somehow translate “work” and “hours” to Chinese, too.

The point here is not that programs without quoted text are clearly good or that programs with quoted text are clearly bad. The real point is that a programmer has to be especially careful about passing around instructions and using quoted instructions. Using eval means accepting the burden of using instructions will make sense by the time they are delivered. That burdened is best avoided, which is why experienced programmers avoid eval, but some of the examples illustrate cases where the burden is not avoidable or where the actions enabled by eval make the burden worthwhile.

Using `eval` in Racket

In the context of Racket, the multiple-language analogy is relatively accurate, because Racket is about having many programming languages work together and allowing programmers to define ever better languages and language constructs. In Racket, it’s especially likely that a library written in one language is used in a context where another language is the default.

Newcomers to Racket sometimes stumble over the fact that

 #lang racket
 (define my-x 1)
 (eval '(+ my-x 2))

or even

 #lang racket
 (eval '(+ 1 2))

does not work at all, and yet if the program

 #lang racket
 (define my-x 1)

is loaded into a read-eval-print loop—for example, by clicking the “Run” button in DrRacket and then typing into the lower interactions panel—then

 (eval '(+ my-x 2))

works as expected.

DrRacket’s interactions window has to use eval in the sense that it reads an expression to evaluate and then passes it on to the interpreter for an answer. More generally, to make various pieces of the environment fit together, DrRacket sets eval globally to use the module’s language while evaluating expressions in the interactions window. In Racket terminology, DrRacket sets the current-namespace parameter to the module’s namespace when it initializes the interactions window. In contrast, while the module body is being evaluated, eval treats expressions as being in the language that is empty by default, which is why eval during the module evaluation produces a different result from eval during the interactions windows.

You may wonder why DrRacket doesn’t initialize the namespace of eval to be the module’s namespace from the start, so that in-module uses of eval and the interactions window behave the same. In a program that is implemented by multiple modules, which module’s language should be used? In particular, if the language it’s always the main module’s language, then a module may behave differently on its own than as part of a larger program. In the process of developing Racket and DrRacket, we’ve seen many such problems, and so Racket now arranges for the default language to be empty (which is different from any useful language) to help programmers remember that there’s a language issue to consider whenever eval is used.

The Racket Guide’s chapter 15 covers in more depth the issues and namespace tools of Racket for harnessing the power of eval:

http://docs.racket-lang.org/guide/reflection.html

Think of eval as a power tool. For some tasks, there’s no real substitute, and so we want eval around. At the same time, eval should be used with care. In dynamic languages generally, that means a reluctant and targeted use eval. In Racket specifically, it means knowing the namespace toolbox and being as explicit as possible about the intended context for dynamic evaluation.

Racket v5.1.3

2011-08-16T04:15:00.000-04:00

Racket version 5.1.3 is now available from

  http://racket-lang.org/

This is a bugfix release, resolving the DrRacket issue with the contour view. In addition, two tex files with problematic licensing were removed.

Racket v5.1.2

2011-08-03T15:47:00.004-04:00

Racket version 5.1.2 is now available from

  http://racket-lang.org/

The download page includes 64-bit installers for Mac OS X, Windows, and two Debian flavors. Racket now supports OS X Lion.
Racket now includes a new racket/place library to support parallelism, complementing racket/future. Racket's parallel build process is now based on places instead of multiple OS processes.
Places support share-nothing parallelism and message-passing communication. Compared to futures, places are heavyweight, but they have a simpler performance model.
The syntax-certificate system has been replaced by a syntax-taint system. Both certificates and taints were designed to protect otherwise inaccessible bindings from abuse when they appear in macro expansions. Taints are simpler and lighter, and the switch closes known holes in the certificate system. Macros that are not implemented with syntax-rules or define-syntax-rule, however, must explicitly use syntax-protect to protect their expansions from abuse.
The net/url library supports HTTPS connections, but beware that by default all sites are accepted (equivalent to ignoring a browser's warnings about untrusted certificates).
Error messages in the student languages use a simplified vocabulary and consistent phrasings. If you maintain curriculum material or teachpacks then please consider updating. See the “Error Message Composition Guidelines” section in the documentation for details.
Typed Racket: almost all core Racket data structures and operations are now accessible in Typed Racket (most of this work is due to prolific contributor Eric Dobson). The performance of the typechecker has been significantly improved.
The scriblib/bibtex library supports BibTeX-formatted citation databases in Scribble documents. BibTeX can be tricky to parse, so please report failed entries as bug reports.
The for forms now support an #:unless clause, and a nonnegative integer can be used as a sequence. The new compose1 function creates single-valued composition functions. The racket/function library now provides identity, thunk, and thunk*.
The license has been clarified: we now use LGPLv2.1 uniformly. (The license file used to specify LGPLv2, contrary to the download pages.)

RacketCon Recap

2011-07-29T02:36:00.002-04:00

We held RacketCon here at Northeastern over the past weekend, and it was a great time! Matthew Flatt consulted his magic 8-ball about the future of Racket (it involves Hawaiian shirts); Matthias Felleisen announced the upcoming Realm of Racket book; Robby Findler demoed DrRacket with online syntax checking; Ryan Culpepper taught us how to make Rackety libraries; Prabhakar Ragde explained how the University of Waterloo teaches Racket and Program by Design to 1500 students a year (and how to embed math into Scribble documents).

Danny Yoo has already blogged about his talk about his Whalesong Racket->JavaScript compiler. Hopefully other people will write more about some of the individual talks; I'll update this post to link to any that I see.

As promised, the talks were videotaped by Jeff Dlouhy of the NUACM; the videos are in post-production and should be up within a few weeks. We'll announce their availability on this blog. Slides from the individual talks will be up on the RacketCon webpage soon.

Come to RacketCon!

2011-06-29T17:25:00.003-04:00

Calling All Racketeers!

Join us at RacketCon, 23 & 24 July 2011.

http://con.racket-lang.org/

The schedule for RacketCon is now available, and includes presentations by key members of the Racket team, including Matthew Flatt on the future of Racket and Matthias Felleisen on the Program By Design curriculum, as well as tutorials on building web applications and accessing low-level libraries.

The meeting will take place at Northeastern University in Boston, MA. It will be an opportunity for developers, researchers, and educators using Racket, DrRacket, Program By Design, and related technologies to come together to share plans, ideas, and enthusiasm. To make it accessible, we will not charge any registration fees; we'll also serve you breakfast and lunch.

We're excited about RacketCon, and hope you'll join us there. To register for RacketCon, please send an email with your name and affiliation to racketcon@racket-lang.org.

Additional information, including registration, transportation and hotel information, can be found on the web page: http://con.racket-lang.org

Multi-file code coverage viewing tool

2011-05-26T13:46:00.008-04:00

I'm very pleased to announce the availability of a multi-file code coverage viewer, written by Jonathan Walsh.

Torn between separating your test cases into another file and actually seeing the coverage? Well, go ahead and pull them apart, because the multi-file coverage tool displays coverage information for the files required by the present one, including both percentage covered (on a line-by-line basis) and optionally a list of uncovered lines (no more inching through your code, looking for the red highlighting.

Back End:

One reason I expect this tool to be long-term robust is that it makes absolutely no changes to the back-end; that is, it just uses the existing code coverage framework. The only thing going on here is that the tool provides a way to store, load, and display this information. This means that the tool displays coverage for un-compiled files only. We thought about fiddling with this, but finally decided that the existing behavior was probably about as useful as anything else we'd come up with, and a lot more robust.

URL for docs:

http://planet.racket-lang.org/package-source/jowalsh/code-coverage.plt/1/3/planet-docs/code-coverage/index.html

As you might expect, it's a one-line install:

#lang racket
(require (planet jowalsh/code-coverage))

Please let us know about bugs you discover!

Racket on FLOSS Weekly

2011-05-26T08:21:00.004-04:00

Our own Matthew Flatt was interviewed about Racket on FLOSS Weekly. Also available on YouTube.

Racket v5.1.1

2011-04-30T04:19:00.002-04:00

Racket version 5.1.1 is now available from

  http://racket-lang.org/

The new racket/stream library provides stream-first, stream-rest, a lazy stream-cons, and so on. Streams are a subtype of sequences, so they work in for forms. Some sequence generators, such as in-range, now produce streams. A racket/sequence library replaces the old racket/stream library.
The new racket/syntax library contains facilities useful for writing macros. The new syntax/srcloc and syntax/location libraries provide support for manipulating source locations.
The racket/gui library now supports multi-column list boxes and scrolling panels.
The new ffi/file library is useful for writing foreign library bindings that cooperate with Racket's security guard mechanism.
Generators from the racket/generator library can now have formal arguments that are used when the generator is fired up.
Single-precision floating-point support is now enabled by default. Single-precision floats print differently from their default double-precision counterparts, new primitives convert between the two precisions, and new reader syntax supports single-precision literals.
JIT improvements include a small change to aid x86 branch prediction on function-call returns, which can speed up some programs significantly.
Typed Racket:
- The numeric tower has been entirely overhauled. TR programs can now use more precise types than before, and check more numeric properties, such as sign or range properties.
- Fixnum optimizations have been improved and should apply more broadly.
- The performance of the typechecker has been improved. In particular, dispatch on large union types should typecheck much faster than before.
The Stepper can now step through Lazy Racket programs.
The racket/future library includes fsemaphore values, the future primitive no longer freezes futures (so a future can spawn new futures), and future log messages are more informative.
PLaneT development links are now version-specific.
The 2htdp/image library now includes overlay/align, underlay/align, overlay/align/offset and underlay/align/offset.
The network protocol for universes in 2htdp/universe has changed, so that v5.1.1 is incompatible with earlier versions.
The "DrScheme" application (which simply ran DrRacket in the last few releases) has been removed. The "MrEd" GUI executables for Windows and Mac OS X have also been removed, although the "mred" console executable remains for Unix and Mac OS X to support old scripts.

Writing ‘syntax-case’ Macros

2011-04-04T14:00:00.001-04:00

Disclaimer: This is not really a tutorial on macros, it's more of a quick introduction to using Racket's syntax-case-based macros for people who are familiar with symbolic macros and miss their “simplicity”. It's also not comprehensive or thorough or complete, it's just intended to provide a rough quick overview of how to write macros. It was originally posted on comp.lang.scheme in a thread called “Idiot's guide to Scheme macros”, but I avoided that title here, since it's not a general purpose guide. (Also, it's yet another attempt to dispel the irrational “macrophobia” some people have when it gets to hygienic macros, leading them back to using defmacro with all its problems.)

The main idea with Racket's macro system (and with other syntax-case systems) is that macros are syntax to syntax functions, just like the case of defmacro, except that instead of raw s-expressions you're dealing with syntax objects. This becomes very noticeable when identifiers are handled: instead of dealing with plain symbols, you're dealing with these syntax values (called “identifiers” in this case) that are essentially a symbol and some opaque information that represents the lexical scope for its source. In several syntax-case systems this is the only difference from defmacro macros, but in the Racket case this applies to everything — identifiers, numbers, other immediate constants, and even function applications, etc — they are all the same s-expression values that you're used to, except wrapped with additional information. Another thing that is unique to Racket is the extra information: in addition to the opaque lexical context, there is also source information and arbitrary properties (there are also certificates, but that's ignorable for this text).

With this in mind, explaining the rest is not too difficult:

(syntax-source stx), (syntax-position stx), (syntax-line stx), (syntax-column stx) — retrieve parts of the source location information.
(syntax-e stx) — takes a syntax value and returns the value it “wraps”. For example, if stx is an identifier you'd get a symbol, and if it's a number you'd get the number. If it's a simple parenthesized form, you'd get a list of syntax values for the subforms. Note that the list can be improper, with the last element being a syntax object that contains a proper list. (But the list will actually be improper if the original syntax was a dotted list.)
(syntax->datum stx) — takes a syntax value and returns the plain s-expression that it holds. This is done by recursive uses of syntax-e. (It would be a simple definition that does what you'd think it should do.)
(syntax->list stx) — sometimes you want to pull out the list of syntax values from a given parenthesized syntax, but syntax-e does too little (can still return an improper list) and syntax->datum does too much (gives you back raw s-expressions). syntax->list is a utility function that uses syntax-e as many times as needed to get back a proper list of syntax values. If that's not possible (if the input syntax was not a proper list), it returns #f, so it serves as a predicate too.
(syntax-property stx prop) — returns the given property value from stx, if any, and #f if none. For example, try
```
(syntax-property #'[foo] 'paren-shape)
```
(The #' is similar to a quote, but for syntax values — I'll get to that later on.)
Note that there is no accessor for the opaque lexical scope, and as you'll see next, you don't need one.
To create a piece of syntax you use datum->syntax, and you give it an s-expression which will be the “contents” of the resulting syntax object. (The input can contain syntax values, which are left as is.) But when you do that you need to give it the other bits — including the lexical context thing, which you have no access to. The way that's done is:
```
(datum->syntax context-stx input-sexpr)
```
This returns a syntax value that wraps the input-sexpr value, using the lexical scope from context-stx. A common way to “break hygiene” and create a binding that is visible to the macro user's code is:
```
(datum->syntax stx 'foo)
```
where stx is some syntax value that you get from the user input to the macro. It returns a foo identifier that has the same lexical context information as stx, so it's as if it came from there.
Note that there is actually another optional argument that specifies the source (either using another syntax object, or as an explicit list), and another for copying the properties from — so an alternative to the above would be:
```
(datum->syntax stx 'foo stx stx)
```
which also makes the source information and the properties be the same as those of stx (for example, this can matter in case of syntax errors ).
There is also (quote-syntax blah) which creates a quoted syntax, with its lexical source from the place it appears.
Finally, define-syntax does the magic of tying a name with a transformer function.

And that's almost everything that you need in order to write hygienic (and non-hygienic) macros. Very inconveniently.

For example, here's a simple while macro (use this in a file that starts with “#lang racket”):

(define-syntax (while stx)
  (define subs (syntax->list stx))
  (datum->syntax
   stx
   `(let loop ()
      (when ,(cadr subs)
        ,@(cddr subs)
        (loop)))
   stx))

which breaks like this:

(define x 2)
(let ([let 5])
  (while (< x 10)
    (printf "x = ~s\n" x)
    (set! x (add1 x))))

The problem is that all of those quoted names are getting the context of the user input, which is not the right thing (it's close to a defmacro). To fix this, you need to quote-syntax all of these identifiers, so they'll have the macro source instead of the input source:

(define-syntax (while stx)
  (define subs (syntax->list stx))
  (datum->syntax
   stx
   `(,(quote-syntax let) ,(quote-syntax loop) ()
     (,(quote-syntax when) ,(cadr subs)
      ,@(cddr subs)
      (,(quote-syntax loop))))
   stx))

But that's clearly insane... More than being tedious, it's still incorrect since all of those function application parens will have the user's lexical context (Racket has a special implicit #%app macro that gets used in all function applications, and in this case the context of this application will make it unhygienic). Instead of doing this, a better approach would be to create the resulting syntax with the lexical context of the macro source by changing just that argument:

(define-syntax (while stx)
  (define subs (syntax->list stx))
  (datum->syntax
   (quote-syntax here)
   `(let loop ()
      (when ,(cadr subs)
        ,@(cddr subs)
        (loop)))
   stx))

And that's simple again, and works fine now.

The problem is that it's tedious wrt to deconstructing the input (which happens to be trivial in this case), and wrt slapping together an output value — and that's where syntax-case comes in. It addresses the both by using pattern matching, where identifiers in patterns are bound as “syntax patterns”. A new form is added — syntax — which is similar to a quote, except that (a) it actually quotes things similarly to quote-syntax, with the lexical context of the syntax form; and (b) pattern variables are substituted with what they matched. With this, the above macro becomes much easier:

(define-syntax (while stx)
  (syntax-case stx ()
    [(_ test body ...)
     (syntax (let loop ()
               (when test
                 body ...
                 (loop))))]))

The first line specifies that you want to match the stx input syntax, and that you have no “keywords” (in the same sense as in syntax-rules). The second line is the pattern that is matched against this input — with two pattern variables that match the second subexpression and the sequence of expressions from the third and on. (The first subexpression is matched against _ which is a wild-card that matches anything without binding a pattern variable — the head part is often not needed, since it's just the macro name.) The last line is the output, specified using syntax, which means that it's very similar to the previous version where everything is given the lexical context of the macro and the two pattern variables are replaced with the two matches (so body gets spliced into the resulting syntax).

Now, say that you want an unhygienic user-visible piece of syntax. For example, bind the always entertaining it thing to the test result. This:

(define-syntax (while stx)
  (syntax-case stx ()
    [(_ test body ...)
     (syntax (let loop ()
               (let ([it test])
                 (when it
                   body ...
                   (loop)))))]))

won't work because it has the macro source — it's hygienic and therefore not visible. Instead, you need to use datum->syntax with the user syntax:

(define-syntax (while stx)
  (syntax-case stx ()
    [(_ test body ...)
     (let ([it (datum->syntax stx 'it)])
       (syntax (let loop ()
                 (let ([it test])
                   (when it
                     body ...
                     (loop))))))]))

But this doesn't really work since it needs to be bound as a pattern variable rather than a plain binding. syntax-case can be used here again: (syntax-case <name> () [foo <body>]) will match foo against the <name> syntax, and if it's a name then it will be bound as a pattern variable in the <body>.

(define-syntax (while stx)
  (syntax-case stx ()
    [(_ test body ...)
     (syntax-case (datum->syntax stx 'it) ()
       [it (syntax (let loop ()
                     (let ([it test])
                       (when it
                         body ...
                         (loop)))))])]))

Note that since it is a pattern variable, it doesn't need to be unquoted — syntax will do that.

Finally, there are some more conveniences. First, with-syntax is a macro that binds pattern variables (by a similar translation to syntax-case):

(define-syntax (while stx)
  (syntax-case stx ()
    [(_ test body ...)
     (with-syntax ([it (datum->syntax stx 'it)])
       (syntax (let loop ()
                 (let ([it test])
                   (when it
                     body ...
                     (loop))))))]))

and there's the #' reader macro for syntax:

(define-syntax (while stx)
  (syntax-case stx ()
    [(_ test body ...)
     (with-syntax ([it (datum->syntax stx 'it)])
       #'(let loop ()
           (let ([it test])
             (when it
               body ...
               (loop)))))]))

and there are also #` and #, and #,@ which are implemented by translating them to uses of with-syntax.

Note that the last example uses the lexical context of the whole form for the new identifier, but that's not only the option. You could use any other part of the macro input — for example, you could use the macro keyword:

(define-syntax (while stx)
  (syntax-case stx ()
    [(hd test body ...) ; need the head now
     (with-syntax ([it (datum->syntax #'hd 'it)])
       ... same ...)]))

or the test expression (use #'test). Each of these choices has subtle differences that are especially important when you're composing macros (for example, using a second macro that expands to a while, where the test expression comes from that macro rather than the user code). Demonstrating these things is a popular way to pass the time in some circles, but I'll avoid it here. In fact, a great way to avoid this whole thing altogether is not create unhygienic bindings in the first place. It sounds like doing so excludes cases where you really want to have a new binding visible in user code, but racket provides “syntax parameters” that can be used more conveniently (and less confusingly) — see an earlier post for a description of that. As a side note, these options are a good hint that a hygienic macro system is more expressive than a symbolic defmacro system, where no such choices exist. Creating such macros using defmacro can appear easier simply because of this lack of choice — in the same way that CPP-style string-based macros are “simpler” than defmacro since they're less expressive (just appending lexical tokens, no structural information).

There are other important aspects of the Racket macro system that are not covered here. The most obvious of them is worth mentioning here: Racket separates the “runtime phase” from the “syntax phase”. For example, if you want to try these examples with “#lang racket/base”, you'll need to add (require (for-syntax racket/base)) since the racket/base language doesn't have a full language in its syntax phase.

Roughly speaking, this makes sure that source code is deterministically compilable by having each level live in its own world, limiting macros to deal only with the input syntax only and not runtime values. (For example, a CLOS implementation in this system cannot check the value of an identifier bound to a class to determine how some macro should expand.) This results in reliable compilations that do not depend on how things were loaded, or whatever happened on the REPL.

The important bottom line here is that you get to write macros with the full language available — and phase separation means that Racket is explicitly designed to make running code at the macro level and using it by the compiler as robust as possible, so you don't have to worry about using any complex system as part of your macro. You just need to keep in mind that the macro world is completely separate from the runtime, and the direct benefit of not worrying about weird interactions with compilation and file loading orders.

Languages as Libraries, PLDI 2011

2011-03-19T10:58:00.001-04:00

We've just finished up the final version of our PLDI 2011 paper on language extension in Racket. The paper describes how the module system and the syntax system work together to support new languages with new static semantics, such as Typed Racket. Here's the abstract:

Programming language design benefits from constructs for extending the syntax and semantics of a host language. While C's string-based macros empower programmers to introduce notational shorthands, the parser-level macros of Lisp encourage experimentation with domain-specific languages. The Scheme programming language improves on Lisp with macros that respect lexical scope.
The design of Racket—a descendant of Scheme—goes even further with the introduction of a full-fledged interface to the static semantics of the language. A Racket extension programmer can thus add constructs that are indistinguishable from “native” notation, large and complex embedded domain-specific languages, and even optimizing transformations for the compiler backend. This power to experiment with language design has been used to create a series of sub-languages for programming with first-class classes and modules, numerous languages for implementing the Racket system, and the creation of a complete and fully integrated typed sister language to Racket's untyped base language.
This paper explains Racket's language extension API via an implementation of a small typed sister language. The new language provides a rich type system that accommodates the idioms of untyped Racket. Furthermore, modules in this typed language can safely exchange values with untyped modules. Last but not least, the implementation includes a type-based optimizer that achieves promising speedups. Although these extensions are complex, their Racket implementation is just a library, like any other library, requiring no changes to the Racket implementation.

To learn how to implement your own new language in Racket, start with this documentation.

Racket v5.1

2011-02-14T22:52:00.000-05:00

Racket version 5.1 is now available from

  http://racket-lang.org/

The most significant change in version 5.1 is a rewrite of the GUI library. Unix/X users will see the biggest difference with this change, because DrRacket and all Racket GUI programs now take on the desktop theme for menus, buttons, and other GUI widgets.

In the long run, Racket GUI programs on all platforms will improve as a result of the library rewrite. In the short run, beware that this first release of a new library will inevitably include a new set of bugs.

Version 5.1 changes in more detail:

The racket/draw library—which implements the drawing half the GUI toolkit—can be used independent of the racket/gui/base library and without a graphics display (e.g., without an X11 connection).
The new library has one small incompatibility with the old GUI toolbox: 'xor drawing is no longer supported. The new library has many additional features: rotation and general affine transformations, PDF and SVG drawing contexts, gradients, and alpha-channel bitmaps.
The GRacket executable is no longer strictly necessary for running GUI programs, because the racket/gui/base library can be used from Racket. To the degree that a platform distinguishes GUI and console applications, however, the GRacket executable still offers some additional GUI-specific functionality (e.g., single-instance support).
The new racket/gui/base library includes small incompatibilities with the old GUI toolbox: the send-event, current-ps-afm-file-paths, and current-ps-cmap-file-paths functions have been removed. The racket/gui/base library re-exports racket/draw, so it includes the same drawing functionality as before (except for 'xor drawing).
The new racket/snip library can be used independently of racket/gui/base to work with graphical editor content (e.g., images in student programs). Like racket/draw, the racket/snip library is re-exported by racket/gui/base.
The Web Server includes a backwards incompatible change that prevents X-expressions and lists of bytes from being directly returned from servlets. This change will increase performance for those types of responses and allow easier experimentation with response types. Please see "collects/web-server/compat/0/README" in the installation to learn about porting your servlets forward. Don't worry. It's easy.
The new raco demodularize tool collapses a module's dependencies into a single module comprising the whole program. This transformation currently provides no performance improvement, but is the basis for cross-module optimization and dead-code elimination tools to come. The transformation is currently useful for static analysis of whole Racket programs.
The picturing-programs teachpack, formerly installed via PLaneT, is now bundled with the standard distribution. Use the teachpack with (require picturing-programs) instead of (require installed-teachpacks/picturing-programs). The old PLaneT-based installation procedure still works, but it now merely installs a stub that invokes the bundled version.
Slideshow picts, racket/draw bitmaps, and images created with 2htdp/image can now be used directly in Scribble documents. More generally, the new file/convertible protocol enables any value that is convertible to a PNG and/or PDF stream to be used as an image in a Scribble document.
The Same game sports a new look and an improved scoring system. (The current known high score is 12,429; can you beat that?)

Rebuilding Racket’s Graphics Layer

2010-12-08T10:59:00.005-05:00

Racket version 5.1, which is scheduled for release in early February, will look a little different on the outside. Unix/X users will see the biggest difference: DrRacket and all Racket GUI programs will take on the desktop theme for menus, buttons, and other GUI widgets. Text handling is also better than before on Unix/X, especially when printing. Windows and Mac OS X users will see smaller changes, such as better printing, better handling of mouse-wheel events, and support for 64-bit Windows and Mac OS X.

On the inside, version 5.1 is the biggest single change in Racket (or PLT Scheme) history. We’ve reimplemented the GUI layer, which meant throwing out about 200,000 lines of C++ code that built on Xt, Win32, and Carbon. We’ve replaced that C++ code with about 30,000 lines of Racket code that builds on Gtk, Win32, Cocoa, Cairo, and Pango. This change modernizes Racket's graphics support while significantly reducing the cost of maintaining the GUI and drawing libraries.

In the space between the GUI implementation and the surface, there are many API improvements:

You can run GUI programs with just racket, instead of having to use gracket. Depending on how much your platform distinguishes between GUI and console applications, there may still be an advantage to using gracket (i.e., to tell the OS that you mean to start a GUI application or that you want a single instance of the application), but the difference is minor.
Most of the drawing library has moved to racket/draw, which you can use without the rest of the GUI library – and, in the case of Unix platforms, without an X-server connection. After detangling the graphics and GUIs libraries, the graphics library is now integrated in more places, such as adding pict support for Scribble documents.
The drawing library includes some new capabilities, such as rotation, affine transformations, and bitmaps with alpha channels.

Replacing hundreds of thousands of lines of C++ code with tens of thousands of lines of Racket code sounds like a no-brainer. The old library was implemented in C++ because we started in 1995 by gluing together a Scheme interpreter with a portable GUI library. Then the GUI code stayed in C++, because the interpreter wasn’t fast enough and the foreign interface was too clumsy. Racket is now plenty fast and its foreign interface has improved a lot since then.

Still, the reimplementation took about 18 months. Smoothly integrating cross-platform GUI support with a programming language can be more difficult than it sounds, and mating new libraries with a legacy API creates additional challenges. Finally, many Racket tools depend Racket’s “eventspaces,” which are multiple process-like entities in the same virtual machine, each with its own GUI event loop. Implementing eventspaces on top of modern GUI toolkits turns out to be tricky, because the toolkits insist on a single event-loop per process and they cannot tolerate event-loop actions during certain callbacks. Fortunately, delimited continuations can help work around those limitations.

Cairo and Pango are the two big enablers of the Racket graphics rewrite. The old Racket graphics library depended on many toolkits (X11, Win32, QuickDraw, Quartz, PostScript, and more), and it had poor font handling. Again, the problem was that we chose the previous technology in 1995. Cairo and Pango have since solved the portable-graphics problem, and we were able to trade in 80,000 lines of C++ glue for about 8,000 lines of Racket glue. The code could be much less if we didn’t have to match most of the old drawing API, but we're still very happy with the result.

On the GUI side, the remaining 22,000 lines of Racket code replace similar C++ code that binds to three different toolkits. The set of underlying toolkits has changed, and a few eventspace tricks are new, but the approach is essentially the same as before. The code is nevertheless much more compact, because (no surprise) Racket is better than C++. Interestingly, the amount of toolkit-specific code is right around 6,500 lines for each toolkit, even though the way that a C programmer uses the different toolkits seems very different: Objective-C classes (Cocoa) versus signal callbacks with explicit wiring (Gtk) versus a single callback function for message handling (Win32). Maybe they're the same because we built a Racket mini-language for each toolkit that makes them all about equally convenient.

The rewrite is not perfectly compatible with old code, and no doubt we have many bugs to find before the release. The process is well on track, though, and the new library implementations give a us a solid foundation to keep making Racket better.

To try out the current development version, visit

http://pre.racket-lang.org/installers

Racket v5.0.2

2010-11-07T01:33:00.000-04:00

Racket version 5.0.2 is now available from

  http://racket-lang.org/

Typed Racket's optimizer is now turned on by default; error messages have been simplified and clarified.
Contracts: contracts on mutable containers allow functions or other higher order values, and contracts are checked when updating or dereferencing the containers. The new contracts are slower than the old ones, so the old check-once functionality is still available.
A new dependent function contract combinator, ->i, properly assigns blame for contracts that violate themselves and the generated wrappers are more efficient than ->d. (Although it does more checking so your contracts may be faster or slower).
See the docs for box/c, hash/c, vector/c, vectorof and ->i for more details.
The when, unless, cond, case, and match forms (in racket/base and derived languages) now allow immediate internal definitions.
Web server: the formlets library provides more HTML form elements; make-xexpr-response supports a preamble for DTD declarations; serve/servlet supports stateless servlets.
New WebSocket implementation, see net/websocket for details.
The new data collection contains implementations of several data structures, including growable vectors and order-based dictionaries.
racket/match is now significantly faster.
The Racket documentations are built in parallel by default.
The stepper is now compatible with programs using the Universe teachpack.
2htdp/image: pinholes are now supported in the library (if you don't use pinhole primitives you will not see them); a number of new triangle functions added; supports conversion of images to color lists and back. Also, cropping has been improved for scenes; see the documentation section on the nitty-gritty of pixels for details.
Signatures have been moved to ASL, BSL, BSL+, ISL, and ISL+ (HtDP teaching languages) no longer support checked signatures.
Student languages: one-armed check-error in all levels; ASL is extended with hash operations, and define-datatype.
DMdA languages: Checking for parametric signatures is now eager. This catches errors earlier, but retains the asymptotic complexity properties; signatures for record types now have generators; list-of and any signatures are now provided.

The Two-State Solution: Native and Serializable Continuations Accord

2010-10-03T00:57:00.000-04:00

The Racket Web Server allows an expressive way of writing Web applications using first-class continuations to capture the control-flow of the server while it is waiting for the client to respond. For example:

#lang web-server/insta
(define (get-number p)
  (string->number
   (extract-binding/single
    'num
    (request-bindings
     (send/suspend
      (λ (k-url)
        `(html 
          (body
           (form ([action ,k-url])
                 ,p nbsp (input ([name "num"]))
                 (input ([type "submit"])))))))))))
(define (start req)
  (define how-many
    (get-number "How many numbers to add?"))
  (number->string
   (foldr 
    + 0
    (build-list 
     how-many
     (λ (i)
       (get-number 
        (format "Provide number: ~a" 
                (add1 i))))))))

This application creates a re-usable get-number interaction abstraction and uses it in a number of different contexts. In particular, it uses it in the higher-order context of build-list. This application also reuses useful third-party library functions like foldr, etc.

Such an application would be complicated to write in a traditional Web programming environment because the continuation of each get-number invocation is considerably more complex than is typical. Yet, the first-class continuations in Racket ensure that this continuation is captured exactly, correctly, every time.

Unfortunately, the native first-class continuations of Racket are not serializable, so they impose a per-session resource expenditure on the server. This can be alleviated through expiration policies, but such policies are inherently unsound because continuations URLs are global roots.

In the past, PLT has provided tools that automatically restructure this kind of program into one that uses serializable continuations through an acronym soup of source transformations: CPS, lambda-lifting, defunctionalization, SPS, and so on. These tools effectively create automatically what most Web programmers write manually, except the tools don't mistakes. But the tools also don't take into consideration what functions actually contribute to the interaction context and transform library functions like foldr (which is unnecessary in the continuation) the same as functions like build-list (which are necessary.)

Our past work (based on another PLT paper) alleviates this problem by only requiring functions like build-list to be transformed. From the perspective of a programmer, "transformed" is tantamount to "rewritten" because the source code for a third-party library may not be readily available. Programmers would have to program add-many-numbers.com as:

#lang web-server
(require web-server/servlet-env)
(define (get-number p)
  (string->number
   (extract-binding/single
    'num
    (request-bindings
     (send/suspend
      (λ (k-url)
        `(html 
          (body
           (form ([action ,k-url])
                 ,p nbsp (input ([name "num"]))
                 (input ([type "submit"])))))))))))
(define (build-list n f)
  (for/list ([i (in-range n)])
    (f i)))
(define (start req)
  (define how-many
    (get-number "How many numbers to add?"))
  (number->string
   (foldr 
    + 0
    (build-list
     how-many
     (λ (i)
       (get-number 
        (format "Provide number: ~a"
                (add1 i))))))))
; This requires a pre-release version
; to run in an un-named DrRacket buffer
(serve/servlet start #:stateless? #t)

where build-list has been re-implemented, but functions like foldr have not. This application, despite its striking similarity to the first, requires absolutely no per-session server state, so it is considerably more scalable.

Do we need to re-implement build-list? What if the third-party, higher-order function (build-list) that we use with a higher-order argument that causes Web interaction (get-number) is too complicated to re-implement?

Naturally this blog post would not exist if we didn't solve this problem.

Our new approach, dubbed The Two-State Solution, allows the programmer to transparently use a very small amount of per-session server state to store just the part of the continuation inside functions like build-list while serializing everything else to the client.

The key is to use delimited, composable continuations to isolate the appropriate part of the continuation. The programmer designates this piece of the continuation through the serial->native and native->serial annotations. The programmer can write the application as:

#lang web-server
(require web-server/servlet-env)
(define (get-number p)
  (string->number
   (extract-binding/single
    'num
    (request-bindings
     (send/suspend
      (λ (k-url)
        `(html 
          (body
           (form ([action ,k-url])
                 ,p nbsp (input ([name "num"]))
                 (input ([type "submit"])))))))))))
(define (start req)
  (define how-many
    (get-number "How many numbers to add?"))
  (number->string
   (foldr 
    + 0
    (serial->native
     (build-list
      how-many
      (λ (i)
        (native->serial
         (get-number 
          (format "Provide number: ~a"
                  (add1 i))))))))))
; This requires a pre-release version
; to run in an un-named DrRacket buffer
(serve/servlet start #:stateless? #t)

The important distinction here is that both the build-list and the get-number abstractions do not need to change. We simply mark the context as being a "serial" or "native" context through the annotation forms. This re-written version will be more scalable than a purely native version, but represents an easier to achieve step in the evolution of a program, because third-party, higher-order functions can be used as is.

This work will be presented at OOPSLA 2010. It is also described in a paper with same name this blog post: The Two-State Solution: Native and Serializable Continuations Accord.

Extending Typed Racket, Part 1

2010-09-15T15:34:00.001-04:00

The Typed Racket team is pleased to announce a number of new additions to our system. We'll be writing a few blog posts about them, all of which you can read here.
This post begins with the core of the Typed Racket type system. The fundamental idea at the heart of Typed Racket is called occurrence typing. This is the technique that allows us to typecheck existing Racket programs without requiring rewrites. Here's a simple example:

(if (number? x) (add1 x) 0)

The typechecker can figure out from the use of number? that the second occurrence of x is always going to be a number. This simple form of occurrence typing is enough to take Typed Racket a long way. But because we want to be able to handle all the sophisticated reasoning that programmers are already using to write their Racket programs, we have been working on extending the system further.
The new design of our system is described in a paper, Logical Types for Untyped Languages, in the upcoming International Conference on Functional Programming. The introduction provides an overview that's acessible to any Racket programmer, but here's the key example:

(cond
  [(and (number? x) (string? y)) — 1 —]
  [(number? x)                   — 2 —]
  [else                          — 3 —])

In expression 1, we know that x is a number and y is a string. In 2, we know that x is a number and y is not a string, by the logical properties of and and cond. This form of logical reasoning is enabled by the new foundation of the system, and makes the entire system significantly more expressive.
All of these improvements are available in the current version of Racket.

Racket v5.0.1

2010-08-03T22:36:00.000-04:00

Racket version 5.0.1 is now available from

  http://racket-lang.org/

Datalog is a lightweight deductive database system with Racket integration. It is now available in the datalog collection and with #lang datalog.
Racklog provides Prolog-style logic programming in Racket, adapted from Dorai Sitaram's Schelog package. It is available in the racklog collection and now as #lang racklog.
By default make install and raco setup compile collections in parallel on all available processors. (Use raco setup -j 1 to disable, if necessary.)
Changes (as part of 5.0) in the racket language compared to the scheme language: constructor-style printing, a struct alternative to define-struct that fits more naturally with match and constructor-style printing, bytecode-dependency management via SHA-1 hashes instead of just timestamps (where the openssl/sha1 library provides the SHA-1 hash function), a reorganization of scheme/foreign into ffi/unsafe and associated libraries, and new printing functions eprintf and displayln. Also, a generator from racket/generator is required to have the form (generator () body ...), which supports a planned extension to let a generator accept arguments.
Changes to the racket language (since 5.0): internal-definition positions allow mixing expressions with definitions, full continuations can escape past a continuation barrier, custodians can attempt to terminate subprocesses and subprocess groups (see current-subprocess-custodian-mode, subprocess-group-enabled), the JIT supports additional unboxing flonum operations and unsafe variants, ffi/unsafe provides an asychronous-call mechanism to deal with foreign threads, a new "." modifier for format string directives (e.g., "~.s" and "~.a") limits the respective output to (error-print-width) characters.
The core type system of Typed Racket has been substantially revised. In particular, Typed Racket can now follow significantly more sophisticated reasoning about the relationships between predicates. Additionally, Typed Racket now allows variable arity types in more places, allowing programmers to specify variable-arity lists.
We are working on an optimizing version of Typed Racket that takes advantage of type information for certain classes of programs. This project is a work in progress. For those interested, see the documentation for #:optimized.
The web-server/formlets library adds a formlet* form that allows dynamic formlet construction, as opposed to formlet which requires syntactic Xexprs and static formlets. Several new library formlets are added.
The syntax/parse library has new support for matching literals at different phases using the #:phase argument for literals and literal sets.
RackUnit now includes a GUI test runner as rackunit/gui.
The 2htdp/image library now includes flip-vertical and flip-horizontal operations that mirror images (vertically and horizontally).

Racket

2010-06-07T14:50:00.004-04:00

PLT is happy to announce the release of Racket, available from

  http://racket-lang.org/

With Racket, you can script command shells and web servers; you can quickly prototype animations and complex GUIs; regexps and threads are here to serve you. To organize your systems, you can mix and match classes, modules or components. Best of all, you start without writing down types. If you later wish to turn your script into a program, equip your Racket modules with explicit type declarations as you wish. And Racket doesn't just come as a typed variant; you can also write your modules in a purely functional and lazy dialect.

Racket comes in so many flavors because Racket is much more than a standard scripting language or a plain programming language. Racket supports language extensibility to an unequaled degree. A Racket programmer knows that making up a new language is as easy as writing a new library.

To help you start quickly, Racket includes batteries in all shapes and sizes, most importantly, extensive documentation and all kinds of libraries.

Racket occupies a unique position between research and practice. It inherits many major ideas from language research, among them type safety (when the type system says that x is a number, then at runtime it always is a number) and memory safety (when some memory is reclaimed by the garbage collector it is impossible to still have a reference to it). At the same time, user demand governs rigid adherence to purely theoretical principles.

Racket, formerly PLT Scheme, is a product of over 15 years of development. Although Racket starts with a mature software base and an established user community, its new name reflects our view that this is just the beginning of Racket's evolution.

PLT Scheme v4.2.5

2010-04-02T21:51:00.001-04:00

PLT Scheme version 4.2.5 is now available from

  http://plt-scheme.org/

PLT now supports multi-core parallelism via futures. Futures create tasks that run in parallel, as long as the tasks stay in the "fast path" of the runtime system. For more information, see the guide.
Our unit testing framework, schemeunit, is now included in the distribution. A graphical test runner is available via schemeunit/gui.
The support languages for the "Programming Languages: Application and Interpretation" textbook by Shriram Krishnamurthi are now part of PLT Scheme. In addition the PLAI GC language comes with a random mutator generator (to help test collectors) and an improved heap visualizer.
New Russian and Ukranian translations, thanks to Sergey Semerikov.
A number of improvements to Redex's typesetting facilities.
Typed Scheme users can now automatically generate predicates from types with define-predicate. Typed code can be inserted in untyped modules by requiring with-type from typed/scheme.
The scheme/class library now provides contract combinators for classes (class/c) and objects (object/c). See the Reference and Guide for details. Also, a backwards-compatible object-contract version of object/c has replaced the old object-contract combinator.
Writing new kinds of contracts is now easier with keyword-based constructors (make-contract and make-flat-contract), a simpler set of structure properties (prop:contract and prop:flat-contract), and the introduction of blame objects for tracking contract metadata.
The Scheme-implemented bytecode reader fails less often. This is used by "mzc --decompile". The Scheme-implemented bytecode writer uses the compact bytecode format and fails less often. This may be used in the future for Scheme-implement bytecode processors.
The language dialog now suggests using "#lang" more strongly as the default language. DrScheme no longer uses the term `Module language'.

Talk at Flourish

2010-03-08T11:35:00.006-05:00

The image in this post shows a tree where the interior nodes represent directories and the leaf nodes represent files in the PLT source code. The leaves are colored based on the programming language used. (To avoid clutter, if there is more than one file in a given directory written in a particular language, that language only gets a single dot.)

Some highlights: the blues are Scheme-like languages, the reds are langauges we use to write documentation (see Scribble for more about them), the greens are teaching languages, orange is the language we use to bootstrap new languages, and yellow is a language for metadata about nearby files.

Curious about how we managed to write and use so many different languages? I'll be giving a talk at Flourish 2010 next week (3/19 @11am, UIC in Chicago) explaining how. Come to learn more!

DAGs vs Trees

2010-02-28T20:16:00.006-05:00

As I wondering whether or not there is a better layout algorithm for the module browser window, I looked into tree maps. Of course, the modules in a program form a DAG, not a tree, so I wondered just how big the tree would get if all of the shared structure in the DAG were replicated. Hey, I figured, if a tree map can handle showing me my entire filesystem, maybe that could work.

... yeah, no. Turns out to be hopeless. In the spirit of a geeky take off on a jelly bean counting contest, lets see if you can guess just how big these things get. Consider the module graph from the program #lang scheme (ie, the graph that just contains an empty program). This program loads 170 modules with 917 connections between modules (counting the main file that just contains the #lang scheme).

So, the question: how many nodes are there in the unsharified tree? First one to come within 1 billion of the right answer gets all of the fame and glory that this blog brings to bear (har har). I'll post the answer in the comments in a few days (and no fair cheating, those of you that know enough to be able to get your hands on the DAG).

The Racket Blog

Scribble Your Blogs

Fixed Racket v5.2.1 Installers

Racket v5.2.1

Release Highlights:

Zack Galler's Experience with Stateful vs Stateless Web Apps

Racket v5.2

Release Highlights:

Additional Items:

On-line Check Syntax:

On eval in dynamic languages generally and in Racket specifically

What is eval?

When is eval Good?

Using eval in Racket

Racket v5.1.3

Racket v5.1.2

RacketCon Recap

Come to RacketCon!

Multi-file code coverage viewing tool

Racket on FLOSS Weekly

Racket v5.1.1

Writing ‘syntax-case’ Macros

Languages as Libraries, PLDI 2011

Racket v5.1

Rebuilding Racket’s Graphics Layer

Racket v5.0.2

The Two-State Solution: Native and Serializable Continuations Accord

Extending Typed Racket, Part 1

Racket v5.0.1

Racket

PLT Scheme v4.2.5

Talk at Flourish

DAGs vs Trees

What is `eval`?

When is `eval` Good?

Using `eval` in Racket