27 Jul 2007

call/cc and self-modifying code

posted by matthias

Today I wrote this short illustration of call/cc and posted it on wikipedia:

;; [LISTOF X] -> ( -> X u 'you-fell-off-the-end-off-the-list)
(define (generate-one-element-at-a-time a-list)
  ;; (-> X u 'you-fell-off-the-end-off-the-list)
  ;; this is the actual generator, producing one item from a-list at a time
  (define (generator)
     (call/cc control-state)) 
  ;; hand the next item from a-list to "return" (or an end-of-list marker)'
  (define (control-state return)
        (lambda (an-element-from-a-list)
           (set! return ;; fixed
               (lambda (resume-here)
                 (set! control-state resume-here)
                 (return an-element-from-a-list)))))
     (return 'you-fell-off-the-end-off-the-list))
  ;; time to return the generator

It reminded of all the talk in the 1980s and 1990s that self-modifying code is bad. But look at the elegant assignment to control-state within its body. It’s such a poem, I thought I’d share it with people since nobody else blogs here anywya.

more →

14 Jun 2007

Small is Beautiful, Large is Useful, and Scheme is Both

posted by matthias

They say, Scheme is small and this is good.

Have you heard of X? No? It is the smallest computational basis. It is a single function that can compute everything a Turing machine can compute; a Church lambda calculus; a Post model; a RAM; a what-have-you model of computation. Indeed, X is so simple that two equations suffice to specify it completely [Barendregt, page 166]. Imagine that: a complete language report in two lines; a compiler that fits in a few K instead of Ms; no more arguments about smallness.

Small alone can’t be any good. If you used X alone, your programs would be the size of the universe or something like that. That’s what the theory of computability teaches us [Church and Turing]. Adding LAMBDA and a few primitives to get a pure functional language isn’t good enough either. That’s what the theory of expressiveness shows [Felleisen; Mitchell; Riecke]. And, using an R5RS Scheme to build large systems with many people at a dozen sites isn’t doable either. That’s what the PLT experience determined.

When we set out to construct DrScheme using MzScheme, we also conducted an experiment:

Could we really build a graphical system that manages (shared) resources and that provides excellent error feedback with just plain Scheme?

Could we just add enough libraries to do all this? Or would we have to change the kernel of the language? As much as we tried to keep MzScheme small, it became clear quickly that we needed exceptions, structures, module-like features, a mechanism for concurrency, a way to manage resources such as windows, tcp connections, and so on. The list isn’t infinite but it is much longer than I expected. Our “Revenge of the Son of the LISP machine” paper is a good summary for the state of the art around 1999 [Flatt and Son].

As language designers we stepped back time and again to look at our monster. What could we remove? What would we have to add in response? For some five years, we had first-class modules (units) and first-class classes in the core of the language. We had almost banned macros. They were so ugly I stopped teaching about them because I did want to use our own dog food in my courses but I couldn’t stomach the macro system. It was such a step back from Eugene’s extend-syntax. But then Matthew figured out the next big step in macro and module technology [Flatt, You Want It When?]. And with that out went units and classes from the core and many other things. So we learned lessons, and we need to keep building systems to learn more.

I have no question that the idea of Scheme is beautiful. At the same time, I have also learned that if I wish to use this beautiful idea in practice, I need to add the ingredients that it takes to build large systems. R6RS reflects this insight, and I am happy about it.

more →

09 Jun 2007

R6RS is “perfect”

posted by matthias

When I read the “side by side” and “head to head” descriptions of the alternatives facing the Scheme community (see Comp.Lang.Scheme and the R6RS mailing list), I am wondering which one is which and which one is better.

  • Is it really good that Scheme (the spec) doesn’t support a module system?

  • Is it really good that almost all major implementations support their own version of a module system?

  • Is it really good that programmers can’t even leave the module structure intact when porting code?

Imagine your own similar questions and add them here. We have lived in a side-by-side universe for a long time, and there are quite a few programmers who have suffered from this not-really-the-same-language problem. Besides the module system, there are other not-quite-the-same-but-related features that implementations have and programmers wish to use. The R6RS process has pushed several major implementors/implementations to agree on a design for module systems and other constructs. Their report declares that they are ready to put a large amount of work in to get from r5rs to r6rs. I believe that this step would help the community in several arenas, listed in increasing order of relevance:

  • the academic publishing business

  • the fund raising business

  • adapting each others innovations

  • supporting programmers who learn on one and switch to another implementation

  • supporting commercial programmers who need reassurance that there is more than one implementation and implementor [ever attended Commercial Uses of Functional Programming?]

Is the document perfect? Is every construct exactly the ‘right thing’? Of course not! Guy and Gerry revised their first Scheme report because they didn’t get it ‘right’. R3RS and R4RS and R5RS revised flaws in R(n–1)RS because the authors/editors didn’t get it ‘right’. It is extremely difficult, and usually impossible, to get the design of a complex artifacts (such as a programming language) ‘right’ the first time. In these cases, it’s all about the feedback loop and revising your design based on observations. (Remember the ‘science’ part in the name of our discipline?) Indeed, ‘right’ doesn’t exist; what exists is ‘most pragmatic and internally beautiful,’ and nothing else. Our choice is quite simple: move forward as a community with some amount of convergence (r6rs) or split into dozens of mutually incompatible sub-communities (status quo, including SRFIs). Also posted as “R6RS is perfect” at the R6RS discussion list.

more →

22 May 2007

PLT Scheme version 370

posted by Jens Axel Søgaard

PLT Scheme version 370 is now available from


Some highlights:

  • The conservative garbage collector (CGC) has been replaced with a precise garbage collector (3m) in the standard build. For most users, this change simply amounts to “better performance in space and time”. For example, a long-running DrScheme instance typically uses much less memory than before.

The new memory manager also supports a new “Limit Memory…” option (in DrScheme’s “Scheme” menu) to limit the memory use of a programming running inside DrScheme.

For those who work with C-implemented libraries and extensions, the switch to precise collection may complicate interoperability. To a large extent, however, (lib “foreign.ss”) works the same with both collectors. (But note that the 3m is a moving collector, so be careful with passing Scheme objects to C.)

Although our pre-built binaries use the new collector, builds from source using the conservative collector are still supported.

  • For a program written with one of the the “How to Design Programs” (HtDP) languages, DrScheme saves the program with meta-information that identifies the language and records the teachpacks used by the program. DrScheme’s teachpack GUI now works only with the HtDP languages. In other languages, use require to access teachpacks.

The meta-information is in the form of a reader extension that turns the file content into a module-based program, which means that teaching-language files can be loaded directly into MzScheme or used with other PLT Scheme tools.

  • The HtDP “world.ss” and “image.ss” teachpacks have been revised, including support for the creation of animated GIFs.

  • Unit-based servlets are no longer supported in the web server. Use module-based servlets, instead. (Servlets can be implemented using a unit within a module, but the web server’s API is provided through a module.)

  • A new (lib “unit.ss”) library replaces the old one and provides a simpler and more flexible syntax. The (lib “unitsig.ss”) library is deprecated but still available as (lib “unitsig200.ss”), and the old (lib “unit.ss”) is available as (lib “unit200.ss”). Feedback Welcome, The PLT Scheme Team

more →

16 May 2007

Looking for small Scheme scripts

posted by Sam Tobin-Hochstadt

As part of the Typed Scheme project, we are looking for small Scheme scripts that we can port from PLT Scheme to Typed Scheme. We would like to investigate if Typed Scheme is capable of checking idiomatic PLT Scheme code, as represented by scripts written by members of the PLT Scheme community.

Therefore, if you have a simple PLT Scheme program which handles a scripting/processing task, and you are willing to share it with us for the improvement of Typed Scheme, please let me know. Typed Scheme currently handles all of ‘core’ MzScheme, as well as many of the collections (the major exceptions are the class and unit systems).

In return, we will inform you of any bugs that we discover during the port.

More information about Typed Scheme is available from the webpage: http://www.ccs.neu.edu/~samth/typed-scheme.html

more →

10 May 2007

Adjusting DrScheme’s Keybindings

posted by Robby Findler

Check out Kyle Smith’s blog post on how to change DrScheme’s keybindings:

Keybinding 101 in DrScheme

more →

10 May 2007

XML Transformation in Scheme

posted by Noel

Selenium is a tool for testing web applications, the core of which is a Javascript library that controls a web browser. With the Selenium IDE you can convert your actions in a web browser into tests, and with the Selenium Remote Control you can control a web browser from code. I’ve recently been working on adding Selenium Remote Control bindings to PLT Scheme, which has resulted in a nice and hopefully instructional demonstration of XML transformation in PLT Scheme

The Selenium Remote Control is controlled by sending simple messages over HTTP. The format of the messages isn’t important. What is, is that there are a lot of them, and the API is specified in a file called iedoc.xml that comes with Selenium. The Java/Python/Ruby bindings are generated using XSL. If I was to use XSL I’d have a processing pipeline that uses three languages (XSL, Java, Scheme) which is two more than I’d like. Hence I turned to WebIt!, an XML transformation DSL written in Scheme, to create an all Scheme pipeline. The rest of this post wshows the steps I used to transform the Selenium API into Scheme code using WebIt! I think this is interesting in its own right, but also serves as a nice demonstration of the power of macros, which WebIt! makes extensive use of.

My first step is to get an idea of the structure of the XML. The bits I’m interested in look like this:

<function name="click">
  <param name="locator">an element locator</param>
  <comment>Clicks on a link, button, checkbox or radio button.
  If the click action causes a new page to load (like a link usually
  does), call waitForPageToLoad.</comment>

Let’s read in the XML file and extract all the function elements. For this I’ll use SSAX and SXPath:

 (planet "ssax.ss" ("lizorkin" "ssax.plt" 1))
 (only (planet "sxml.ss" ("lizorkin" "sxml.plt" 1)) sxpath))

(define api
  (with-input-from-file "iedoc.xml"
    (lambda () (ssax:xml->sxml (current-input-port) '()))))

(define functions
  ((sxpath '(// function)) api))

Ok, so we have all the functions. Now let’s parse them into a more useful datastructure. Here’s my first attempt:

(require (planet "xml.ss" ("jim" "webit.plt" 1 5)))

;; struct function : string (listof string)
(define-struct function (name params))

;; parse-function : sxml -> function
(define (parse-function fn)
  (xml-match fn
    [(function name: ,name
               (param name: ,param-name ,desc) ...
               (comment ,_ ...))
     (make-function name (list param-name ...))]))

(map parse-function functions)

The xml-match macro is a pattern matcher for SXML. You specify the “shape” of the SXML, and if the input matches the pattern the following expressions are evaluated:

(xml-match value
  [(pattern expression _..._)]_..._)

The simplified form of a pattern is:

  • (element _..._) matches an element with the given name.

  • name: value matches an attribute with the given name and value.

  • ,binding binds the value of binding to the given name in the scope of the following expressions.

  • ... matches zero or more of the preceeding patterns.

In our example the pattern is:

     (function name: ,name
               (param name: ,param-name ,desc) ...
               (comment ,_ ...))

So we’re looking for an element called function with an attribute called name the value of which is bound to name. Then follows zero or more param elements, with attribute name, the value of which is bound to param-name. Finally we expect a comment element which can contain any amount of data. The use of _ as the binding name is a common convention to indicate data we don’t care about but must still match to make our pattern complete.

I run the code in DrScheme and see the result:

_xml-match: no matching clause found_

Oops. So our pattern isn’t complete. We’ve also seen one flaw of WebIt!: it doesn’t give very good error messages. However we can easily fix this by adding a catch all pattern that raises an error telling us what we failed to match. The code follows. Notice that I’ve also added pretty printing to make the unmatched SXML easier to read.

(require (lib "pretty.ss"))

;; parse-function : sxml -> function
(define (parse-function fn)
  (xml-match fn
    [(function name: ,name
               (param name: ,param-name ,desc) ...
               (comment ,_ ...))
     (make-function name (list param-name ...))]
    [,err (let ([op (open-output-string)])
            (pretty-print err op)
            (error (format "Didn't match ~n~a~n" (get-output-string op))))]))

Run this code and you’ll see the error occurs as we don’t allow the description to contain more than one element. This is easily fixed by extending the pattern to ,desc .... The next error is more interesting. The function element contains a return element. The WebIt! pattern language doesn’t allows us to express optional patterns, so we have to duplicate our pattern and include the case of return. This also requires we extend the defintion of the function structure.

;; struct function : string string (listof string)
(define-struct function (name return params))

;; parse-function : sxml -> function
(define (parse-function fn)
  (xml-match fn
    [(function name: ,name
               (param name: ,param-name ,desc ...) ...
               (comment ,_ ...))
     (make-function name "void" (list param-name ...))]
    [(function name: ,name
               (return type: ,type ,return-desc ...)
               (param name: ,param-name ,desc ...) ...
               (comment ,_ ...))
     (make-function name type (list param-name ...))]
    [,err (let ([op (open-output-string)])
            (pretty-print err op)
            (error (format "Didn't match ~n~a~n" (get-output-string op))))]))

This works! This is as far as I want to go in this article. We’ve seen how we can use SSAX. SXPath, and WebIt! to create XML transforms in pure Scheme. There is a lot more to all of these packages but what we’ve used is sufficient for many uses. The rest of the code to create Scheme from the API is quite straightforward and specific to Selenium. If you’re curious read the source of the Selenium PLaneT package, which will be released soon.

This post also appears on Untyping

more →

03 May 2007

Macros Matter

posted by matthias

Thank you Jens for setting up this Blog.

PLT Scheme is a 12-year old project now and it is definitely time to open it up to the world. The language and the project has contributed numerous ideas and products to the world. This covers programming languages (units, mixins, an implementation of cml-style concurrency, etc); programming tools (drscheme, check-syntax, transparent repls, module browsers, etc), programming pedagogy (htdp, htdc); program engineering (we resurrected the “expression” problem, web programming and continuations); and some more.

Time and again, people have asked me what I consider the one ‘feature’ that distinguishes us from the rest of the hordes of programming languages. I always respond with a single word:


We have pushed macros hard, and we have accomplished a lot with them. I conjecture that without macros, we would never have achieved the level of productivity that this group displays.

Of course, everyone else in academia works on types. ML’s module type system of the third kind and Haskell’s system-complete type system are serious challenges to anyone. It is probably true that you shouldn’t consider yourself a programmer if you can’t read and write some of those type-laden programs, and I seriously believe that they are the next generation of influential languages.

For the generation-after-the-next then, I see “macros” as one of the big topics (next to concurrency). A real programmer will have to know how Lisp and Scheme-style macros can reduce labor by orders of magnitude, how macros provide the tools for creating the “ultimate abstraction” in the form of domain-specific and embedded languages (Hudak’s words). And there is no better place to start with than PLT Scheme’s macro system.

So I would like to dedicate this blog to all things macros and everything else that matters in (and to) PLT Scheme.

more →

Made with Frog, a static-blog generator written in Racket.
Source code for this blog.