For the second consecutive year, Google has conducted its Summer of Code™ initiative, a program by which student developers are sponsored for their contributions within open source organizations willing to mentor the participants. The 2006 campaign has run between April and September, with active development work taking place between May 23 and August 21.
Around mid April, when the program had just started, some Boost members began considering the possibility to enter Summer of Code as a mentoring organization. Despite the lack of time and the fact that most of us were completely new to this initiative, Boost managed to successfully apply for the program. As a result ten projects were selected and mentored, most of which are expected to become full contributions to Boost in the near future.
We give here a summary report of this experience, along with a short analysis of the main problems we found, so that we can work at solving them and do better next year.
There are three types of participants in Google Summer of Code:
The 2006 campaign of Google Summer of Code took place between April 14 and September 25. A total of 102 mentoring organizations participated. Of the 6,338 applications submitted by 3,044 students around the globe, 630 were finally selected and funded. Google has spent more than US$3 million in student stipends and compensations to the mentoring organizations.
On April 14, the same day Google Summer of Code started, Julio M. Merino Vidal (later to become one of the selected students) sent a message encouraging Boost members to participate in this program as a mentoring organization. This call sparked the interest of the community; although time was already short for doing all the preparation labors, Boost moderators put rapidly themselves to work and conducted the preliminary registration steps. In the meantime, a Wiki page was grown with project ideas provided by Boost members, totalling more than twenty proposals.
By the beginning of May Boost was officially accepted into the program and Boost moderators set out to form a group of mentors, selected on an invitation basis. As student selection is a delicate process, involving the assessment of individuals on their technical skills, all subsequent discussions were conducted by the selected mentors on a private mail list established for their collaboration.
We were not prepared for the avalanche of student applications that followed. On day two after the application period was open, we had received three proposals; next day it was 14, and within a week the count exceeded 50. By the end of the application period the total number of proposals received was 174, which forced us to go through a very intensive ranking process and recruit additional mentors. Two rules were followed so as rationalize the process of selection among dozens of different proposals:
The different proposals were classified according to its related technological area so that each cluster could be handled by an appointed mentor with the required expertise on the subject. Mentors submitted then "focus reports" summarizing the applications under their responsibility; these reports served as a first filter to help reduce the number of final applications to be evaluated jointly. Along the process, students with the most promising proposals were asked to refine their ideas and provide further information.
Although not enforced by the official rules, we agreed upon a one-to-one ratio of mentors to students, which ultimately marked a hard limit on the maximum number of eligible projects.
Google accepted and funded the ten top-ranked projects endorsed by Boost. Of these, eight projects are libraries or library components targeted for future inclusion into Boost, while the remaining two consist of utility programs heavily relying on Boost.
C++ Coroutine Library
Giovanni Piero Deretta, mentored by Eric Niebler.
Library for the management through a modern C++ interface of OS-provided coroutine facilities.
Concurrency Library
Matthew Calabrese, mentored by David Abrahams.
STL-inspired generic framework for high-level specification and execution of parallelizable algorithms.
TR1 Math Special Functions
Xiaogang Zhang, mentored by John Maddock.
Implementation of the 23 special mathematical functions specified in C++ standard library extension proposal TR1.
The Boost.Process library
Julio M. Merino Vidal, mentored by Jeff Garland.
Portable library for process launching and basic management.
Out-of-Core Graphs and Graph Algorithms
Stéphane Zampelli, mentored by Jeremy Siek.
Extension of the Boost Graph Library to deal with out-of-core structures, i.e. data sets too large to be kept in main memory at once.
MISC (M)ulti (I)ndex (S)pecialized (C)ontainers
Matías Capeletto, mentored by Joaquín M López Muñoz.
Families of specialized containers internally based on Boost.MultiIndex.
Generic Tree Container
Bernhard Reiter, mentored by René Rivera.
Design and implementation of a family of STL-compatible tree containers.
Viewer utility for FSMs
Ioana Tibuleac, mentored by Andreas Huber Dönni.
Utility program for the visualization of finite state machines (FSMs) specified with Boost.Statechart.
Modular C++ preprocessor, using Boost.Spirit
Hermanpreet 'Lally' Singh, mentored by Joel de Guzman.
Implementation with Boost.Spirit and Boost.Wave of a front-end translator from Modular C++ (as specified in a proposal to add modules to C++ by Daveed Vandevoorde) to standard C++.
Implementing a state of the art Mincut/Maxflow algorithm.
Stephan Diederich, mentored by Douglas Gregor.
Implementation of a fast mincut/maxflow routine for the Boost Graph Library based on a new algorithm devised by Vladimir Kolmogorov.
Two main facilities were set up to assist students and mentors during the development phase: a mailing list and a Trac/SVN project management system with separate directories for each project. One of the students, Matías Capeletto, out of personal initiative registered a Google Group aimed at giving students with Boost a place for informal interaction and discussion of common problems.
After the initial warm-up period, each student-mentor pair performed development work mostly privately. The usage of the Boost mailing lists was scarce, and only by the end of the program did some students publicly announced their results.
By the date the development period was officially closed, the status of the different projects was as follows:
We examine the various stages of Boost participation in Summer of Code, with an emphasis on discovering opportunities for improvement.
In a mid project presentation at OSCON 2006, Chris DiBona from Google provided some data about the organizations which received the most applications:
| Organization | No of applications |
|---|---|
| KDE | 244 |
| Ubuntu & Bazaar | 236 |
| Python Software Foundation | 212 |
| GNOME | 199 |
| Apache Software Foundation | 190 |
| Boost | 174 |
| Gaim | 152 |
| The GNU Project | 148 |
| Drupal | 146 |
The numbers shown here have been estimated from a chart included in the presentation slides. This chart contains an additional column labeled "Google" which actually accounts for the applications dismissed because of their low quality.
The fact that Boost is ranked the sixth most attractive organization out of a total of 102 was entirely unexpected, especially considering the wide popularity of the rest of top-rated organizations. There is a more or less implicit consensus among Boost members that ours is a relatively niche project, known for its quality standards by seasoned C++ practitioners, but with a limited penetration among entry level programmers: maybe the figures above should make us reconsider this assumption. A cursory examination of the applications submitted to Boost reveals that most applicants were regular users of Boost: many cite the Boost status among the C++ community as an appealing factor in order to apply.
If we look at the number of funded projects with respect to the applications received, figures are not so favorable to Boost.
| Organization | No of projects | Project/app ratio |
|---|---|---|
| KDE | 24 | 9.8 % |
| Ubuntu & Bazaar | 22 | 9.3 % |
| Python Software Foundation | 23 | 10.8 % |
| GNOME | 19 | 9.5 % |
| Apache Software Foundation | 27 | 14.2 % |
| Boost | 10 | 5.7 % |
| Gaim | 8 | 5.3 % |
| The GNU Project | 10 | 6.8 % |
| Drupal | 14 | 9.6 % |
It turns out that the project/application ratio for almost any other organization among the top nine is considerably higher than that of Boost. As it happens, Google initially requested that organizations submitted the maximum number of projects they felt they could cope with, and we got funding for exactly what we aimed for, so the limiting factor lies entirely on Boost's side.
Contributing to Boost relies on a fair number of guidelines and protocols for coding, documentation, testing and maintenance. Many of the required tools are exclusively used within Boost, and some of them are not trivial, like for instance Boost.Build. Although the Boost web site contains information about all these tools and procedures, this intelligence is scattered through unrelated pages and sometimes is very hard to come by.
So, there is a good deal of expertise required to begin working at Boost. Some
students have reported on startup difficulties getting to know these details and
familiarizing themselves with the tools, most notably bjam and Quickbook. Each
student overcome the startup difficulties on their own or resorting to their
mentors (see the section on public
communication issues).
Once students got past the startup stage, most projects advanced without serious complications. In the majority of cases, it was realized at some point during the development that there was no time to complete it. Some participants had to redefine the goals in an effort to keep the project within schedule, while others simply decided that they would continue working after the official deadline of Summer of Code.
The information flow between each student and their mentor was usually reported by both parties to be satisfactory. The projects suffering from lack of communication have been precisely those yielding the poorest results. In general, mentors have not felt overwhelmed by requests from their students, and even in a couple of cases the projects were run practically unattendedly. This fact is witness to the high competence of the students recruited into the program.
The degree of usage of the Trac/SVN system has varied. Some students did frequent updates, while others have just used the repository to dump the final results for the official submission to Google.
Students and mentors had at their disposal three different forums for the public interchange of information and support:
In hindsight, it has become apparent that most projects were too ambitious to be completed within the three months of duration of the program, and even those that were considered a success will need weeks or months of polishing up before the material is ready for a formal review. In contrast with other organizations participating in the Summer of Code program, Boost has as of this writing included no results into its code base. No formal review for any project has been requested yet, either.
These scope issues are very dependent on the particular type of project. We can classify the Boost projects for Summer of Code as follows:
As for those projects involving the design and realization of full-fledged libraries, there is little hope that the goals and scope can be kept modest enough for a three-month schedule. Boost candidate libraries developed by professional authors usually take much longer than three months to be accepted; some libraries have been evolving through several years before being included into Boost. So, the best we can hope for if we are to support the realization of library projects for Boost inside Summer of Code is that the results by the end of the program can be evaluated to constitute a viable potential contribution to Boost. When this is the case, it is crucial that the student commits to further working on the project up to completion and formal review. Perhaps more important than getting libraries coded is to engage new authors into a long-term relationship with the Boost project.
The following proposals aim to alleviate some of the problems we have identified during the development of Summer of Code within Boost. These action points are related only to the issues found in connection with Boost: we are not addressing other areas of improvement associated to the Summer of Code program itself.
Much work can be done before the actual program begins. The following preparation activities can already be launched:
Create a pool of ideas for projects. This action will provide valuable extra time for evaluation and refining of ideas before the Summer of Code begins. The experience has shown that those projects with more preparation work, especially in the area of design, were ultimately more successful. The pool can also be used to retain interesting ideas that arise at the mailing lists and very often are not given proper attention and become abandoned.
Create a student pool. Prior involvement with Boost is clearly an advantage both in the selection phase and later during project development. Those students with a serious interest in participating in Summer of Code with Boost can enter the pool and begin exploring ideas and interacting with the community well in advance of the summer, so as to put themselves in a favorable position for the selection. Advertisement for the student pool can be initiated in the beginning of 2007 through the usual channels (web site and mailing lists): additionally, Boost members involved with the University can spread this information locally and help raise the interest of students in their environment.
Create a mentor pool. Given the rush with which Boost entered the 2006 Summer of Code campaign, the invitation of mentors has to be done on an on-demand basis as it became all too evident that the task was growing bigger and bigger. It is important that the organization is better prepared next year so that a number of people with the ability and will to participate as Boost mentors are identified in advance.
Prepare a startup package. In order to facilitate the initial period of getting familiarized with the various Boost guidelines, protocols and tools, it would be extremely useful to prepare a compilation of startup material for students. This package can consist of a single document gathering the currently dispersed information, or go beyond this and provide some bundle of documentation and pre-built tools, an approach that one of the students is currently working on.
It is crucial that students get involved with the community as soon as possible and grow to appreciate the advantages of public development with respect to solitary coding.
Mandate (bi)weekly reports. These reports should be directed to the public mailing lists so as to give all Boost members an opportunity to follow the work in progress and contribute. Reporting has the extra benefit for students of forcing them to reflect on their own work periodically and struggle with the often difficult task of presenting their ideas to others.
Conduct student-mentor exclusively through public channels. This might be too drastic a policy, as some matters need privacy, and depending on the amount of information exchanged flooding problems may arise. Less severe variations involve allowing for some private interchange at the mentors' discretion and moving this kind of communication to a dedicated public mailing list different from the general ones.
The two most important issues to improve upon with respect to the management are:
Create a best practices document. This document can serve as a guideline for project management, an area in which Boost traditionally imposes no requirements. Students might lack the expertise in this area that is usually taken for granted in the traditional model where contributions to Boost are made by professional programmers.
Mandate a design phase. Having a concrete design set up and clearly described early in the project will help estimate the necessary effort for completion of the work. This is also an opportunity for public discussion.
Maintain code, docs and tests in parallel. All too often, novice programmers do the coding in one fell swoop and only then move to testing and documenting their work. This is unacceptable by all current methodology standards, and can result in serious underestimations of the time to completion.
Encourage the KISS principle. It is much better to finish a simpler library and then iteratively evolve it, once it has been exposed to public scrutiny and usage.
More Trac updates. The repository should be viewed as an everyday work tool, not only as the place into which to dump the final results. Updating often leads to more visibility of the work by the mentor and the public in general.
Informal reviews. The typical Summer of Code Boost project will not be completed by the official deadline, as have been discussed earlier. To somehow officialize the work done within the Summer of Code proper, and also to allow the students to reach some sort of psychological milestone, informal reviews can be instituted where Boost members evaluate the work done at then end of Summer of Code.
Engage students. This experience has shown that it is possible to guide willing and bright students to the competence levels required for contributing to Boost. The best possible outcome of Summer of Code campaigns are the incorporation of new people into the circle of Boost active contributors. Strive to make the students commit to Boost.
Despite the lack of previous experience in Boost, our participation in Google Summer of Code has been extremely fruitful: much useful material has been produced, and, perhaps more importantly, some of the students are likely to commit on a long-term basis and grow to be regular Boost contributors. Traditionally, becoming a productive Boost author has a very high entry barrier due to the extreme quality standards, lack of public support and the very specific culture of the project. The appeal of Summer of Code itself and the possibility of being gently mentored into the world of Boost have most likely been key factors in lowering this entry barrier.
The process has not been without some difficulties, either, as it was expected of a newcomer organization as Boost. We have tried to identify in this paper the areas of improvement and suggest specific actions so that the upcoming Google Summer of Code 2007 can be an even more rewarding experience.
This paper couldn't have been written without the numerous reports and contributions kindly provided by Boost students and mentors: Many thanks to all the participants for sharing their experiences with me. Thank you also to the people at Google who have promoted and conducted the Summer of Code initiative.
Revised October 17th 2006
© Copyright 2006 Joaquín M López Muñoz. Distributed under the Boost Software License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)