Project Release Information
In WorkQueue, memory errors that lead to a SEGFAULT, race conditions in signal handling for workers, and incorrect handling of the -C option where a worker would not use the same catalog server as work_queue_pool were corrected. Tasks now have a unique process group to properly kill all task children on abort. Escape codes such as \n and \t are now properly interpreted in Makeflow files.
WorkQueue now does string interpolation on task input files. Currently, only the $OS and $ARCH variables are replaced. WorkQueue has a new work_queue_name API to get the project name. A buffer overflow has been corrected in some regular expressions. WorkQueue now gracefully handles more than 1024 workers by moving from select to poll. Example Bioinformatics Makeflow applications for BLAST. An issue where Makeflow would crash with very long
commands has been corrected.
This release includes new support for running workflows on EC2, new Python and Perl bindings to the Work Queue system, a new example of replica exchange running directly on Work Queue, new comprehensive documentation for all components, and an improved build procedure tested on over 50 OS/architecture combinations.
This release fixes a race condition that would occasionally result in a crash after running a local process. Users are advised to upgrade.
This release adds new batch system drivers for Makeflow: Hadoop, Moab, and Work Queue with a shared filesystem. It improves configuration to accommodate a greater variety of operating systems, now including Cygwin on Windows. There are many minor bugfixes and improvements.
Makeflow is a workflow engine for executing large complex applications on clusters, clouds, and grids. It can be used to drive several different distributed computing systems, including Condor, SGE, and the included Work Queue system. It does not require a distributed filesystem, so you can use it to harness whatever collection of machines you have available. It is typically used for scaling up data-intensive scientific applications to hundreds or thousands of cores.