Ruffus is a free and open source lightweight way to add support for running computational pipelines.
Computational pipelines are often conceptually quite simple, especially if we breakdown the process into simple stages, or separate tasks.
Each stage or task in a computational pipeline is represented by a python function Each python function can be called in parallel to run multiple jobs.
The purpose of a pipeline is to determine automatically which parts of a multistage process needs to be run and in what order in order to reach an objective ("targets")
Computational pipelines, especially for analysing large scientific datasets are in widespread use. However, even a conceptually simple series of steps can be difficult to set up and to maintain, perhaps because the right tools are not available.
Here are some key features of "Ruffus":
· Managing dependencies
· Parallel jobs
· Re-starting from arbitrary points, especially after errors
· Display of the pipeline as a flowchart
· Reporting
Requirements:
· Python
What`s New in This Release: [ read full changelog ]
· Touch files without running the pipeline
· Parameter substitution for `inputs(...)` / `add_inputs(...)`
· Simplifying `@transform` syntax with suffix
· Advanced form of `@split`