Summary of the paper
| Title | LoonyBin: Keeping Language Technologists Sane through Automated Management of Experimental (Hyper)Workflows |
|---|---|
| Authors | Jonathan H. Clark and Alon Lavie |
| Abstract | Many contemporary language technology systems are characterized by long pipelines of tools with complex dependencies. Too often, these workflows are implemented by ad hoc scripts; or, worse, tools are run manually, making experiments difficult to reproduce. These practices are difficult to maintain in the face of rapidly evolving workflows while they also fail to expose and record important details about intermediate data. Further complicating these systems are hyperparameters, which often cannot be directly optimized by conventional methods, requiring users to determine which combination of values is best via trial and error. We describe LoonyBin, an open-source tool that addresses these issues by providing: 1) a visual interface for the user to create and modify workflows; 2) a well-defined mechanism for tracking metadata and provenance; 3) a script generator that compiles visual workflows into shell scripts; and 4) a new workflow representation we call a HyperWorkflow, which intuitively and succinctly encodes small experimental variations within a larger workflow. |
| Topics | Tools, systems, applications |
Full paper |
LoonyBin: Keeping Language Technologists Sane through Automated Management of Experimental (Hyper)Workflows |
Slides |
LoonyBin: Keeping Language Technologists Sane through Automated Management of Experimental (Hyper)Workflows |
| Bibtex | @InProceedings{CLARK10.570, author = {Jonathan H. Clark and Alon Lavie}, title = {LoonyBin: Keeping Language Technologists Sane through Automated Management of Experimental (Hyper)Workflows}, booktitle = {Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Bente Maegaard and Joseph Mariani and Jan Odijk and Stelios Piperidis and Mike Rosner and Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |