I've submitted a pull request that adds a "logDirectory" argument to Queue. Quoting myself:
Complex Queue pipelines with a large number of intermediate steps can generate more log files than actual output, cluttering the working directory. This patch introduces a command-line parameter --logDirectory/-logDir that allows those files to be sequestered into a single directory.
The use of absolute pathnames changes the behavior of logDir. The possibilities are:
- logDir is not specified: No change from the existing behavior (the .out file is in the same directory as the @Output)
- The location for the @Output element is relative: The relative path for @Output will be rooted in logDir (whether logDir is relative or absolute)
- The @Output element is absolute: logDir is ignored, as if it were not specified
My assumption is that absolute directories will be rare, and that when they are used they should override any other considerations. I've tested that this works as described when logDir is not specified and when it has a value of "log" (i.e., a relative path).
At the urging of the GSA team, and following a nice model introduced by @Johan_Dahlberg, I'd like to open this change up for public comment. Obviously, I'm only considering a fairly restricted use case. Does this behavior make sense to others? Does it break anybody's workflow in any significant way?