FOREACH file/item

Builds a list of paths on the machine the job runs on and runs the ops nested underneath it once per path, with the path in a loop variable. The list comes from a directory listing, a whole directory tree, or the item paths of the nature currently being processed.

It shares its form with Load files/items into stash, which computes the same list and hands it back as one value instead of looping over it. Reach for that one when you want to count the paths or pass them along in one piece, and for this one when you want to act on each path.

Fields

Variable

The name of the loop variable each path is placed in. Type a bare name and do not leave it empty.

There is a fallback of file, but it only applies to an op whose form has never been saved. Apply the form with this field empty and the loop variable is created with an empty name instead, which no nested op can read. Nothing flags it.

This one field is taken literally, so ${bar} here creates a variable whose name contains those characters rather than reading bar from the stash. Every other field on the form does expand placeholders.

The variable lives for the duration of the loop. Once the last pass ends it reverts to whatever it held before the op, so collect anything you want to keep into a separate variable with PUSH VAR.

Rule quality analysis does not know this variable is created here, so nested ops that read ${myfile} are reported as using an unknown variable. The report is wrong in this case and the rule runs correctly; see Rule quality analysis.

Return Key on the Options tab does nothing on this op. The loop variable is the only way in.

Path

The base path to search, expanded from the stash, so ${job_dir}/${project}/dist works.

Required for both file modes. Left blank there, the op stops the job with a message about the root path not being configured.

A path with no * or ? in it has to exist, or the op stops the job. A path containing either character skips that check and is used as a pattern.

In Nature Items mode the field changes meaning: it is optional, and when filled it is stripped off the front of each path rather than searched. The existence check still runs on it, so a prefix that names a directory the job never created stops the job there.

The field is a multi-line box, but the whole contents are treated as one path. It is not a list.

Path Mode

Choice	What it lists
Files, non Recursive	the direct contents of Path, one level deep
Recursive Files	everything under Path, at any depth
Nature Items	the item paths of the nature currently in scope

Default is Files, non Recursive. There, a Path pointing at a directory lists that directory's children, and a Path containing a wildcard is expanded as a shell pattern. No other mode accepts wildcards.

That expansion brings two shell habits with it. Entries whose name starts with a dot are never listed, so a .env or a .gitignore in the directory is invisible to this mode. And a space anywhere in the path splits the pattern in two, which normally leaves you with an empty list and no error. Point the op at a directory without spaces, or use Recursive Files, which has neither problem.

Recursive Files walks a tree and wants a plain directory. A wildcard here is taken literally and the op stops the job when it cannot open that directory. The Path directory itself is part of the walk, so under Only Directories and Files and Directories the first entry you get is the root you pointed at. Drop it with an exclude row when the body cannot cope with it.

Nature Items ignores Path when finding files and reads the item paths of the nature being processed. Those are set up by IF EXISTS nature THEN, listed in the palette as FOR nature DO, and they last only as long as that block. Outside one, the list is empty and nothing warns you. Paths are resolved under the job directory and entries that no longer exist on disk are dropped, so items the changeset deleted do not turn up.

Dir Mode

Choice	Meaning
Files only	plain files, no directories
Files and Directories	both
Only Directories	directories, no files

Default is Files only.

Files and Directories does what it says in Recursive Files and Nature Items mode. It does not in Files, non Recursive: there, anything other than Files only returns directories alone, so a flat listing asking for both hands you the subdirectories and none of the files. When you need both from a single directory, place the op twice.

Return relative paths?

Off by default. Ticked, the leading Path is cut off every entry, so /opt/jobs/DEV-1/myapp/bar/baz.war arrives in the loop variable as bar/baz.war.

In Nature Items mode with Path filled, the paths were already cut to that prefix on the way out, so the box changes nothing there.

Ticking it while Path is blank produces an empty list and therefore zero passes. That combination is reachable in Nature Items mode, where Path is optional, and it fails silently: no error, no log line, the body never runs.

Filters

A two-tab panel holding the two grids below. Each grid is one unlabelled column with Delete and Add above it. Add appends a row pre-filled with .*, which matches everything. One click on a row starts editing it, and Delete removes the row you have selected. A row cannot be saved empty.

Include Paths

Regular expressions, one per row. A path survives when it matches at least one row. An empty grid lets everything through, which is the normal setting.

These are regular expressions and not shell patterns. *.jar is not what you want; write \.jar$. The match is against the whole path string, so /target/ filters by directory.

For a case-insensitive row, or any other expression flag, wrap the pattern in double exclamation marks and put the flags after it: !!\.JAR$!!i.

Exclude Paths

Regular expressions, one per row, applied after the include rows. One match drops the path. Exclude wins over include.

Which rule dropped which path is written to the job log, at debug level and only while the server itself runs with debugging on. Switch the log's debug filter on to read it. With debugging off, a file that vanishes from the loop leaves no trace at all.

Behaviour worth knowing

The listing is of the filesystem the job itself runs on. To walk files on a remote node, run a command there with Run a Remote Script instead.

A list that comes back empty is not an error. The body runs zero times and nothing is logged, which looks identical to a rule where the op was never reached.

Order depends on the mode. Files, non Recursive comes back sorted by name, because the pattern expansion sorts it. Recursive Files walks level by level, parents before children, and within a directory it takes whatever order the filesystem gives. Nature Items keeps the order the items were loaded in. Nothing is de-duplicated.

The whole list is built before the first pass. Files created by the body do not join the loop, and files the body deletes are still handed out on later passes.

A failure in the body ends the loop and the job. Wrap the body in TRY statement with CATCH statement when one bad file should not stop the rest.

Combining with other ops

Put it inside FOR projects with changes DO so that ${project} and the project variables are available for building Path.

Inside the body, Ship File Remotely, Replace Strings and Delete Local File all take the loop variable directly.

Use Load files/items into stash with the same settings when you want the count first, then branch on it with IF var condition THEN before committing to a loop.

Examples

Ship every archive built for the current project, skipping the ones marked as sources.

Variable    myfile
Path        ${job_dir}/${project_lc}/target
Path Mode   Files, non Recursive
Dir Mode    Files only
Include Paths   \.jar$
Exclude Paths   -sources\.jar$
  -> Ship File Remotely    from ${myfile}

Walk every file the current nature matched, relative to the job directory.

Variable    myitem
Path        ${job_dir}
Path Mode   Nature Items
Dir Mode    Files only
Return relative paths?   off
  -> Replace Strings    in ${myitem}