FileTree(parent, name, children, value)

Fields:

• parent::Union{FileTree, Nothing} – The parent node. nothing if it's the root node.

• name::String – Name of the root.

• children::Vector – children

• value::Any – the value at the node, if no value is present, a NoValue() sentinal value.

  FileTree(tree::FileTree; parent, name, children, value)

Copy over all fields from tree, but use any fields provided as keyword arguments.

FileTree(dirname::String)

Construct a FileTree to reflect directory from disk in the current working directory.

File(parent, name, value=NoValue())

Fields:

• parent::Union{FileTree, Nothing} – The parent node. nothing if it's the root node.

• name::String – Name of the root.

• value::Any – the value at the node, if no value is present, a NoValue() sentinal value.

name(node::Union{FileTree, File})

Get the file or directory name.

path(file::Union{File, FileTree)

Returns an AbstractPath object which is the path of the file from the root node leading up to this file.

parent(node::Union{FileTree, File})

Get the parent node. Returns nothing if there are no parents.

get(node)

Get the value stored in the node. NoValue() is returned if there is no value stored.

values(tree::FileTree; dirs=true)

Get a vector of all non-null values from nodes in the tree.

dirs=false will exclude any value stored in FileTree sub nodes.

nodes(tree::FileTree, dirs=true)

Get a vector of all nodes in the tree.

dirs=false will return only File nodes.

files(tree::FileTree)

Get a vector of all files in the tree.

dirs(tree::FileTree, dirs=true)

Get a vector of all directories in the tree.

rename(node::Union{FileTree, File}, newname)

Return a copy of node with name set to newname.

setparent(node::Union{FileTree, File}, parent)

Return a copy of node with parent set as the parent.

setvalue(node::Union{FileTree, File}, val)

Return a copy of node with val set as the value.

map(f, tree::FileTree; walk=FileTrees.postwalk, dirs=true)

apply f to every node in the tree. To only visit File nodes, pass dirs=false.

walk can be either FileTrees.postwalk or FileTrees.postwalk.

filter(f, tree::FileTree; walk=FileTrees.postwalk, dirs=true)

remove every node x from tree where f(x) is true. f(x) must return a boolean value.

mapsubtrees(f, t::FileTree, pattern::Union{GlobMatch, Regex})

For every node that matches the pattern provided, apply the function f.

If f returns either a File or FileTree, this new node will replace the matched node.

If f returns nothing, the matched node will be deleted

If f returns any other value, the value will be used as the value of the node and the node itself will be emptied of children.

This will allow use of mapsubtrees for complex use cases.

Suppose you would like to combine the values of a subdirectory with the function hcat and in turn those values using vcat, you can use mapsubtrees to accomplish this:

Here is a demo:

t = maketree("dir"=>([string(j)=>[(name=string(i), value=(i,j)]
                        for i=1:2] for j=1:3]

t1 = mapsubtrees("*") do subtree
    reducevalues(vcat, subtree)
end

reducevalues(hcat, t1)

merge(t1::FileTree, t2::FileTree; combine)

Merge two FileTrees. If files at the same path contain values, the combine callback will be called with their values to result in a new value.

If one of the dirs does not have a value, its corresponding argument will be NoValue() If any of the values is lazy, the output value is lazy as well.

diff(t1::FileTree, t2::FileTree)

For each node in t2 remove a node in t1 at the same path if it exists. Returns the difference tree.

mv(t::FileTree,
   from_path::Regex,
   to_path::SubstitutionString; combine)

move nodes in the file tree whose path matches the from_tree regular expression pattern by renaming it to to_path pattern. Any sub-pattern in from_path which is surrounded by paranthesis will be read as a matched substring which can be accessed in the to_path substitution pattern using \1, \2 etc. positional matches.

If a file overwrites an existing node after copy, combine will be called to combine them together. By default combine will error.

Example:

julia> t = maketree("dir" => [string(j) =>
                                [string(i)=>["data.csv"]
                                    for i = 1:2] for j=1:2])
dir/
├─ 1/
│  ├─ 1/
│  │  └─ data.csv
│  └─ 2/
│     └─ data.csv
└─ 2/
   ├─ 1/
   │  └─ data.csv
   └─ 2/
      └─ data.csv

julia> mv(t, r"^([^/]*)/([^/]*)/data.csv$", s"\1/\2.csv")
dir/
├─ 1/
│  ├─ 1.csv
│  └─ 2.csv
└─ 2/
   ├─ 1.csv
   └─ 2.csv

cp(t::FileTree,
   from_path::Regex,
   to_path::SubstitutionString; combine)

copy nodes in the file tree whose path matches the from_tree regular expression pattern by renaming it to to_path pattern. Any sub-pattern in from_path which is surrounded by paranthesis will be read as a matched substring which can be accessed in the to_path substitution pattern using \1, \2 etc. positional matches.

If a file overwrites an existing node after copy, combine will be called to combine them together. By default combine will error.

Example:

julia> t = maketree("dir" => [string(j) => [string(i)=>["data.csv"] for i = 1:2] for j=1:2])
dir/
├─ 1/
│  ├─ 1/
│  │  └─ data.csv
│  └─ 2/
│     └─ data.csv
└─ 2/
   ├─ 1/
   │  └─ data.csv
   └─ 2/
      └─ data.csv

julia> cp(t, r"^([^/]*)/([^/]*)/data.csv$", s"
/.csv")
dir/
├─ 1/
│  ├─ 1/
│  │  └─ data.csv
│  ├─ 1.csv
│  ├─ 2/
│  │  └─ data.csv
│  └─ 2.csv
└─ 2/
   ├─ 1/
   │  └─ data.csv
   ├─ 1.csv
   ├─ 2/
   │  └─ data.csv
   └─ 2.csv

rm(t::FileTree, pattern::Union{Glob, String, AbstractPath, Regex})

remove nodes which match pattern from the file tree.

load(f, t::FileTree; dirs=false)

Walk the tree and optionally load data for nodes in it.

f(file) is the loader function which takes File as input. Call path(file) to get the String path to read the file.

If dirs = true then f can either get a File or FileTree. nodes within FileTree will have already been loaded.

If NoValue() is returned by f, no value is attached to the node. hasvalue(x) tells you if x already has a value or not.

mapvalues(f, x::FileTree)

(See load to load values into nodes of a tree.)

Apply f to the value of all nodes in x which have a value. Returns a new tree where every value is replaced with the result of applying f.

f may return NoValue() to cause no value to be associated with a node.

reducevalues(f, t::FileTree; associative=true, init=nothing)

Use f to combine values in the tree.

• associative=true assumes f can be applied in an associative way

• init keyword argument will be returned if the file tree is empty. if init is not provided, reduce over an empty tree will cause an error to be thrown

save(f, x::Node)

Save a FileTree to disk. Creates the directory structure and calls f with File for every file in the tree which has a value associated with it.

(see load and mapvalues for associating values with files.)

compute(tree::FileTree; cache=true)

Compute any lazy values (Thunks) in tree and return a new tree where the values refer to the computed values (maybe on remote processes). The tree still behaves as a Lazy tree. exec on it will fetch the values from remote processes.

exec(x)

If x is a FileTree, computes any uncomputed Thunks stored as values in it. Returns a new tree with the computed values. If x is a Thunk (such as the result of a reducevalues), then exec will compute the result. If x is anything else, exec just returns the same value.

exec(ctx, x)

Same as exec(x) with a ctx being passed to Dagger when computing any Thunks.