marv

Access control lists

Access control lists.

Use access control lists (ACLs) to control who can perform which actions. An ACL is a function that returns a dictionary mapping route function names to list of groups being allowed to access the route. The special groups __authenticated__ and __unauthenticated__ are automatically assigned within authenticated, resp. unauthenticated sessions.

marv_webapi.acls.authenticated()[source]

Require authentication.

  • require authentication for everything

  • only admins may delete datasets

marv_webapi.acls.public()[source]

Allow public access.

  • anyone can read anything

  • authenticated users can comment, tag and compare

  • only admins may delete datasets

Creating datasets

Dataset scanner.

Datasets are created based on information provided by scanners. A scanner is responsible to group files into named datasets:

def scan(dirpath, dirnames, filenames):
    return [DatasetInfo(os.path.basename(x), [x])
            for x in filenames
            if x.endswith('.csv')]

Scanners are called for every directory within the configured scanroots, while files and directories starting with a . and directories containing an (empty) .marvignore file are ignored and will not be traversed into.

Further, traversal into subdirectories can be controlled by altering the dirnames list in-place. To block further traversal, e.g. for a directory-based dataset type, set it to an empty list – os.walk() is used behind the scenes:

dirnames[:] = []
class marv.scanner.DatasetInfo(name, files)

Bases: tuple

property files

Alias for field number 1

property name

Alias for field number 0

Declaring nodes

marv.node(schema=None, header=None, group=None, version=None)[source]

Turn function into node.

Parameters
  • schema – capnproto schema describing the output messages format

  • header – This parameter is currently not supported and only for internal usage.

  • group (bool) – A boolean indicating whether the default stream of the node is a group, meaning it will be used to published handles for streams or further groups. In case of marv.input.foreach specifications this flag will default to True. This parameter is currently only for internal usage.

  • version (int) – This parameter currently has no effect.

Returns

A Node instance according to the given arguments and input() decorators.

marv.input(name, default=None, foreach=None)[source]

Declare input for a node.

Plain inputs, that is plain python objects, are directly passed to the node. Whereas streams generated by other nodes are requested and once the handles of all input streams are available the node is instantiated.

Parameters
  • name (str) – Name of the node function argument the input will be passed to.

  • default – An optional default value for the input. This can be any python object or another node.

  • foreach (bool) – This parameter is currently not supported and only for internal usage.

Returns

The original function decorated with this input specification. A function is turned into a node by the node() decorator.

Interacting with marv

exception marv.Abort[source]

Bases: Exception

marv.api_endpoint(url_rule, defaults=None, methods=None, version=None, cls=<class 'marv_webapi.tooling.APIEndpoint'>, registry=None, force_acl=None)[source]
marv.api_group(url_prefix=None, cls=<class 'marv_webapi.tooling.APIGroup'>)[source]
marv.create_group(name, **header)[source]
marv.create_stream(name, **header)[source]

Create a stream for publishing messages.

All keyword arguments will be used to form the header.

marv.fork(name, inputs, group)[source]
marv.get_logger()[source]
marv.get_requested()[source]
marv.get_stream(node, name='default', setid=None)[source]
marv.make_file(name)[source]
marv.pull(handle, enumerate=False)[source]

Pull next message for handle.

Parameters
  • handle – A stream.Handle or GroupHandle.

  • enumerate (bool) – boolean to indicate whether a tuple (idx, msg) should be returned, not unlike Python’s enumerate().

Returns

A Pull task to be yielded. Marv will send the corresponding message as soon as it is available. For groups this message will be a handle to a member of the group. Members of groups are either streams or groups.

Examples

Pulling (enumerated) message from stream:

msg = yield marv.pull(stream)
idx, msg = yield marv.pull(stream, enumerate=True)

Pulling stream from group and message from stream:

stream = yield marv.pull(group)  # a group of streams
msg = yield marv.pull(stream)
marv.pull_all(*handles)[source]

Pull next message of all handles.

marv.push(msg)[source]
marv.select(node, name)[source]

Select specific stream of a node by name.

Parameters
  • node – A node producing a group of streams.

  • name (str) – Name of stream to select.

Returns

Node outputting selected stream.

marv.set_header(**header)[source]

Set the header of a stream or group.