d1_cli.impl package

DataONE Command-line Client.

Submodules

d1_cli.impl.access_control module

Create and manipulate access control objects.

class d1_cli.impl.access_control.AccessControl

Bases: object

get_list()

add_allowed_subject(subject, permission)

remove_allowed_subject(subject)

clear()

d1_cli.impl.check_dependencies module

Check the dependencies by attempting to import them.

d1_cli.impl.check_dependencies.are_modules_importable(module_list=None)

d1_cli.impl.client module

CN and MN clients of the DataONE Command Line Interface.

The logic was factored out of these and reimplemented in more natural locations. This is just a placeholder for now. May remove.

class d1_cli.impl.client.CLIClient

Bases: object

class d1_cli.impl.client.CLIMNClient(*args, **kwargs)

Bases: d1_cli.impl.client.CLIClient, d1_client.mnclient_2_0.MemberNodeClient_2_0

class d1_cli.impl.client.CLICNClient(*args, **kwargs)

Bases: d1_cli.impl.client.CLIClient, d1_client.cnclient_2_0.CoordinatingNodeClient_2_0

class d1_cli.impl.client.CLIBaseClient(*args, **kwargs)

Bases: d1_client.baseclient_2_0.DataONEBaseClient_2_0

d1_cli.impl.command_parser module

• Generic boiler plate for Python CLI apps.

• Simple command tokenizing and validation.

class d1_cli.impl.command_parser.CLI

Bases: cmd.Cmd

preloop()

Initialization before prompting user for commands.

Despite the claims in the Cmd documentation, Cmd.preloop() is not a stub.

postloop()

Take care of any unfinished business.

Despite the claims in the Cmd documentation, Cmd.postloop() is not a stub.

precmd(line)

This method is called after the line has been input but before it has been interpreted.

If you want to modify the input line before execution (for example, variable substitution) do it here.

postcmd(stop, line)

If you want to stop the console, return something that evaluates to true.

If you want to do some post command processing, do it here.

emptyline()

Do nothing on empty input line.

default(line)

Called on an input line when the command prefix is not recognized.

run_command_line_arguments(cmd_line_list)

do_help(line)

Get help on commands “help” or “?” with no arguments displays a list_objects of commands for which help is available “help <command>” or “?

<command>” gives help on <command>

do_history(line)

history Display a list of commands that have been entered.

do_exit(line)

exit Exit from the CLI.

do_quit(line)

quit Exit from the CLI.

do_eof(line)

Exit on system EOF character.

do_set(line)

set [parameter [value]] set (without parameters): Display the value of all session variables.

set <session variable>: Display the value of a single session variable. set <session variable> <value>: Set the value of a session variable.

do_load(line)

load [file] Load session variables from file load (without parameters): Load session from default file ~/.dataone_cli.conf load.

<file>: Load session from specified file.

do_save(line)

save [config_file] Save session variables to file save (without parameters): Save session to default file ~/.dataone_cli.conf save.

<file>: Save session to specified file.

do_reset(line)

reset Set all session variables to their default values.

do_allowaccess(line)

allowaccess <subject> [access-level] Set the access level for subject Access level is “read”, “write” or “changePermission”.

Access level defaults to “read” if not specified. Special subjects: public: Any subject, authenticated and not authenticated authenticatedUser: Any subject that has authenticated with CILogon verifiedUser: Any subject that has authenticated with CILogon and has been verified by DataONE

do_denyaccess(line)

denyaccess <subject> Remove subject from access policy.

do_clearaccess(line)

clearaccess Remove all subjects from access policy Only the submitter will have access to the object.

do_allowrep(line)

allowrep Allow new objects to be replicated.

do_denyrep(line)

denyrep Prevent new objects from being replicated.

do_preferrep(line)

preferrep <member node> [member node …] Add one or more preferred Member Nodes to replication policy.

do_blockrep(line)

blockrep <member node> [member node …] Add one or more blocked Member Node to replication policy.

do_removerep(line)

removerep <member node> [member node …] Remove one or more Member Nodes from replication policy.

do_numberrep(line)

numberrep <number of replicas> Set preferred number of replicas for new objects If the preferred number of replicas is set to zero, replication is also disallowed.

do_clearrep(line)

clearrep Set the replication policy to default.

The default replication policy has no preferred or blocked member nodes, allows replication and sets the preferred number of replicas to 3.

do_get(line)

get <identifier> <file> Get an object from a Member Node.

The object is saved to <file>.

do_meta(line)

meta <identifier> [file] Get the System Metadata that is associated with a Science Object.

If the metadata is not on the Coordinating Node, the Member Node is checked.

Provide file to save the System Metada to disk instead of displaying it.

do_list(line)

list [path] Retrieve a list of available Science Data Objects from Member Node The response is filtered by the from-date, to-date, search, start and count session variables.

See also: search

do_log(line)

log [path] Retrieve event log from Member Node The response is filtered by the from-date, to-date, start and count session parameters.

do_resolve(line)

resolve <identifier> Find all locations from which the given Science Object can be downloaded.

do_create(line)

create <identifier> <file> Create a new Science Object on a Member Node.

The System Metadata that becomes associated with the new Science Object is generated from the session variables.

do_update(line)

update <old-pid> <new-pid> <file> Replace an existing Science Object in a.

MN with another.

do_package(line)

package <package-pid> <science-metadata-pid> <science-pid> [science- pid.

…] Create a simple OAI-ORE Resource Map on a Member Node.

do_archive(line)

archive <identifier> [identifier …] Mark one or more existing Science Objects as archived.

do_updateaccess(line)

updateaccess <identifier> [identifier …] Update the Access Policy on one or more existing Science Data Objects.

do_updatereplication(line)

updatereplication <identifier> [identifier …] Update the Replication Policy on one or more existing Science Data Objects.

do_listformats(line)

listformats Display all known Object Format IDs.

do_listnodes(line)

listnodes Display all known DataONE Nodes.

do_search(line)

search [query] Comprehensive search for Science Data Objects across all available MNs.

See https://releases.dataone.org/online/api- documentation-v2.0.1/design/SearchMetadata.html for the available search terms.

do_ping(line)

ping [base-url …] Check if a server responds to the DataONE ping() API method ping (no arguments): Ping the CN and MN that is specified in the session ping <base-url> [base-url …]: Ping each CN or MN.

If an incomplete base-url is provided, default CN and MN base URLs at the given url are pinged.

do_queue(line)

queue Print the queue of write operations.

do_run(line)

run Perform each operation in the queue of write operations.

do_edit(line)

edit Edit the queue of write operations.

do_clearqueue(line)

clearqueue Remove the operations in the queue of write operations without performing them.

d1_cli.impl.command_processor module

Process and execute CLI operations.

class d1_cli.impl.command_processor.CommandProcessor

Bases: object

get_session()

get_operation_queue()

get_nodes()

get_format_ids()

ping(hosts)

search(line)

CN search.

list_format_ids()

list_nodes()

resolve(pid)

Get Object Locations for Object.

update_access_policy(pids)

update_replication_policy(pids)

science_object_get(pid, path)

First try the MN set in the session.

Then try to resolve via the CN set in the session.

system_metadata_get(pid, path)

log(path)

list_objects(path)

science_object_create(pid, path, format_id=None)

Create a new Science Object on a Member Node.

science_object_update(pid_old, path, pid_new, format_id=None)

Obsolete a Science Object on a Member Node with a different one.

create_package(pids)

science_object_archive(pids)

d1_cli.impl.const module

Constants (mostly names of things}.

class d1_cli.impl.const.SessionVariable(sect, name)

Bases: object

d1_cli.impl.exceptions module

Classes used for passing exceptions within and between components of the DataONE Command Line Client.

exception d1_cli.impl.exceptions.InvalidArguments(value)

Bases: Exception

exception d1_cli.impl.exceptions.CLIError(value)

Bases: Exception

d1_cli.impl.format_ids module

Retrieve, cache, manipulate list of known Object Format IDs.

class d1_cli.impl.format_ids.FormatIDs

Bases: object

get(cn_base_url)

format(cn_base_url)

d1_cli.impl.nodes module

Retrieve, cache, manipulate list of known DataONE nodes.

class d1_cli.impl.nodes.Nodes

Bases: object

get(cn_base_url)

format(cn_base_url)

d1_cli.impl.operation_executer module

Execute queued write operations.

class d1_cli.impl.operation_executer.OperationExecuter

Bases: object

execute(operation)

d1_cli.impl.operation_formatter module

Pretty print an operation.

class d1_cli.impl.operation_formatter.OperationFormatter

Bases: object

Print an operation according to the template.

The template contains all parameters that can be in any of the operations and determines the relative position of each parameter that is present in the operation.

print_operation(operation)

d1_cli.impl.operation_maker module

Put together all the information required for executing a given operation.

class d1_cli.impl.operation_maker.OperationMaker(session)

Bases: object

create(pid, path, format_id=None)

update(pid, path, pid_new, format_id=None)

create_package(pids)

archive(pid)

update_access_policy(pid)

update_replication_policy(pid)

d1_cli.impl.operation_queue module

Hold a queue of operations and perform commands on the queue.

class d1_cli.impl.operation_queue.OperationQueue(session)

Bases: object

append(operation)

display()

edit()

execute()

clear()

d1_cli.impl.operation_validator module

Sanity checking of the values which are required by a given operation.

class d1_cli.impl.operation_validator.OperationValidator

Bases: object

assert_valid(operation)

assert_valid_create(operation)

assert_valid_update(operation)

assert_valid_create_package(operation)

assert_valid_archive(operation)

assert_valid_update_access_policy(operation)

assert_valid_update_replication_policy(operation)

d1_cli.impl.replication_policy module

Create and manipulate replication policies.

class d1_cli.impl.replication_policy.ReplicationPolicy

Bases: object

clear()

get_preferred()

get_blocked()

add_preferred(mns)

repremove(mns)

add_blocked(mns)

set_replication_allowed(replication_allowed)

get_replication_allowed()

get_number_of_replicas()

set_number_of_replicas(number_of_replicas)

print_replication_policy()

d1_cli.impl.session module

Hold and manipulate session variables.

class d1_cli.impl.session.Session(nodes, format_ids)

Bases: object

reset()

get(variable)

get_access_control()

get_replication_policy()

set(variable, value)

set_with_conversion(variable, value_string)

Convert user supplied string to Python type.

Lets user use values such as True, False and integers. All variables can be set to None, regardless of type. Handle the case where a string is typed by the user and is not quoted, as a string literal.

print_variable(variable)

print_single_variable(variable)

print_all_variables()

load(pickle_file_path=None, suppress_error=False)

save(pickle_file_path=None, suppress_error=False)

is_verbose()

get_default_pickle_file_path()

d1_cli.impl.system_metadata module

Create System Metadata documents based on session variables.

class d1_cli.impl.system_metadata.SystemMetadataCreator

Bases: object

create_system_metadata(operation)

create_system_metadata_for_update(operation)

create_system_metadata_for_package(resource_map, create_package_operation)

create_access_policy(operation)

create_replication_policy(operation)

d1_cli.impl.util module

Utilities shared between components of the DataONE Command Line.

d1_cli.impl.util.confirm(prompt, default='no', allow_blank=False)

d1_cli.impl.util.output(file_like_object, path, verbose=False)

Display or save file like object.

d1_cli.impl.util.assert_file_exists(path)

d1_cli.impl.util.copy_file_like_object_to_file(file_like_object, path)

d1_cli.impl.util.copy_requests_stream_to_file(response, path)

d1_cli.impl.util.print_debug(msg)

d1_cli.impl.util.print_error(msg)

d1_cli.impl.util.print_warn(msg)

d1_cli.impl.util.print_info(msg)