TINC Object classes¶

Parameter Classes¶

class tinc.parameter.Parameter(tinc_id: str, group=None, minimum: float = -99999.0, maximum: float = 99999.0, default_value: float = 0.0, tinc_client=None)¶

Bases: TincObject

A parameter space dimension that describes the possible values for the parameter and holds a ‘current’ value

Parameters

tinc_id – the name that identifies the parameter
group – the name of the group the parameter belongs to
minimum – the minimum value for the parameter. If the value is set to less than this, it is set to minimum
maximum – the maximum value for the parameter. If the value is set to more than this, it is set to maximum
default_value – The parameter’s default value
tinc_client – The tinc.tinc_client.TincClient the parameter belongs to. This should be left as None when calling directly.

clear_callbacks()¶: Remove all callbacks registered with object

get_current_id()¶

get_current_ids()¶

get_current_index()¶

get_ids_for_value(value)¶

get_osc_address()¶

get_space_stride()¶

get_values(remove_duplicates=True)¶

property ids¶

A list of string values that correspond to each parameter value in the parameter space.

This can be used to map values to strings or to directory structures.

interactive_control()¶: Return an interactive control widget for jupyter notebook. Requires ipywidgets.

interactive_widget()¶: Return an interactive widget for jupyter notebook. Requires ipywidgets.

property maximum¶: The maximum the parameter can take.

property minimum¶: The minimum the parameter can take.

next()¶: Move to the next value (by index) in the values list. If values list is empty or currently at the last item, there is no effect.

previous()¶: Move to the previous value (by index) in the values list. If values list is empty or currently at the first item, there is no effect.

register_callback(f, synchronous=True)¶

Register a function that will be called whenever the parameter’s value changes. Note that what is registered is the function object, so if you redefine the function after registering, the registered function will still use the previously registered function.

Parameters

f – Function to be called.
synchronous – if False, the callback will be called from a separate thread and will not block

register_callback_async(f)¶

remove_callback(f)¶

set_at(index)¶

set_ids(ids)¶

set_maximum(maximum)¶

set_minimum(minimum)¶

set_space_representation_type(space_representation_type)¶

set_value(value)¶: Set the current value for the parameter

set_values(values)¶

sort()¶

property space_representation_type¶

property value¶

The current value for the parameter

Returns: the current value

property values¶: A list of the possible values the parameter can take.

class tinc.parameter.ParameterBool(tinc_id: str, group: str = '', default_value=False, tinc_client=None)¶: Bases: Parameter

class tinc.parameter.ParameterChoice(tinc_id: str, group: str = '', minimum: int = 0, maximum: int = 127, default_value: int = 0, tinc_client=None)¶

Bases: Parameter

get_current_elements()¶

interactive_widget()¶: Return an interactive widget for jupyter notebook. Requires ipywidgets.

set_elements(elements)¶

set_value(value)¶: Set the current value for the parameter

class tinc.parameter.ParameterColor(tinc_id: str, group: str = '', default_value=[0, 0, 0, 0], tinc_client=None)¶: Bases: Parameter

class tinc.parameter.ParameterInt(tinc_id: str, group: str = '', minimum: int = 0, maximum: int = 127, default_value: int = 0, tinc_client=None)¶

Bases: Parameter

interactive_widget()¶: Return an interactive widget for jupyter notebook. Requires ipywidgets.

class tinc.parameter.ParameterString(tinc_id: str, group: str = '', default_value: str = '', tinc_client=None)¶

Bases: Parameter

interactive_widget()¶: Return an interactive widget for jupyter notebook. Requires ipywidgets.

set_value(value)¶: Set the current value for the parameter

set_values(values)¶

class tinc.parameter.ParameterVec(tinc_id: str, group: str = '', size=3, tinc_client=None)¶

Bases: Parameter

set_value(value)¶: Set the current value for the parameter

set_values(values)¶

class tinc.parameter.Trigger(tinc_id: str, group: str = '', default_value=False, tinc_client=None)¶

Bases: ParameterBool

set_value(value)¶: Set the current value for the parameter

trigger()¶

class tinc.parameter.parameter_space_representation_types(value)¶

Bases: IntEnum

An enumeration.

ID = 2¶

INDEX = 1¶

VALUE = 0¶

Parameter Space¶

Created on Tue Sep 1 17:13:15 2020

@author: Andres

class tinc.parameter_space.ParameterSpace(tinc_id='', tinc_client=None)¶

Bases: TincObject

The ParameterSpace class contains a set of tinc.parameter.Parameter objects and organizes access to them

Parameter spaces can be linked to specific directories in the file system. This can be useful to locate data according to paramter values. For example when the data is a result of a parameter sweep that generated multiple directories. See set_current_path_template() and generate_relative_run_path() for more information.

clear_cache()¶

create_dimension(name, values=None)¶

Create a parameter/dimension in the parameter space of type tinc.parameter.Parameter.

Parameters

name – Name of the parameter to create
values – (optional) values the parameter can take

Returns

the created parameter

disable_cache()¶

enable_cache(directory='python_cache')¶: Enable caching for processes run through run_process()

get_common_id(dimensions, indeces=None)¶

get_current_relative_path()¶

get_dimension(param_id, group=None)¶

Returns parameter that matches param_id name and group. If group is None, the first match is returned.

Parameters

param_id – Parameter name to match
group – group to match

Returns

The matched parameter or None if no match

get_dimensions()¶

get_parameter(param_id, group=None)¶

get_parameters()¶

get_root_path()¶

is_filesystem_dimension(dimension_name)¶

register_dimension(param)¶: Register a parameter as a dimension for the parameter space

register_dimensions(params)¶: Register a list of parameters as dimensions for the parameter space

register_parameter(param)¶

register_parameters(params)¶

remove_dimension(param)¶

remove_parameter(param)¶

resolve_template(path_template, index_map=None)¶: Resolve a path template according to the current parameter values.

run_process(function, args=None, dependencies=[], force_recompute=False)¶

running_paths(fixed_dimensions=[])¶

set_current_path_template(path_template)¶: Set the current path template for the paramter space to map current values to a location in the filesystem. You can use %% to delimit dimension names, e.g. “value_%%ParameterValue%%” where %%ParameterValue%% will be replaced by the current value (in the correct representation as ID, VALUE or INDEX) of the dimension whose id is “ParameterValue”. You can specify a different representation than the one set for the ParameterSpaceDimension by adding it following a ‘:’. For example: “value_%%ParameterValue:INDEX%%” will replace “%%ParameterValue:INDEX%%” with the current index for ParameterValue. For parameters that have mutiple ids for the same value, you can specify any muber of parameters separated by commas. the function get_common_id() will be called. For example, for %%param1,param2%% the common id for the their current values will be inserted. Any representation type (ID, VALUE, INDEX) is ignored, as only ids are used. Using this method can be useful as it can avoid having to define a custom generateRelativeRunPath() function

set_root_path(root_path)¶

stop_sweep()¶

sweep(function, params=None, dependencies=[], force_recompute=False, force_values=False)¶

Sweep the parameter space running the function for all value combinations

Parameters

function – Function to run for each parameter space sample
params – If set, only the parameters provided will be sweept. The rest of the parameters in the ParameterSpace will be kept constant at their current value
dependencies –
force_recompute – Always recompute function even if there is cache available
force_values – Set all parameters’ value every sample. This will trigger parameter callbacks. This is required when you query the parameter values within the function rather than taking the parameter values as function parameters

sweep_async(function, args=None, dependencies=[], force_recompute=False, force_values=False, num_threads=1)¶

CacheManager¶

Created on Thu Oct 15 14:32:03 2020

@author: Andres

class tinc.cachemanager.CacheEntry(timestamp_start, timestamp_end, files, user_info, source_info, cache_hits, stale)¶

Bases: NamedTuple

cache_hits: int¶: Alias for field number 5

files: List[str]¶: Alias for field number 2

source_info: SourceInfo¶: Alias for field number 4

stale: bool¶: Alias for field number 6

timestamp_end: str¶: Alias for field number 1

timestamp_start: str¶: Alias for field number 0

user_info: UserInfo¶: Alias for field number 3

class tinc.cachemanager.CacheManager(directory='python_cache', metadata_file='tinc_cache.json')¶

Bases: object

append_entry(entry)¶

cache_directory()¶

clear_cache()¶

dump()¶

entries(count=0)¶

find_cache(source_info, verify_hash=True)¶

update_from_disk()¶

write_to_disk()¶

class tinc.cachemanager.FileDependency(file, modified, size, hash)¶

Bases: NamedTuple

file: DistributedPath¶: Alias for field number 0

hash: str¶: Alias for field number 3

modified: str¶: Alias for field number 1

size: int¶: Alias for field number 2

class tinc.cachemanager.SourceArgument(id, value)¶

Bases: NamedTuple

id: str¶: Alias for field number 0

value: VariantValue¶: Alias for field number 1

class tinc.cachemanager.SourceInfo(type, tinc_id, command_line_arguments, working_path_rel, working_path_root, arguments, dependencies, file_dependencies)¶

Bases: NamedTuple

arguments: List[SourceArgument]¶: Alias for field number 5

command_line_arguments: str¶: Alias for field number 2

dependencies: List[SourceArgument]¶: Alias for field number 6

file_dependencies: List[FileDependency]¶: Alias for field number 7

tinc_id: str¶: Alias for field number 1

type: str¶: Alias for field number 0

working_path_rel: str¶: Alias for field number 3

working_path_root: str¶: Alias for field number 4

class tinc.cachemanager.UserInfo(user_name, user_hash, ip, port, server)¶

Bases: NamedTuple

ip: str¶: Alias for field number 2

port: int¶: Alias for field number 3

server: bool¶: Alias for field number 4

user_hash: str¶: Alias for field number 1

user_name: str¶: Alias for field number 0

class tinc.cachemanager.VariantValue(nctype, value)¶

Bases: NamedTuple

nctype: VariantType¶: Alias for field number 0

value: any¶: Alias for field number 1

tinc.cachemanager.arguments_from_param_list(param_list)¶

DataPool¶

Created on Tue Sep 1 15:47:46 2020

@author: Andres

class tinc.datapool.DataPool(tinc_id='_', parameter_space=None, slice_cache_dir='./', tinc_client=None)¶

Bases: TincObject

The DataPool class can unify through a single interface homogeneous data spread across the filesystem.

The DataPool class gathers data files across directories that span a parameter space. The parameter space and its current values determine where the files in the data pool are found. This class is useful to manage data files that are the result of parameter sweeps, generating the same type of file in different directories, where each directory represents a sample of the parameter space.

This base class provides the basic functionality and interface, but in practice you will want to use one of the derived classes that matches your data, like tinc.datapool.DataPoolJson.

Parameters

tinc_id – the name that identifies the data pool
parameter_space – The tinc.parameter_space.ParameterSpace for the data
slice_cache_dir – The folder where slices from the data pool are stored
tinc_client – The tinc.tinc_client.TincClient the parameter belongs to. This should be left as None when calling directly.

clear_registered_data_files()¶

create_data_slice(field, slice_dimensions, override_value=None)¶

Write data slice to file.

The slice will be created as a NetCDF4 file with a single variable called “data” that spans a dimension “values”. The result will be a multi- dimensional slice containing the values of the “field” across all values for slice_dimensions (that must be registered in the parameter space).

Parameters

field – Name of the field to extract
slice_dimensions – The name of a dimension or a list of dimension names
override_value – (optional) Override current values in the parameter space. Should contain a map of names to values

Returns

Filename to the new data slice

get_current_files()¶

get_parameter_space()¶: Get the tinc.parameter_space.ParameterSpace that controls this data pool

get_registered_data_files()¶

get_slice(field, slice_dimensions, override_value=None)¶

Get data slice from the data pool.

Called readDataSlice() in the C++ API.

Parameters

field – Name of the field to extract
slice_dimensions – The name of a dimension or a list of dimension names
override_value – (optional) Override current values in the parameter space. Should contain a map of names to values

get_slice_file(field, slice_dimensions)¶

list_fields(verify_consistency=False)¶: List fields available in the data files for this data pool

register_data_file(filename, dimension_in_file)¶

Register with the DataPool the filename and the dimension from the parameter space that is contained within the file.

These file should be located relative to the current directory for the parameter space. It should be present in all data paths for the parameter space.

Parameters

filename – The relative file name
dimension_in_file – string with the dimension file name

class tinc.datapool.DataPoolJson(tinc_id='_', parameter_space=None, slice_cache_dir='./', tinc_client=None)¶

Bases: DataPool

DataPool to read JSON files

DiskBuffer classes¶

class tinc.disk_buffer.DiskBuffer(tinc_id, base_filename, rel_path='', root_path='', tinc_client=None)¶

Bases: TincObject

Base class for all disk buffers.

A DiskBuffer is a filesystem based buffer for distributed applications that allows notifying other nodes of changes. Although data is shared through the file system, the interface to disk buffer objects is presented as in memory data.

The DiskBuffer subclasses implement specific data format decoding: tinc.diskbuffer.DiskBufferJson, tinc.diskbuffer.DiskBufferImage

All children class must implement the data getter and setter functions and _parse_file

cleanup_round_robin_files()¶

property data¶: Read the data in the disk buffer. This data is cached in memory until changed.

done_writing_file(filename: str = '', notify=True)¶

enable_round_robin(cache_size=2, clear_locks: bool = True)¶

get_base_filename()¶

get_current_filename()¶

get_filename_for_writing(timeout_secs=0)¶

get_full_path()¶

get_relative_path()¶

get_root_path()¶

load_data(filename, notify=True)¶

lock(outname)¶

register_update_callback(f)¶

set_base_filename(filename)¶

set_relative_path(rel_path)¶

set_root_path(root_path)¶

unlock(outname)¶

use_file_lock(use: bool = True, clear_locks: bool = True)¶

class tinc.disk_buffer.DiskBufferBinary(tinc_id, base_filename, rel_path='', root_path='', tinc_client=None)¶

Bases: DiskBuffer

property data¶: Read the data in the disk buffer. This data is cached in memory until changed.

class tinc.disk_buffer.DiskBufferImage(tinc_id, base_filename, path, root_path='', tinc_client=None)¶

Bases: DiskBuffer

property data¶: Read the data in the disk buffer. This data is cached in memory until changed.

interactive_widget()¶

write_pixels(pixels)¶

class tinc.disk_buffer.DiskBufferJson(tinc_id, base_filename, rel_path='', root_path='', tinc_client=None)¶

Bases: DiskBuffer

property data¶: Read the data in the disk buffer. This data is cached in memory until changed.

class tinc.disk_buffer.DiskBufferNetCDFData(tinc_id, base_filename, rel_path='', root_path='', tinc_client=None)¶

Bases: DiskBuffer

property data¶: Read the data in the disk buffer. This data is cached in memory until changed.

set_attributes(attrs)¶

write_from_array(array, filename=None, attributes={})¶

class tinc.disk_buffer.DiskBufferText(tinc_id, base_filename, rel_path='', root_path='', tinc_client=None)¶

Bases: DiskBuffer

property data¶: Read the data in the disk buffer. This data is cached in memory until changed.

interactive_widget()¶

Processor classes¶

class tinc.processor.ComputationChain(tinc_id='_', input_dir='', input_files=[], output_dir='', output_files=[], running_dir='')¶: Bases: Processor

class tinc.processor.Processor(tinc_id, input_dir='', input_files=[], output_dir='', output_files=[], running_dir='')¶

Bases: TincObject

configuration = {}¶

debug = False¶

done_callback = None¶

Base functionality and interface for TINC Processors.

You will want to instantiate specific processors: tinc.processor.ProcessorScript or tinc.processor.ProcessorScriptDocker.

enabled = True¶

ignore_fail = False¶

input_dir = ''¶

input_files = ''¶

output_dir = ''¶

output_files = ''¶

prepare = None¶

process(force_recompute=False)¶

register_parameter(dim, triggers_processor=True)¶

running_dir = ''¶

set_input_files(infiles)¶

set_output_files(outfiles)¶

start_callback = None¶

class tinc.processor.ProcessorCpp(name='_', input_dir='', input_files=[], output_dir='', output_files=[], running_dir='')¶: Bases: Processor

class tinc.processor.ProcessorScript(tinc_id='_', input_dir='', input_files=[], output_dir='', output_files=[], running_dir='')¶

Bases: Processor

capture_output(capture=True)¶

command: str = ''¶

command_line_flag_template = ''¶: A tinc.processor.Processor that executes a system process.

process(force_recompute=False)¶

sanitize_name()¶

script_name: str = ''¶

set_argument_template(template)¶

set_command_line(cmd_line)¶

class tinc.processor.ProcessorScriptDocker(tinc_id='_', input_dir='', input_files=[], output_dir='', output_files=[], running_dir='')¶

Bases: ProcessorScript

container_id = ''¶

find_container_id(name)¶

path_map = {}¶: A tinc.processor.Processor that executes a system process on a Docker container

set_container_id(container_id)¶

set_path_map(local_path, container_path)¶

Other classes¶

TINC Client¶

class tinc.tinc_client.TincClient(server_addr: str = 'localhost', server_port: int = 34450, auto_connect=True)¶

Bases: object

The TincClient class allows connecting to a TINC server to share parameters and data.

Parameters

server_addr – The IP address for the TINC server to connect to
server_port – The port for the TINC server
auto_connect – If true will connect to the server on creation of a TincClient object. If false, client can connect via the start() function.

barrier(group=0, timeout_sec=0)¶

create_parameter(parameter_type, param_id, group=None, min_value=None, max_value=None, space=None, default_value=None, space_representation_type=None)¶

get_datapool(datapool_id)¶

Get tinc.data_pool.DataPool registered with client.

Parameters: datapool_id – the name of the data pool to match.
Returns: None is there is no match

get_disk_buffer(db_id)¶

Get tinc.disk_buffer.DiskBuffer registered with client.

Parameters: db_id – the name of the disk buffer to match.
Returns: None is there is no match

get_parameter(parameter_id, group=None)¶

Returns a tinc.parameter.Parameter (or similar) with matching name and group that is registered with the TINC client. The parameter could have been registered in the current instance of the client, or on the server.

Parameters

parameter_id – The name of the parameter
group – group name to match. If None, the first parameter_id match is returned

get_parameter_space(ps_id)¶

Get tinc.parameter_space.ParameterSpace registered with client.

Parameters: ps_id – the name of the parameter space to match.
Returns: None is there is no match

get_parameters(group=None)¶

Returns all the parameters registered with the client that match the group

Parameters: group (str, None, optional) – the name of the group to match. If none, all parameters are returned

get_processor(processor_id)¶

Get tinc.processor.Processor registered with client.

Parameters: processor_id – the name of the processor to match.
Returns: None is there is no match

print()¶: Print details about this TincClient object. Prints details of all registered objects.

register_disk_buffer(db)¶

register_parameter(new_param)¶

register_parameter_space(new_ps)¶

register_processor(message)¶

remove_parameter(param_id, group=None)¶

request_data_pools()¶

request_disk_buffers()¶

request_parameter_spaces()¶

request_parameters()¶

request_processors()¶

send_metadata()¶

server_status()¶

start(server_addr='localhost', server_port=34450)¶

Start the TINC client, attempting to connect to the TINC server at the provided address and port

Parameters

server_addr – The IP address for the TINC server to connect to
server_port – The port for the TINC server

stop()¶: Stop the TINC client. Disconnects from TINC server

synchronize()¶

wait_for_server_available(timeout=3000.0)¶

After client is started this function can be called to ensure server has accepted connection.

Parameters: timeout – timeout in seconds

exception tinc.tinc_client.TincTimeout(arg)¶: Bases: ValueError

TINC Server¶

class tinc.tinc_server.TincServer¶

Bases: TincClient

set_root_map_entry(server_path, client_path, host='')¶

start()¶

Start the TINC client, attempting to connect to the TINC server at the provided address and port

Parameters

server_addr – The IP address for the TINC server to connect to
server_port – The port for the TINC server

stop()¶: Stop the TINC client. Disconnects from TINC server

TincObject¶

Created on Tue Dec 8 11:08:40 2020

@author: Andres

class tinc.tinc_object.TincObject(tinc_id='')¶: Bases: object

tinc.tinc_object.TincVersion()¶

PresetHandler¶

Created on July 19 2021

@author: Andres

class tinc.preset_handler.PresetHandler(root_dir='presets/')¶

Bases: object

Manage preset read and write for parameters

Parameters: root_dir – Path where presets are stored

available_presets()¶: Return a list of currently available presets.

interactive_store_widget()¶

interactive_widget()¶

recall_preset(name)¶: Recall parameter values from preset.

recall_preset_index(index)¶

register_parameter(parameter)¶

Register a parameter with the preset handler

Parameters that have been registered will be modified when presets are recalled and will provide their current value when presets are stored.

rename_preset(name, new_name)¶

set_preset_map(preset_map_name)¶

Set the current map for indeces to preset names

The maps are stored in the file system with extension .presetMap

set_sub_dir(subdir)¶

store_preset(name)¶

Store the current values in a preset.

The preset name will determine the file name. See tinc.preset_handler.PresetHandler.store_preset_index()

Will overwrite preset if name exists.

Parameters: name – Name for the preset.

store_preset_index(index, name='', overwrite=True)¶: Store preset by index

TincBridge¶

class tinc.tinc_bridge.TincBridge(executable=None, running_path='./')¶

Bases: TincClient

launch()¶

quit()¶

Tools¶

tinc.parameter_space_tools module¶

Created on Mon Oct 4 17:13:15 2021

@author: Andres

tinc.parameter_space_tools.create_datapool_from_output(data_root, output_file, read_file_func=None, ignore_params=[], depth=3, dp_name='dp', ps_name='ps', debug=False)¶

tinc.parameter_space_tools.extract_parameter_space_data(data_dir, config_file, parameter_start_key, parameter_end_key, parameter_increment_key, debug=False)¶

tinc.parameter_space_tools.extract_parameter_space_from_output(data_root, output_file, read_file_func, ignore_params=[], depth=3, ps_name='ps', debug=False)¶

tinc.parameter_space_tools.make_parameter_space(data_dir, config_file, parameter_start_key, parameter_end_key, parameter_increment_key, ps_name=None, debug=False)¶

tinc.parameter_space_tools.make_parameter_space_from_dict(merged_space, data_dir=None, ps_name=None, debug=False)¶

tinc.process_args module¶

Created on Thu Apr 25 17:15:01 2019

@author: andres

class tinc.process_args.TincArgumentParser(**kwargs)¶

Bases: ArgumentParser

get_args()¶

tinc.utils module¶

Data structures¶

tinc.distributed_path module¶

Created on Mon Aug 9 15:47:18 2021

@author: Andres

class tinc.distributed_path.DistributedPath(filename='', relative_path='', root_path='', protocol_id='')¶

Bases: object

get_full_path()¶

set_paths(rel_path, root_path)¶

tinc.message module¶

Created on Wed Sep 2 10:41:37 2020

@author: Andres

class tinc.message.Message(data=None)¶

Bases: object

append(data)¶

consume(num_bytes=-1)¶

empty()¶

get_byte()¶

get_float()¶

get_int32()¶

get_string()¶

get_uint16()¶

get_uint32()¶

get_uint64()¶

get_variant()¶

get_vector_string()¶

insert_as_uint16(int_value)¶

insert_as_uint32(int_value)¶

insert_string(string)¶

insert_vector_string(stringlist)¶

remaining_bytes()¶

tinc.variant module¶

class tinc.variant.VariantType(value)¶

Bases: IntEnum

An enumeration.

VARIANT_BOOL = 99¶

VARIANT_CHAR = 2¶

VARIANT_DOUBLE = 6¶

VARIANT_FLOAT = 5¶

VARIANT_INT16 = 3¶

VARIANT_INT32 = 4¶

VARIANT_INT64 = 10¶

VARIANT_INT8 = 1¶

VARIANT_MAX_ATOMIC_TYPE = 12¶

VARIANT_NONE = 0¶

VARIANT_STRING = 12¶

VARIANT_UINT16 = 8¶

VARIANT_UINT32 = 9¶

VARIANT_UINT64 = 11¶

VARIANT_UINT8 = 7¶

TINC Object classes¶

Parameter Classes¶

Parameter Space¶

CacheManager¶

DataPool¶

DiskBuffer classes¶

Processor classes¶

Other classes¶

TINC Client¶

TINC Server¶

TincObject¶

PresetHandler¶

TincBridge¶

Tools¶

tinc.parameter_space_tools module¶

tinc.process_args module¶

tinc.utils module¶

Data structures¶

tinc.distributed_path module¶

tinc.message module¶

tinc.variant module¶

Table of Contents

This Page