API#

This is the full API reference for all public classes and functions.

Decorators#

otf.function(f=None, /, *, strict=True)#

Wraps the decorated function in its own portable environment.

The function be will be compiled in its own module and will only have access to globals passed in via the environment() decorator:

@function
@environment(i=0)
def counter() -> int:
  "A counter that increments by 1 every time it's called"
  global i
  i+=1
  return i

By default the decorator will vet the function for issues like using globals that aren’t declared in its environment:

>>> @function
... def counter() -> int:
...  global i
...  i+=1
...  return i
Traceback (most recent call last):
...
SyntaxError: variable 'i' not found in the environment

This can be turned off with the strict argument:

@function(strict=False)
def counter() -> int:
   ...
Parameters

strict – do not check the wrapped function.

Return type

Closure

otf.environment(**kwargs)#

Attach an environment to the function.

All keyword arguments will be declared as variables in the function’s globals.

Runtime Classes#

class otf.Closure(environment, target)#

A callable otf function

class otf.Environment(**kwargs)#

An otf environment contains functions and values.

function(fn=None, /, *, lazy=False)#

A decorator to add a function to this environment.

The decorator can be called directly on a function:

@env.function
def f(x: int) -> int:
  ...

Or with arguments:

@env.function(lazy=True)
def f(x: int) -> int:
  ...
workflow(fn)#

Workflow decorator

Turn a function written with async/await into a workflow.

class otf.Suspension(code, *, position, environment, variables, awaiting=None)#

Represents a checkpoint in the execution of a workflow

A suspension captures the code of a workflow, a position in that code and all the local variables. It’s a continuation that can be reified.

property lineno#

The line we are currently awaiting on (set to None if we haven’t entered the body of the function yet).

class otf.Workflow(environment, origin)#
freeze(*args, **kwargs)#

Create a suspension of this workflow on the first line

All the runtime classes support pickle but we also have our own serialization format.

Serialisation#

The otf serialisation format is designed to be read by humans. It is a subset of python’s syntax:

>>> otf.dump_text([1, 2, 3, 4, None])
'[1, 2, 3, 4, None]'

>>> otf.load_text("""
... # You can have comments in the values you load
... {
...    tuple([2, 1]): 5,
...    tuple([3, 1]): 13,
...    tuple([4, 1]): 65533,
... }
... """)
{(2, 1): 5, (3, 1): 13, (4, 1): 65533}

The library is extensible and can handle arbitrary python values.

Shared references:

>>> v = []
>>> otf.dump_text([v, v])
'[[], ref(1)]'

Non finite floats:

>>> otf.dump_text([1., 2., -math.inf])
'[1.0, 2.0, -inf]'

Adding support for new types:

>>> import fractions
...
>>> @otf.register
... def _(fraction: fractions.Fraction):
...   return fractions.Fraction, str(fraction)
...
>>> otf.dump_text(fractions.Fraction(227, 73))
"fractions.Fraction('227/73')"

Serialisation functions and classes#

otf.dump_text(obj, indent=None, width=60, format=None)

Serialise obj

Dumps supports several formats. Let’s take a sample value with a shared reference:

>>> v = {'nan': math.nan, '1_5':[1,2,3,4,5]}
>>> v2 = [v, v]

  • COMPACT means it will all be printed on one line.

    >>> print(dump_text(v2, format = COMPACT))
[{'nan': nan, '1_5': [1, 2, 3, 4, 5]}, ref(10)]

  • PRETTY will use the width and indent argument to pretty print the output.

    >>> print(dump_text(v2, format = PRETTY, width=20))
[
    {
        'nan': nan,
        '1_5': [
            1,
            2,
            3,
            4,
            5
        ]
    },
    ref(10)
]

  • EXECUTABLE will print code that can run in a python environment where the last statement is the value we’re building:

    >>> print(dump_text(v2, format = EXECUTABLE, width=40))
_0 = {
    'nan': float("nan"),
    '1_5': [1, 2, 3, 4, 5]
}

[_0, _0]
Parameters
otf.load_text(s)

Load a value encoded as a string

Parameters

s (str) –

otf.register(function=None, /, *, type=None, pickle=False)

Register a function to use while packing object of a given type.

function is expected to take objects of type T and to return a tuple describing how to recreate the object: a function and serialisable value.

If type is not specified, register() uses the type annotation on the first argument to deduce which type register function for.

If register() is used as a simple decorator (with no arguments) it acts as though the default values for all of it parameters.

Here are three equivalent ways to add support for the complex type:

>>> @register
... def _reduce_complex(c: complex):
...   return complex, (c.real, c.imag)

>>> @register()
... def _reduce_complex(c: complex):
...   return complex, (c.real, c.imag)

>>> @register(type=complex)
... def _reduce_complex(c: complex):
...   return complex, (c.real, c.imag)
Parameters

  • function – The reduction we are registering

  • type – The type we are registering the function for

  • pickle – If set to True, function is registered via copyreg.pickle() to be used in pickle.

class otf.NamedReference(obj)#

Serialisable wrapper around a module, class or function.

NamedReference are serialised by copying the name to the wrapped object:

>>> import math
>>> fl = NamedReference(math.floor)
>>> dump_text(fl)
"otf.NamedReference('math.floor')"

If the wrapped object is callable then the wrapped will pass calls transparently to the object:

>>> fl(12.6)
12

Attribute accesses are also transparently passed through to the wrapped value:

>>> m = NamedReference(math)
>>> m.ceil(5.6)
6

This means that, in most cases, the NamedReference wrapper is transparent. Sometimes you actually need to access the wrapped value:

>>> WrappedComplex = NamedReference(complex)
>>> k = WrappedComplex(5)
>>> isinstance(k, WrappedComplex)
Traceback (most recent call last):
  ...
TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union
>>>

You can use the ~ operator to access the wrapped value:

>>> ~WrappedComplex
<class 'complex'>
>>> isinstance(k, ~WrappedComplex)
True
Parameters

v (class|module|function) – The object that will be wrapped.

Raises

  • ValueError – if obj cannot be reloaded via its name (e.g.: if obj is defined inside a function).

  • TypeError – if obj is not a module, class or function or if obj is a lambda.

Modules#