Functions

There are numerous generations options that can be set in order to change function bindings options.

See options.py: all the function related options begin wth fn_ or fn_params (when they deal with function parameters)

Exclude functions and/or params¶

Extract from options.py, showing the related options:

    ################################################################################
    #    <functions and method adaptations>
    ################################################################################

    # Exclude certain functions and methods by a regex on their name
    fn_exclude_by_name__regex: str = ""

    # Exclude certain functions and methods by a regex on any of their parameter type and/or return type
    # (those should be decorated type)
    # For example:
    #     options.fn_exclude_by_param_type__regex = "^char\s*$|^unsigned\s+char$|Callback$"
    # would exclude all functions having params of type "char *", "unsigned char", "xxxCallback"
    #
    # Note: this is distinct from `fn_params_exclude_types__regex` which removes params
    # from the function signature, but not the function itself.
    fn_exclude_by_param_type__regex: str = ""

    # ------------------------------------------------------------------------------
    # Exclude some params by name or type
    # ------------------------------------------------------------------------------
    # Remove some params from the python published interface. A param can only be removed if it has a default value
    # in the C++ signature
    fn_params_exclude_names__regex: str = ""
    fn_params_exclude_types__regex: str = ""

As an example, let’s consider the code below, where we would want to:

• exclude all functions beginning with “priv_”

• exclude a function parameter if its type name starts with “Private”

By default the generated code will be:

But we can set some options to change this.

In the generated code below, look closely at the C++ binding code: you will see that it takes steps to generate a default value for the parameter of type PrivateOptions

Return policy¶

See relevant doc from pybind11:

Python and C++ use fundamentally different ways of managing the memory and lifetime of objects managed by them. This can lead to issues when creating bindings for functions that return a non-trivial type. Just by looking at the type information, it is not clear whether Python should take charge of the returned value and eventually free its resources, or if this is handled on the C++ side. For this reason, pybind11 provides a several return value policy annotations that can be passed to the module_::def() and class_::def() functions. The default policy is return_value_policy::automatic.

See relevant doc from nanobind:

nanobind provides several return value policy annotations that can be passed to module_::def(), class_::def(), and cpp_function()...

return_value_policy::reference¶

In the C++ code below, let’s suppose that C++ is responsible for handling the destruction of the values returned by MakeWidget and MakeFoo: we do not want python to call the destructor automatically.

In that case, we can set options.fn_return_force_policy_reference_for_pointers__regex and/or options.fn_return_force_policy_reference_for_references__regex, and the generated pydef binding code, will set the correct return value policy.

Custom return value policy¶

If you annotate the function declaration with return_value_policy::..., then the generator will use this information:

Handle mutable default param values¶

See “Why are default values shared between objects?” in the Python FAQ

There is a common pitfall in Python when using mutable default values in function signatures: if the default value is a mutable object, then it is shared between all calls to the function. This is because the default value is evaluated only once, when the function is defined, and not each time the function is called.

The Python code below shows this issue:

This is fundamentally different from C++ default arguments, where the default value is evaluated each time the function is called.

For bound C++ functions, in most cases the default value still be reevaluated at each call. However, this is not guaranteed, especially when using nanobind!

In order to handle this, litgen provides a set of options that can be set to change the behavior of the generated code. By default,those options are disabled, and the default parameters values might be shared between calls (but your mileage may vary).

Recommended settings for nanobind:

• options.fn_params_adapt_mutable_param_with_default_value__to_autogenerated_named_ctor = True

• options.fn_params_adapt_mutable_param_with_default_value__regex = r".*"

• you may call options.use_nanobind() to set these options as well as the library to nanobind

There are a few related options that can be set to change the behavior of the generated code. See the extract from options.py below:

    # ------------------------------------------------------------------------------
    # Make "mutable default parameters" behave like C++ default arguments
    # (i.e. re-evaluate the default value each time the function is called)
    # ------------------------------------------------------------------------------
    # Regex which contains a list of regexes on functions names for which this transformation will be applied.
    # by default, this is disabled (set it to r".*" to enable it for all functions)
    fn_params_adapt_mutable_param_with_default_value__regex: str = r""
    # if True, auto-generated named constructors will adapt mutable default parameters
    fn_params_adapt_mutable_param_with_default_value__to_autogenerated_named_ctor: bool = False
    # if True, a comment will be added in the stub file to explain the behavior
    fn_params_adapt_mutable_param_with_default_value__add_comment: bool = True
    # fn_params_adapt_mutable_param_with_default_value__fn_is_known_immutable_type
    # may contain a user defined function that will determine if a type is considered immutable in python based on its name.
    # By default, all the types below are considered immutable in python:
    #     "int|float|double|bool|char|unsigned char|std::string|..."
    fn_params_adapt_mutable_param_with_default_value__fn_is_known_immutable_type: Callable[[str], bool] | None = None
    # Same as above, but for values
    fn_params_adapt_mutable_param_with_default_value__fn_is_known_immutable_value: Callable[[str], bool] | None = None

If those options are active, litgen will by default wrap the parameter into an Optional[Parameter_type], and then check if the passed value is None. This step will be done for parameters that have a default value which is mutable.

In the example below, use_elems signature in Python becomes use_elems(elems: Optional[List[int]] = None), and the generated code will check if elems is None, and if so, create a new list.

Below is a more advanced example, where we use an inner struct insider another struct: the autogenerated default constructor with named params will use a wrapped Optional type. You can see that the behavior is nicely explained in the generated stub, as an help for the user.

Handle modifiable immutable params¶

Some C++ functions may use a modifiable input/output parameter, for which the corresponding type in python is immutable (e.g. its is a numeric type, or a string).

For example, in the C++ code below, the param inOutFlag is modified by the function.

void SwitchBool(bool* inOutFlag);

In python, a function with the following signature can not change its parameter value, since bool is immutable:

def switch_bool(in_out_v: bool) -> None:
    pass

litgen offers two ways to handle those situations:

• by using boxed types

• by adding the modified value to the function output

Using boxed types¶

You can decide to replace this kind of parameters type by a “Boxed” type: this is a simple class that encapsulates the value, and makes it modifiable.

Look at the example below where a BoxedBool is created:

• its python signature is given in the stub

• its C++ declaration is given in the glue code

• the C++ binding code handle the conversion between bool * and BoxedBool

Adding the modified value to the function output¶

Let’s say that we have a C++ function that modifies a string, and returns a bool that indicates whether it was modified:

bool UserInputString(std::string* inOutStr);

We can ask litgen to add the modified string to the output of the function.

Look at the example below:

• the python function returns a Tuple[bool, str]

• the C++ binding code adds a lambda that does the necessary transformation

C style function params¶

Immutable C array param¶

If a function uses a param whose type is const SomeType[N], then it will be translated automatically, and the C++ binding code will handle the necessary transformations.

Modifiable C array param¶

If a function uses a param whose type is SomeType[N] v, then litgen will understand that any value inside v can be modified, and it will emit code where a C++ signature like this:

void foo(int v[2]);

is transformed into python:

def foo(v_0: BoxedInt, v_1: BoxedInt) -> None:
    pass

C style string list¶

If a pair of function params look like const char * const items[], int item_count, it will be transformed into a python List[str]:

C style variadic string format¶

If a function uses a pair of parameters like char const* const format, ..., then litgen will transform it into a simple python string.

Passing numeric buffers to numpy¶

Simple numeric buffers¶

If a function uses a pair (a more) of parameters which look like (double *values, int count), or (const float* values, int nb) (etc.), then litgen can transform this parameter into a numpy array.

Let’s see an example with this function:

void PlotXY(const float *xValues, const float *yValues, size_t how_many);

We would like it to be published as:

def plot_xy(x_values: np.ndarray, y_values: np.ndarray) -> None:
    pass

We will need to tell litgen:

• Which function are concerned (options.fn_params_replace_buffer_by_array__regex)

• The name of the the “count” param if it is not a standard one (count, nb, etc)

Note: if you look at the pybind11 C++ binding code, you will see that litgen handles the transformation, and ensures that the types are correct.

Template numeric buffers¶

If a template function uses a pair of parameters whose signature looks like (const T* values, int count), then it can be transformed into a numpy array.

In the example below, we would like the following C++ function:

template<typename NumberType> void PlotXY(Color color, const NumberType *xValues, const NumberType *yValues, size_t count);

To be published as:

def plot_xy(color: Color, x_values: np.ndarray, y_values: np.ndarray) -> None:
    pass

For this we need to:

• Set which function names are concerned (options.fn_params_replace_buffer_by_array__regex)

• Optionally, add the name of the template param (options.fn_params_buffer_template_types)

Note: if you look at the generated pybind11 C++ binding code, you will see that it handles all numeric types. This is a very efficient way to transmit numeric buffers of all types to python

Vectorize functions¶

See relevant portion of the pybind11 manual. This feature is not available with nanobind

Within litgen, you can set:

• Which namespaces are candidates for vectorization (options.fn_namespace_vectorize__regex. Set it to r".*" for all namespaces)

• Which function names are candidates for vectorization

• Which optional suffix or prefix will be added to the vectorized functions

Accepting args and kwargs¶

Relevant portion of the pybind11 manual and the nanobind manual

litgen will automatically detect signatures with params which look like (py::args args, const py::kwargs& kwargs) or (nb::args args, const nb::kwargs& kwargs) and adapt the python stub signature accordingly.

Force overload¶

Relevant portion of the pybind11 manual and the nanobind manual

Automatic overload¶

If litgen detect two overload, it will add a call to py::overload_cast automatically:

Manual overload¶

However, in some cases, you might want to add it manually: use options.fn_force_overload__regex

Force usage of a lambda function¶

In some rare cases, the usage of py::overload_cast might not be sufficient to discriminate the overload. In this case, you can tell litgen to disambiguate it via a lambda function. Look at the pybind C++ binding code below: