Python Extensions: Expanding Functionality and Features"

Any code that you write using any compiled language like C, C++, or Java can be integrated or imported into another Python script. This code is considered as an "extension."

A Python extension module is nothing more than a normal C library. On Unix machines, these libraries usually end in .so (for shared object). On Windows machines, you typically see .dll (for dynamically linked library).

Pre-Requisites for Writing Extensions

To start writing your extension, you are going to need the Python header files.

• On Unix machines, this usually requires installing a developer-specific package.

• Windows users get these headers as part of the package when they use the binary Python installer.

Additionally, it is assumed that you have a good knowledge of C or C++ to write any Python Extension using C programming.

First look at a Python Extension

For your first look at a Python extension module, you need to group your code into four parts −

• The header file Python.h.

• The C functions you want to expose as the interface from your module..

• A table mapping the names of your functions as Python developers see them as C functions inside the extension module..

• An initialization function.

The Header File Python.h

You need to include Python.h header file in your C source file, which gives you the access to the internal Python API used to hook your module into the interpreter.

Make sure to include Python.h before any other headers you might need. You need to follow the includes with the functions you want to call from Python.

The C Functions

The signatures of the C implementation of your functions always takes one of the following three forms −

Each one of the preceding declarations returns a Python object. There is no such thing as a void function in Python as there is in C. If you do not want your functions to return a value, return the C equivalent of Python's None value. The Python headers define a macro, Py_RETURN_NONE, that does this for us.

The names of your C functions can be whatever you like as they are never seen outside of the extension module. They are defined as static function.

Your C functions usually are named by combining the Python module and function names together, as shown here −

This is a Python function called func inside the module module. You will be putting pointers to your C functions into the method table for the module that usually comes next in your source code.

The Method Mapping Table

This method table is a simple array of PyMethodDef structures. That structure looks something like this −

Here is the description of the members of this structure −

• ml_name − This is the name of the function as the Python interpreter presents when it is used in Python programs.

• ml_meth − This is the address of a function that has any one of the signatures, described in the previous section.

• ml_flags − This tells the interpreter which of the three signatures ml_meth is using.

  • This flag usually has a value of METH_VARARGS.

  • This flag can be bitwise OR'ed with METH_KEYWORDS if you want to allow keyword arguments into your function.

  • This can also have a value of METH_NOARGS that indicates you do not want to accept any arguments.

• mml_doc − This is the docstring for the function, which could be NULL if you do not feel like writing one.

This table needs to be terminated with a sentinel that consists of NULL and 0 values for the appropriate members.

Example

For the above-defined function, we have the following method mapping table −

The Initialization Function

The last part of your extension module is the initialization function. This function is called by the Python interpreter when the module is loaded. It is required that the function be named initModule, where Module is the name of the module.

The initialization function needs to be exported from the library you will be building. The Python headers define PyMODINIT_FUNC to include the appropriate incantations for that to happen for the particular environment in which we are compiling. All you have to do is use it when defining the function.

Your C initialization function generally has the following overall structure −

Here is the description of Py_InitModule3 function −

• func − This is the function to be exported.

• module_methods − This is the mapping table name defined above.

• docstring − This is the comment you want to give in your extension.

Putting all this together, it looks like the following −

Example

A simple example that makes use of all the above concepts −

Here the Py_BuildValue function is used to build a Python value. Save above code in hello.c file. We would see how to compile and install this module to be called from Python script.

Building and Installing Extensions

The distutils package makes it very easy to distribute Python modules, both pure Python and extension modules, in a standard way. Modules are distributed in the source form, built and installed via a setup script usually called setup.pyas.

For the above module, you need to prepare the following setup.py script −

Now, use the following command, which would perform all needed compilation and linking steps, with the right compiler and linker commands and flags, and copies the resulting dynamic library into an appropriate directory −

On Unix-based systems, you will most likely need to run this command as root in order to have permissions to write to the site-packages directory. This usually is not a problem on Windows.

Importing Extensions

Once you install your extensions, you would be able to import and call that extension in your Python script as follows −

This would produce the following output −

Hello, Python extensions!!

Passing Function Parameters

As you will most likely want to define functions that accept arguments, you can use one of the other signatures for your C functions. For example, the following function, that accepts some number of parameters, would be defined like this −

The method table containing an entry for the new function would look like this −

You can use the API PyArg_ParseTuple function to extract the arguments from the one PyObject pointer passed into your C function.

The first argument to PyArg_ParseTuple is the args argument. This is the object you will be parsing. The second argument is a format string describing the arguments as you expect them to appear. Each argument is represented by one or more characters in the format string as follows.

Compiling the new version of your module and importing it enables you to invoke the new function with any number of arguments of any type −

You can probably come up with even more variations.

The PyArg_ParseTuple Function

re is the standard signature for the PyArg_ParseTuple function −

This function returns 0 for errors, and a value not equal to 0 for success. Tuple is the PyObject* that was the C function's second argument. Here format is a C string that describes mandatory and optional arguments.

Here is a list of format codes for the PyArg_ParseTuple function −

Returning Values

Py_BuildValue takes in a format string much like PyArg_ParseTuple does. Instead of passing in the addresses of the values you are building, you pass in the actual values. Here is an example showing how to implement an add function.

This is what it would look like if implemented in Python −

You can return two values from your function as follows. This would be captured using a list in Python.

This is what it would look like if implemented in Python −

The Py_BuildValue Function

Here is the standard signature for Py_BuildValue function −

Here format is a C string that describes the Python object to build. The following arguments of Py_BuildValue are C values from which the result is built. ThePyObject* result is a new reference.

The following table lists the commonly used code strings, of which zero or more are joined into a string format.

Code {...} builds dictionaries from an even number of C values, alternately keys and values. For example, Py_BuildValue("{issi}",23,"zig","zag",42) returns a dictionary like Python's {23:'zig','zag':42}

The End! should you have any inquiries, we encourage you to reach out to the Vercaa Support Center without hesitation.