Python

Python is an interpreted high-level general-purpose programming language. This Agent allows running a Python file in a Data Stream using either CPU-only or GPU-accelerated processing.

GPU appears as a configuration option if detected on the system where the Stream Host is installed.

This Agent can run on both Windows and Linux Stream Hosts, using the 'Python DLL Path' property to point to the Python executable.

  • For Linux, the 'Python DLL Path' property should point to the .so file.

  • For Windows, the 'Python DLL Path' property should point to the .dll file.

Details of examples and the configuration can be found in the How to Use section.

Note: Do not use v1.61. It was packaged incorrectly, which caused side effects to other Agents in Data Streams running on the same Stream Host.

Upgrade to v1.65 at your earliest convenience, noting the updated PYTHONHOME value if you're using environment variables.

CPU-Only Pre-requisites

The following is necessary to use this Agent with CPU-Only processing:

  • Python 3 must be installed on the system where the Stream Host is installed.

    • Minimum version 3.7, up to 3.12 is supported

  • If you've ticked Use Environment Variables on the Agent's configuration:

    1. Add these Environment Variables:

      • PYTHONHOME = C:\Python312 (the location of the standard python libraries)

      • PYTHONPATH = %PYTHONHOME%;%PYTHONHOME%\Lib\site-packages;%PYTHONHOME%\Lib;%PYTHONHOME%\DLLs; (the location of the python modules e.g. site-packages)

      • PYTHONNET_PYDLL = C:\Python312\Lib\Python312.dll (the path to the python .dll or .so file)

    2. Add PYTHONHOME to the system Path Variable

GPU-Accelerated Pre-requisites

The following is necessary to use this Agent with GPU-accelerated processing:

  • CUDA-supported GPU must be installed on the system where the Stream Host is installed.

  • Configure Numba and the CUDA Toolkit in your environment. Python doesn't have built-in support for installing these modules, you can utilize tools like PIP or other platforms like Anaconda or Miniconda for the installation process.

  • If you've ticked Use Environment Variables on the Agent's configuration:

    1. Add an Environment Variable, PYTHONHOME, with the location of the Python modules

    2. Add PYTHONHOME to the system Path Variable

Below is an example of how to configure Numba and the CUDA Toolkit using either the Anaconda or Miniconda platform:

  1. Open Anaconda Prompt.

    • Skip to step 4 if you want to use the Anaconda base environment with the latest version of Python bundled.

  2. Create an anaconda environment with the desired name and Python version, for example:

    conda create --name myenv python=3.12
  3. Activate the environment:

    conda activate myenv
  4. Install Numba and CUDA Toolkit:

    conda install numba & conda install cudatoolkit
  5. If you've ticked Use Environment Variables on the Agent's configuration, add this Environment Variable pointing to the new environment:

    • PYTHONHOME = C:\anaconda3 or C:\anaconda3\envs\{env name}

Current Version

Please contact XMPro if you're looking for an older version of this Agent.

Release Notes

v1.65, 08 Nov 2023

  • Upgraded to the latest Python3.dll

  • Python versions 3.7 to 3.12 are now supported

    • If you're using environment variables, note the updated PYTHONHOME value

  • The GPU Accelerated option is only available when a supported GPU device is detected on the Stream Host server

  • Added error handling for mismatch configuration of Accelerator against Stream Host server

v1.64, 04 Oct 2023

  • Support added for GPU processing

    • The option is available if GPU capability is detected

    • Refer to Anaconda 3 and the other pre-requisites here

  • Fixed error when using server variables

v1.63, 16 Aug 2023

  • Replaced Python version field with Python DLL path to support Linux systems

  • Added support for server variables to Python Paths

  • Updated the framework version to .NET Standard 2.1

  • Updated the XMPro.IoT.Framework version to v4.1.0

  • Runs in isolated mode

v1.62, 10 Aug 2023

  • Allow the full path to the Python DLL to be configured (manually or via environment variables)

  • Added help text to the path configuration

  • Removed the need to specify the Python version

  • Updated the Python.Net dependency to v3.0.1

2022

v1.61, 08 Aug 2022

Note: Do not use this version. It was packaged incorrectly, which caused side effects to other agents in the data streams running on the same Stream Host.

  • Upgraded to the latest Python3.dll

  • Python versions 3.7 to 3.10 are now supported, previously only 3.7.4

2021

v1.51, 20 Aug 2021

  • Saving the global variable values to cache after executing the script

2020

v1.5, 13 Jul 2020

  • Added check for using environment variables

v1.4, 18 May 2020

  • Added error handling

v1.3, 21 Apr 2020

  • Converted python agent to the original implementation

v1.2, 26 Mar 2020

  • Added global variable setting

v1.1, 23 Mar 2020

  • Added quotes around input data string

v1.0, 12 Mar 2020

  • Initial Release

Last updated