Explore, execute and debug workflows stored in the Python Workflow Definition (PWD) with the visual programming environment pyironflow.
The Python Workflow Definition is a format for exchanging workflows between workflow engines, including AiiDA, jobflow and pyiron. Each node of a PWD workflow is a Python function, and each edge connects the output of one function to the input of another. This repository adds a graphical perspective to the format. pyironflow is a visual programming environment built on ReactFlow. It runs inside Jupyter and works on top of the pyiron_workflow workflow engine. The three example workflows of the PWD, from a simple arithmetic example to a file-based finite element simulation and a density functional theory calculation, are loaded, run and inspected in pyironflow.
This repository is published as a website at pythonworkflow.github.io/visual-programming-with-pyironflow.
- See the workflow you discuss. New workflows are commonly introduced as data flow diagrams. A visual programming environment shows exactly this diagram, generated directly from the machine-readable workflow definition. You can browse through the workflow and zoom in on individual parts to see which data is passed between the steps.
- Debug the data flow step by step. Instead of running the full workflow, you can run individual nodes together with the nodes they depend on (pull), or run a node and everything downstream of it (push). You can then inspect the inputs, outputs and source code of every step directly in the graphical user interface.
- Share workflows across user groups. Users with less programming experience can run workflows developed by experts through a graphical interface, without writing Python code. This speeds up the transfer of knowledge between user groups.
- Stay interoperable. pyironflow reads and writes the same
workflow.jsonfiles as all other PWD workflow engines. A workflow developed with AiiDA or jobflow can be explored in pyironflow, and a workflow built with pyiron_workflow can be run with any other PWD engine.
Every example is stored in the three files of the Python Workflow Definition: a Python module workflow.py with the Python functions, a workflow.json file with the connections between them, and a conda environment with the software dependencies. Run the following in the directory of one of the examples to load the workflow into pyironflow:
from pyironflow import PyironFlow
from python_workflow_definition.pyiron_workflow import load_workflow_json
PyironFlow([load_workflow_json(file_name="workflow.json")]).guiload_workflow_json() translates the PWD workflow.json into a pyiron_workflow Workflow, and PyironFlow displays a list of workflows, each in its own tab of the graphical user interface. To go the other way, export a workflow defined in pyiron_workflow to the Python Workflow Definition:
from python_workflow_definition.pyiron_workflow import write_workflow_json
write_workflow_json(graph_as_dict=wf.graph_as_dict, file_name="workflow.json")The repository contains the same three example workflows that the Python Workflow Definition implements for other workflow engines. Each notebook first reloads the existing workflow.json in pyironflow and runs it. It then shows how the workflow is defined with pyiron_workflow and exported back to workflow.json.
| Example | Description | Software |
|---|---|---|
| Arithmetic | The smallest PWD workflow, which combines three Python functions (get_prod_and_div, get_sum and get_square). It introduces the format of workflow.json and the main elements of pyironflow. |
Python standard library |
| NFDI4Ing | The NFDI4Ing scientific workflow benchmark. It generates a mesh, solves the Poisson equation with the finite element method, post-processes the solution and compiles a LaTeX paper with the results. This shows how file-based workflows that call external codes in separate conda environments are represented. | gmsh, meshio, FEniCS, ParaView, tectonic |
| Quantum ESPRESSO | The energy-volume curve of fcc aluminium. The cell is first relaxed, then five strained structures are computed independently and the results are collected in a plot. This shows how parallel branches and the PWD helper functions get_dict and get_list appear in pyironflow. |
Quantum ESPRESSO, ASE, OPTIMADE |
.
├── README.md # this overview, also the landing page of the website
├── apt.txt # apt dependencies - libGL (used by Binder and Continuous Integration)
├── environment.yml # conda environment for all examples (used by Binder and Continuous Integration)
├── postBuild # post build script (used by Binder)
├── arithmetic/
│ ├── pyironflow.ipynb # notebook: reload and define the workflow with pyironflow
│ ├── workflow.py # Python functions
│ ├── workflow.json # Python Workflow Definition
│ └── screenshot.png # pyironflow screenshot for the website
├── nfdi/
│ ├── pyironflow.ipynb
│ ├── workflow.py
│ ├── workflow.json
│ ├── screenshot.png
│ └── source/ # input templates, scripts and conda environments of the benchmark
│ ├── envs/ # preprocessing.yaml, processing.yaml, postprocessing.yaml
│ ├── unit_square.geo # geometry for gmsh
│ ├── poisson.py # FEniCS solver for the Poisson equation
│ ├── postprocessing.py # ParaView plot over line
│ ├── prepare_paper_macros.py
│ ├── macros.tex.template
│ └── paper.tex # LaTeX paper template
├── quantum_espresso/
│ ├── pyironflow.ipynb
│ ├── workflow.py
│ ├── workflow.json
│ ├── screenshot.png
│ ├── espresso/pseudo/ # pseudopotential for aluminium
│ └── qe_xml_parser/ # parser for the Quantum ESPRESSO XML output
└── .github/
├── _config.yml # Jupyter Book configuration
├── _toc.yml # Jupyter Book table of contents
└── workflows/ # build (pull requests) and deploy (main) of the website
The fastest way to try the examples is Binder, which builds the conda environment from environment.yml automatically. To run the examples locally, create the same environment with conda:
conda env create -f environment.yml
The environment includes pyironflow, pyiron_workflow and python-workflow-definition, together with the dependencies of the arithmetic and Quantum ESPRESSO examples. Two of the examples need additional steps:
- Quantum ESPRESSO: install the bundled output parser with
pip install ./quantum_espresso/qe_xml_parser. Quantum ESPRESSO looks for pseudopotentials in the directory named by theESPRESSO_PSEUDOenvironment variable, so set it to the absolute path ofquantum_espresso/espresso/pseudo. - NFDI4Ing: the individual steps run in three separate conda environments.
conda_subprocessselects them by name, so create them with the namespreprocessing,processingandpostprocessing:Theconda env create -f nfdi/source/envs/preprocessing.yaml -n preprocessing conda env create -f nfdi/source/envs/processing.yaml -n processing conda env create -f nfdi/source/envs/postprocessing.yaml -n postprocessingsource_directoryinput innfdi/workflow.jsonis an absolute path. Regenerate the file with the second part of the NFDI4Ing notebook before running the workflow on a new computer.
The website is built with Jupyter Book. The notebooks are not executed during the build; the stored outputs are rendered as they are.
- Python Workflow Definition: the workflow exchange format, with the same examples implemented for AiiDA, jobflow, pyiron_base and pure Python. It is described in J. Janssen et al., Digital Discovery (2025).
- pyironflow: the visual programming environment used in this repository.
- pyiron_workflow: the graph-based workflow engine behind pyironflow.
- NFDI4Ing Scientific Workflow Requirements: the benchmark behind the NFDI4Ing example.