Skip to content

FileExistsError error when psyplot is imported from multiple parallel processes #52

Closed

Description

@schlunma

Hello!

First of all, thanks for this great package!!

Problem description

We (the ESMValTool community) found an issue when importing psyplot from different parallel processes in a test environment, which leads to a race condition where the package finds that it has no config file and tries to create that, but two processes try that at the same time, resulting in a FileExistsError: [Errno 17] File exists: '/Users/runner/.config/psyplot'. Note that we only found this behavior for OSX.

Code Sample, a copy-pastable example if possible

An error on our CI machine using the import below is given here,

from psyplot.config.logsetup import _get_home

but we also found the same problem when using

import psyplot.project as psy

Please note that due to the race condition, the error cannot be reproduced consistently.

Expected Output

No error and no race condition.

Full error message

Details
 tests/unit/test_platform.py:4: in <module>
    from psyplot.config.logsetup import _get_home
../../../miniconda3/envs/esmvaltool/lib/python3.9/site-packages/psyplot/__init__.py:30: in <module>
    from psyplot.config.rcsetup import rcParams
../../../miniconda3/envs/esmvaltool/lib/python3.9/site-packages/psyplot/config/__init__.py:42: in <module>
    config_path = psyplot_fname()
../../../miniconda3/envs/esmvaltool/lib/python3.9/site-packages/psyplot/config/rcsetup.py:978: in psyplot_fname
    configdir = get_configdir()
../../../miniconda3/envs/esmvaltool/lib/python3.9/site-packages/psyplot/config/rcsetup.py:1028: in get_configdir
    os.makedirs(p)
../../../miniconda3/envs/esmvaltool/lib/python3.9/os.py:225: in makedirs
    mkdir(name, mode)
E   FileExistsError: [Errno 17] File exists: '/Users/runner/.config/psyplot'

Output of psyplot -aV

Details
psyplot:
  requirements:
    matplotlib: 3.5.2
    numpy: 1.22.3
    pandas: 1.4.2
    python: 3.8.13 | packaged by conda-forge
    xarray: 2022.3.0
  version: 1.4.2

NOTE

This is a bug report.

For requesting new features, use this template.

For changing existing features, use this template.

Activity

  1. Chilipp commented on May 18, 2022

    @Chilipp
    Member

    thanks for reporting @schlunma! this should be easy to fix. I'll make a PR

  2. self-assigned this
    on May 18, 2022
  3. added a commit that references this issue on May 18, 2022
    8ef9d1d
  4. added a commit that references this issue on Aug 15, 2022
    a5d8114
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions