Skip to content

Adjust threshold for bad worker on worker table based on worker-ttl #8749

Open

Description

@fjetter

We're currently marking workers as red on the worker table page if they don't heartbeat within 60s. The 60s is quite arbitrary and particularly if users expect heartbeats not to go through (e.g. due to user code holding the gil) this can be a bad UX.

The 60s are currently hard coded here

https://github.com/fjetter/distributed/blob/28804b72acc210935c061a0d68d46d4a6ae50a94/distributed/http/templates/worker-table.html#L19

and it is not obvious to users what the threshold is and how to control this behavior.

Dask considers these workers dead once the last seen goes above distributed.scheduler.worker-ttl (default 5min) and I feel like this paramter should influence the coloring of the row

Maybe 25% yellow/orange, 50% red or something similar instead of hard coding to 60s

Activity

  1. added
    enhancementImprove existing functionality or make things work better
    good first issueClearly described and easy to accomplish. Good for beginners to the project.
    on Jul 4, 2024
  2. jaabberwocky commented on Jul 4, 2024

    @jaabberwocky

    @fjetter Will take a stab at this!

  3. fjetter commented on Jul 5, 2024

    @fjetter
    MemberAuthor

    @jaabberwocky Of course, let me know if there are any questions!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    diagnosticsenhancementImprove existing functionality or make things work bettergood first issueClearly described and easy to accomplish. Good for beginners to the project.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions