Skip to content

GridSearchCV & RandomizedSearchCV GPU support #2406

Closed

Description

@napetrov

Discussed in #2350

Originally posted by idkSeth March 8, 2025
In order to offload to GPU, input data needs to be passed as dpctl.tensor.usm_ndarray as per the guide.

GridSearchCV and RandomizedSearchCV is not patched and when trying to use them with the input data as dpctl.tensor.usm_ndarray, an indexing error is raised.

Can searching be patched to allow for hyper-parameter searching on GPU/CPU with usm_ndarray?

Activity

  1. added theissue type on Apr 6, 2025
  2. david-cortes-intel commented on Aug 8, 2025

    @david-cortes-intel
    Contributor

    This is now solved on the dpctl side: IntelPython/dpctl#2128

    Should be possible to use the classes from sklearn with sklearnex estimators and dpctl/dpnp arrays after version 0.21 of dpctl.

  3. david-cortes-intel commented on Oct 28, 2025

    @david-cortes-intel
    Contributor

    This is now possible to do with array API classes.

    Example with DPNP (>= 0.19):

    import os
    os.environ["SKLEARNEX_VERBOSE"] = "INFO"
    os.environ["SCIPY_ARRAY_API"] = "1"
    import numpy as np
    import dpnp
    from sklearnex import config_context
    from sklearnex.linear_model import Ridge
    from sklearn.datasets import make_regression
    from sklearn.model_selection import GridSearchCV, KFold
    
    X_cpu, y_cpu = make_regression(random_state=123)
    X_gpu = dpnp.array(X_cpu, dtype=np.float32, device="gpu")
    y_gpu = dpnp.array(y_cpu, dtype=np.float32, device="gpu")
    
    cv = GridSearchCV(
        estimator=Ridge(),
        param_grid={"alpha": [1e-1, 1, 1e1]},
        n_jobs=1,
        refit=True,
        cv=KFold(
            n_splits=5,
            shuffle=True,
            random_state=123,
        ),
    )
    with config_context(array_api_dispatch=True):
        cv.fit(X_gpu, y_gpu)

    Example with PyTorch:

    import os
    os.environ["SKLEARNEX_VERBOSE"] = "INFO"
    os.environ["SCIPY_ARRAY_API"] = "1"
    import torch
    from sklearnex import config_context
    from sklearnex.linear_model import Ridge
    from sklearn.datasets import make_regression
    from sklearn.model_selection import GridSearchCV, KFold
    
    X_cpu, y_cpu = make_regression(random_state=123)
    X_gpu = torch.tensor(X_cpu, dtype=torch.float32, device="xpu")
    y_gpu = torch.tensor(y_cpu, dtype=torch.float32, device="xpu")
    
    cv = GridSearchCV(
        estimator=Ridge(),
        param_grid={"alpha": [1e-1, 1, 1e1]},
        n_jobs=1,
        refit=True,
        cv=KFold(
            n_splits=5,
            shuffle=True,
            random_state=123,
        ),
    )
    with config_context(array_api_dispatch=True):
        cv.fit(X_gpu, y_gpu)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions