Skip to content
This repository was archived by the owner on Feb 2, 2024. It is now read-only.

Groupby limitations #782

Merged
AlexanderKalistratov merged 2 commits into from
Mar 26, 2020
Merged

Groupby limitations #782

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6387e2d
Groupby limitations
Mar 25, 2020
b9ef3ab
fix limitations
Mar 26, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Groupby limitations
  • Loading branch information
elena.totmenina
elena.totmenina committed Mar 25, 2020
commit 6387e2dbc2a370bea0c02f211b618533c5345be8
74 changes: 64 additions & 10 deletions sdc/datatypes/hpat_pandas_groupby_functions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -625,7 +625,13 @@ def sdc_pandas_series_groupby_var(self, ddof=1, *args):
sdc_pandas_dataframe_groupby_count.__doc__ = sdc_pandas_dataframe_groupby_docstring_tmpl.format(**{
'method_name': 'count',
'example_caption': 'Compute count of group, excluding missing values.',
'limitations_block': '',
'limitations_block':
"""
Limitations
-----------
- This function may reveal slower performance than Pandas* on user system. Users should exercise a tradeoff
between staying in JIT-region with that function or going back to interpreter mode.
""",

@Rubtsowa Rubtsowa Mar 25, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put this in a separate variable to avoid code duplication

@1e-to 1e-to Mar 25, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think it is necessary here

@Rubtsowa Rubtsowa Mar 25, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kozlov-alexey what do you think?

@1e-to 1e-to Mar 25, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we need to add or change limitations, we will have to rewrite the text from the variables back to the lines. To the fact that so clearly

@densmirn densmirn Mar 25, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I vote for inserting the constant text This function may reveal slower performance than Pandas* on user system. Users should exercise a tradeoff between staying in JIT-region with that function or going back to interpreter mode. where it requires through string formatting:

performance_limitation = `This function may reveal slower performance than Pandas* on user system. Users should exercise a tradeoff
    between staying in JIT-region with that function or going back to interpreter mode.`
...
'limitations_block':
    """
    Limitations
    -----------
    - {}.
    """.format(performance_limitation),
...

@densmirn densmirn Mar 25, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or

performance_limitation = `This function may reveal slower performance than Pandas* on user system. Users should exercise a tradeoff
    between staying in JIT-region with that function or going back to interpreter mode.`
...
'limitations_block':
    f"""
    Limitations
    -----------
    - {performance_limitation}
    """,
...

'see_also':
"""
.. seealso::
Expand All @@ -641,7 +647,13 @@ def sdc_pandas_series_groupby_var(self, ddof=1, *args):
sdc_pandas_dataframe_groupby_max.__doc__ = sdc_pandas_dataframe_groupby_docstring_tmpl.format(**{
'method_name': 'max',
'example_caption': 'Compute max of group values.',
'limitations_block': '',
'limitations_block':
"""
Limitations
-----------
- This function may reveal slower performance than Pandas* on user system. Users should exercise a tradeoff
between staying in JIT-region with that function or going back to interpreter mode.
""",
'see_also': '',
'extra_params': ''
})
Expand All @@ -650,7 +662,13 @@ def sdc_pandas_series_groupby_var(self, ddof=1, *args):
sdc_pandas_dataframe_groupby_mean.__doc__ = sdc_pandas_dataframe_groupby_docstring_tmpl.format(**{
'method_name': 'mean',
'example_caption': 'Compute mean of groups, excluding missing values.',
'limitations_block': '',
'limitations_block':
"""
Limitations
-----------
- This function may reveal slower performance than Pandas* on user system. Users should exercise a tradeoff
between staying in JIT-region with that function or going back to interpreter mode.
""",
'see_also':
"""
.. seealso::
Expand All @@ -666,7 +684,13 @@ def sdc_pandas_series_groupby_var(self, ddof=1, *args):
sdc_pandas_dataframe_groupby_median.__doc__ = sdc_pandas_dataframe_groupby_docstring_tmpl.format(**{
'method_name': 'median',
'example_caption': 'Compute median of groups, excluding missing values.',
'limitations_block': '',
'limitations_block':
"""
Limitations
-----------
- This function may reveal slower performance than Pandas* on user system. Users should exercise a tradeoff
between staying in JIT-region with that function or going back to interpreter mode.
""",
'see_also':
"""
.. seealso::
Expand All @@ -682,7 +706,13 @@ def sdc_pandas_series_groupby_var(self, ddof=1, *args):
sdc_pandas_dataframe_groupby_min.__doc__ = sdc_pandas_dataframe_groupby_docstring_tmpl.format(**{
'method_name': 'min',
'example_caption': 'Compute min of group values.',
'limitations_block': '',
'limitations_block':
"""
Limitations
-----------
- This function may reveal slower performance than Pandas* on user system. Users should exercise a tradeoff
between staying in JIT-region with that function or going back to interpreter mode.
""",
'see_also': '',
'extra_params': ''
})
Expand All @@ -691,7 +721,13 @@ def sdc_pandas_series_groupby_var(self, ddof=1, *args):
sdc_pandas_dataframe_groupby_prod.__doc__ = sdc_pandas_dataframe_groupby_docstring_tmpl.format(**{
'method_name': 'prod',
'example_caption': 'Compute prod of group values.',
'limitations_block': '',
'limitations_block':
"""
Limitations
-----------
- This function may reveal slower performance than Pandas* on user system. Users should exercise a tradeoff
between staying in JIT-region with that function or going back to interpreter mode.
""",
'see_also': '',
'extra_params': ''
})
Expand All @@ -700,7 +736,13 @@ def sdc_pandas_series_groupby_var(self, ddof=1, *args):
sdc_pandas_dataframe_groupby_std.__doc__ = sdc_pandas_dataframe_groupby_docstring_tmpl.format(**{
'method_name': 'std',
'example_caption': 'Compute standard deviation of groups, excluding missing values.',
'limitations_block': '',
'limitations_block':
"""
Limitations
-----------
- This function may reveal slower performance than Pandas* on user system. Users should exercise a tradeoff
between staying in JIT-region with that function or going back to interpreter mode.
""",
'see_also':
"""
.. seealso::
Expand All @@ -716,16 +758,28 @@ def sdc_pandas_series_groupby_var(self, ddof=1, *args):
sdc_pandas_dataframe_groupby_sum.__doc__ = sdc_pandas_dataframe_groupby_docstring_tmpl.format(**{
'method_name': 'sum',
'example_caption': 'Compute sum of groups, excluding missing values.',
'limitations_block': '',
'limitations_block':
"""
Limitations
-----------
- This function may reveal slower performance than Pandas* on user system. Users should exercise a tradeoff
between staying in JIT-region with that function or going back to interpreter mode.
""",
'see_also': '',
'extra_params': ''
})


sdc_pandas_dataframe_groupby_var.__doc__ = sdc_pandas_dataframe_groupby_docstring_tmpl.format(**{
'method_name': 'sum',
'method_name': 'var',
'example_caption': 'Compute variance of groups, excluding missing values.',
'limitations_block': '',
'limitations_block':
"""
Limitations
-----------
- This function may reveal slower performance than Pandas* on user system. Users should exercise a tradeoff
between staying in JIT-region with that function or going back to interpreter mode.
""",
'see_also':
"""
.. seealso::
Expand Down