Skip to content

Solve memory leak to fix issue #7210 - #7247

Merged
sklam merged 12 commits into
numba:masterfrom
ysheffer:master
Nov 29, 2021
Merged

sklam merged 12 commits into
numba:masterfrom
ysheffer:master

Conversation

@ysheffer

@ysheffer ysheffer commented Jul 25, 2021 •

Copy link
Copy Markdown
Contributor

Added a macro to _pymodule.h and set attributes in _num_threads.c, _omppool.cpp,
tbbpool.cpp, workqueue.c using a temporary variable to solve memory leak and fix #7210 .

ysheffer added 2 commits July 24, 2021 14:52
Redefined MOD_INIT sections in _num_threads.c, _omppool.cpp,
tbbpool.cpp, workqueue.c.
ysheffer added 2 commits July 25, 2021 10:34
Redefined MOD_INIT sections in _num_threads.c, _omppool.cpp,
tbbpool.cpp, workqueue.c.
Redefined MOD_INIT sections in _num_threads.c, _omppool.cpp,
tbbpool.cpp, workqueue.c.
@ysheffer ysheffer closed this Jul 25, 2021
ysheffer added 2 commits July 25, 2021 10:45
Redefined MOD_INIT sections in _num_threads.c, _omppool.cpp,
tbbpool.cpp, workqueue.c.
Redefined MOD_INIT sections in _num_threads.c, _omppool.cpp,
tbbpool.cpp, workqueue.c.
@ysheffer ysheffer reopened this Jul 25, 2021
ysheffer added 2 commits July 25, 2021 10:59
Redefined MOD_INIT sections in _num_threads.c, _omppool.cpp,
tbbpool.cpp, workqueue.c.
Redefined MOD_INIT sections in _num_threads.c, _omppool.cpp,
tbbpool.cpp, workqueue.c.
@esc

esc commented Jul 26, 2021

Copy link
Copy Markdown
Member

@ysheffer thank you for submitting this to the Numba issue tracker. It seems like your PR is doing a lot of cosmetics to the code in addition to implementing the feature. Do you think you could submit this PR with only the feature and not the cosmetics, as this makes it harder to review. Thanks!

@esc esc added 4 - Waiting on author Waiting for author to respond to review and removed 3 - Ready for Review labels Jul 26, 2021
Redefined MOD_INIT sections in _num_threads.c, _omppool.cpp,
tbbpool.cpp, workqueue.c.
@ysheffer

Copy link
Copy Markdown
Contributor Author

@esc Thank you for your comment. Sure thing, I have updated the PR accordinaglly.

@esc esc added 3 - Ready for Review Effort - short Short size effort needed and removed 4 - Waiting on author Waiting for author to respond to review labels Jul 26, 2021
@esc

esc commented Jul 26, 2021

Copy link
Copy Markdown
Member

@ysheffer thank you for making those changes, I have added it to the queue for review.

Comment thread numba/_pymodule.h Outdated
#define PyInt_Check PyLong_Check
#define PyInt_CheckExact PyLong_CheckExact
// Define object attribute using a temporary variable to avoid memory leak
#define SetAttrStringFromVoidPointer(m, name, tmp) \

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you had better wrap these three lines with a do { } while (false) block and declare a temporary variable in the block instead of using an external variable.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Snape3058 thanks, agree.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is now applied, but I used while (0) instead of while (false) because false is not defined everywhere the macro is used.

@stuartarchibald stuartarchibald left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ysheffer Many thanks for the patch, couple of minor things to look at else looks good.

Comment thread numba/_pymodule.h Outdated
#define PyInt_Check PyLong_Check
#define PyInt_CheckExact PyLong_CheckExact
// Define object attribute using a temporary variable to avoid memory leak
#define SetAttrStringFromVoidPointer(m, name, tmp) \

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Snape3058 thanks, agree.

Comment thread numba/np/ufunc/omppool.cpp Outdated
Comment thread numba/_pymodule.h Outdated
@stuartarchibald stuartarchibald added 4 - Waiting on author Waiting for author to respond to review and removed 3 - Ready for Review labels Aug 17, 2021
@stuartarchibald stuartarchibald added this to the Numba 0.55 RC milestone Nov 18, 2021
sklam and others added 2 commits November 18, 2021 09:13
Co-authored-by: stuartarchibald <stuartarchibald@users.noreply.github.com>
This avoids it needing to have a temp passed in.

Formatting is also altered slightly to match that of existing code.
@gmarkall

Copy link
Copy Markdown
Member

@stuartarchibald This should be ready for another look now.

@gmarkall gmarkall added 4 - Waiting on reviewer Waiting for reviewer to respond to author and removed 4 - Waiting on author Waiting for author to respond to review labels Nov 25, 2021

@stuartarchibald stuartarchibald left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ysheffer Many thanks for the patch, @gmarkall many thanks for responding to comments and providing fixes against the original patch. Good to see this leak patched.

@stuartarchibald stuartarchibald added 5 - Ready to merge Review and testing done, is ready to merge and removed 4 - Waiting on reviewer Waiting for reviewer to respond to author labels Nov 25, 2021
@stuartarchibald

Copy link
Copy Markdown
Contributor

@ysheffer Congratulations on your first contribution to Numba!

@sklam
sklam merged commit 953b087 into numba:master Nov 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

5 - Ready to merge Review and testing done, is ready to merge Effort - short Short size effort needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Memory leak bugs in module init functions with PyObject_SetAttrString

6 participants