Repository navigation
Sometimes buildbot fails to cleanup working copy #54182
Description
| BPO | 9973 |
|---|---|
| Nosy | @db3l, @pitrou, @zware, @zooba |
| Superseder | |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/zware'
closed_at = <Date 2014-07-07.18:54:23.369>
created_at = <Date 2010-09-28.16:34:19.044>
labels = ['type-bug', 'tests', 'OS-windows']
title = 'Sometimes buildbot fails to cleanup working copy'
updated_at = <Date 2014-07-07.18:54:23.367>
user = 'https://bugs.python.org/ocean-city'bugs.python.org fields:
activity = <Date 2014-07-07.18:54:23.367>
actor = 'zach.ware'
assignee = 'zach.ware'
closed = True
closed_date = <Date 2014-07-07.18:54:23.369>
closer = 'zach.ware'
components = ['Tests', 'Windows']
creation = <Date 2010-09-28.16:34:19.044>
creator = 'ocean-city'
dependencies = []
files = ['19968']
hgrepos = []
issue_num = 9973
keywords = ['patch', 'buildbot']
message_count = 10.0
messages = ['117536', '123562', '127552', '127553', '127554', '127555', '221689', '221691', '221692', '222498']
nosy_count = 6.0
nosy_names = ['db3l', 'pitrou', 'ocean-city', 'BreamoreBoy', 'zach.ware', 'steve.dower']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = '21907'
type = 'behavior'
url = 'https://bugs.python.org/issue9973'
versions = ['Python 3.5']
Activity
ocean-city commentedon Sep 28, 2010
Sometimes, buildbot fails to clean working copy.
/////////////////////////////////////////////
(snip)
Clean started: Project: _hashlib, Configuration: Debug|Win32
Deleting intermediate and output files for project '_hashlib', configuration 'Debug|Win32'
_hashlib : error PRJ0008 : Could not delete file 'c:\buildslave\2.7.moore-windows\build\PCbuild\_hashlib_d.pyd'.
Make sure that the file is not open by another process and is not write-protected.
_hashlib - 1 error(s), 0 warning(s)
Build complete: 8 Projects succeeded, 17 Projects failed, 1 Projects skipped
/////////////////////////////////////////////
This might cause checkout error next time. Maybe attached
patch can improve this situation. Thanks.
db3l commentedon Dec 7, 2010
I wonder if it might be better to have kill_python run at the tail end of test.bat, since there's a potential window where losing a connection to the build master might not run the clean step, so still risk blocking a future build after a reconnect due to still running processes.
pitrou commentedon Jan 30, 2011
As far as I can tell, the same change also needs to be done in build-amd64.bat and clean-amd64.bat.
pitrou commentedon Jan 30, 2011
By the way, the order of these lines in clean-amd64.bat looks wrong:
cd PCbuild
@echo Deleting .pyc/.pyo files ...
del /s Lib\.pyc Lib\.pyo
pitrou commentedon Jan 30, 2011
Oh, and the regrtest command line in test-amd64.bat lacks some options ("-rwW -n") compared to test.bat.
db3l commentedon Jan 30, 2011
Probably - it's just a general sequencing change, so I suppose should apply equally to all platforms. I suppose even better would be to consolidate the two clean scripts into one (with a parameter for 32 v. 64), but just patching both is less of a change. Unrelated, but it's interesting how the amd64 script makes sure to be in the PCBuild tree before deleting byte compiled files, but the standard script doesn't.
Oh, I should probably also update my prior thought in this ticket, since based on a python-dev thread Hirokazu pointed out that just having things at the end of test.bat could be exposed if the actual test itself never returns to the script.
That might have higher odds than the build master losing contact with the slave between build and clean step, so the original patch is likely the better choice. Or maybe do it in both both places.
But just moving things to clean first would cover the vast majority of cases I seem to see on my build slaves, which only rarely lose contact with the master.
BreamoreBoy commentedon Jun 27, 2014
@Steve I'm assuming that this is covered by the work you're doing for 3.5 builds, am I correct?
zooba commentedon Jun 27, 2014
Probably, but that work is not going to be checked in for a while (until we have guarantees that we'll be able to use VC14 and there's a 'go-live' version available). If this is causing problems now, it should be fixed.
The patch looks fine to me, but Zachary Ware does more of the buildbot script maintenance (not sure if I can nosy him without being able to log in...)
Sent from my Windows Phone
From: Mark Lawrence<mailto:report@bugs.python.org>
Sent: 6/27/2014 9:39
To: Steve Dower<mailto:Steve.Dower@microsoft.com>
Subject: [bpo-9973] Sometimes buildbot fails to cleanup working copy
Mark Lawrence added the comment:
@Steve I'm assuming that this is covered by the work you're doing for 3.5 builds, am I correct?
----------
components: +Windows
nosy: +BreamoreBoy, steve.dower
versions: +Python 3.4, Python 3.5 -Python 3.1, Python 3.2
Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue9973\>
zware commentedon Jun 27, 2014
The real simple method here would be to replace clean[-amd64].bat with a call to kill_python_d (if it exists), followed by an "hg --config extensions.purge= purge --all". That ought to give as much of a guarantee of a clean slate as possible, with the added benefit of hg purge reporting files that it just can't remove.
zware commentedon Jul 7, 2014
This should be fixed by bpo-21907 for 3.5, at least; other versions can either have bpo-21907 backported (in whole or in part), or continue to live with it (like they have for the past 4 years).