Repository navigation
Set correct PE checksum for .exe files (helps with virus false-positives) #5579
Description
Is your feature request related to a problem? Please describe.
As you've probably all seen before, virus scanner false-positives are a recurring problem for PyInstaller-generated executables, as well as PyInstaller's bootloader (see e.g. #5490, #5474, #5479, ...).
One piece of information virus scanners seem to use is the PE checksum. Quoting from that article:
- 83% of malware had invalid checksums
- 90% of legitimate files had valid checksums.
It looks like anything generated from PyInstaller (as well as the Windows bootloader in the release?) seems to have a checksum of 00 00 00 00, which is thus invalid.
Describe the solution you'd like
PyInstaller should set a correct PE checksum. This is possible using pefile, which it already seems to depend on:
import pefile
pe = pefile.PE('filename.exe')
pe.OPTIONAL_HEADER.CheckSum = pe.generate_checksum()
pe.close()
pe.write('filename.exe')(You can use pe.verify_checksum() to verify that it is correct)
After doing this, e.g. PyInstaller's Windows-64bit/runw.exe (in the 4.2 release) goes down from 12 false-positives to only 5 - most notably, McAfee and Avast (which are the two affected engines I've actually heard of before...) now are happy with the file.
Doing this with my own project (which is built with PyInstaller), @bitraid found that:
the number of false positives for v2.0.2 goes down from 31 to 7. Similarly v2.0.0 goes down to 6 from 29 and v2.0.1 goes down to 8 from 15.
Describe alternatives you've considered
Submitting false-positive reports to antivirus vendors. A pain.
Also, doing this after PyInstaller has generated the exe - that works, but it seems to me like this should be done in PyInstaller itself.
Additional context
See qutebrowser/qutebrowser#6194
Activity
bwoodsend commentedon Feb 22, 2021
Ah yes! Nice one. How that makes an executable any more trustworthy seeing as anyone could do it is beyond me but if it shuts up the bogus AV software then great.
bwoodsend commentedon Feb 22, 2021
@The-Compiler Fancy making the changes or shall I?
The-Compiler commentedon Feb 22, 2021
Feel free!
I don't think I've contributed before(looks like I did 😆)I'm not really sure what the best place for such a change would be - but I also try to avoid working on Windows stuff if I can, so if you don't mind, go ahead. 👍
bwoodsend commentedon Feb 22, 2021
Alright then. I'll blow the cobwebs off my own Windows partition and do it...
scripts: Fix PE checksum for Windows releases
scripts: Fix PE checksum for Windows releases
Building: Windows: Set EXE checksums (pyinstaller#5579).
andyshinn commentedon Feb 22, 2021
I've been following the virus false-positive stuff for a while and struggling with it myself. Here are the before VirusTotal Windows bootloader analysis so we can compare after. The bigger one I have issues with (that I am mot sure the source for) is Windows Defender. Will be interesting to see if this helps that case.
Windows 32bit:
run_d.exe1/70: https://www.virustotal.com/gui/file/5c8ddb668b8fa768d2037879a22214db104e48cf26136cc311e159f007e6436e/detectionrun.exe11/70: https://www.virustotal.com/gui/file/2e4dc3fd4cb960cdc0dd0c8abc176fe80209dbf353c789e79dd0f271ad02a785/detectionrunw_d.exe2/70: https://www.virustotal.com/gui/file/e70c7cfb63a9ca41c55cbca60ff7acd521b67606ab33ccf2661400e876268c7a/detectionrunw.exe14/70: https://www.virustotal.com/gui/file/ddab59c787d9673a8098a1338098effeedca938dc338e96c5fd4654ed0b76dfa/detectionWindows 64bit:
run_d.exe2/71: https://www.virustotal.com/gui/file/8ef748a59cd81d99b0ee3f5008c2275b5630d5bf8e1aecc2ed7b8a288bea21d7/detectionrun.exe4/70: https://www.virustotal.com/gui/file/0367cd6d43be86912de0f30dd118087ac9e79a9b5f7440e496703b1de4c44fd7/detectionrunw_d.exe5/70: https://www.virustotal.com/gui/file/c27d3749d3614e1b3022c84ff240fbcb3605ba6b72fcc1cf9729088fc4345ccc/detectionrunw.exe12/71: https://www.virustotal.com/gui/file/749666ba5af516692ff3716000b992cbcf2857e825c9c1d94fac7ac8ba11759e/detectionBuilding: Windows: Set EXE checksums (pyinstaller#5579).
Building: Windows: Set EXE checksums (pyinstaller#5579).
Building: Windows: Set EXE checksums (pyinstaller#5579).
Building: Windows: Set EXE checksums (pyinstaller#5579).
Building: Windows: Set EXE checksums (#5579). (#5580)
bwoodsend commentedon Feb 23, 2021
I've done it (#5580). EXEs built by PyInstaller will now have the checksum set. The bootloaders won't until we rebuild them so unless you rebuild the bootloaders yourself, there is no point submitting those to virustotal yet. Instead submit your built applications. I have tried rebuilding a bootloader then submitting it and it came back all green.
cli: Switched pyinstaller CLI from argparse to click.
sjackman commentedon Feb 27, 2021
The Internet Archive flagged Pyinstaller as malware: https://www.virustotal.com/gui/file/af5a1b117ea14224d5993b74b45a42f8aab0a3230e1c81dc5a00d1a27ebe631c/detection/f-af5a1b117ea14224d5993b74b45a42f8aab0a3230e1c81dc5a00d1a27ebe631c-1614111479
andyshinn commentedon Feb 27, 2021
I tried with the new PE changes and it's still blocked in Defender and flagged by a bunch in Virus Total, unfortunately.
https://www.virustotal.com/gui/file/a70a7c7b5682abe70dfe7bd186e6f48b636e2a59908bcbdfc0184a4e52cc1635/detection
Legorooj commentedon Feb 27, 2021
Interesting, using commit f078351, I get the following result:
https://www.virustotal.com/gui/file/f5587397e6ce11d659e3b4cfe1637991b9b5547a3f280873ef48948e606f7434/detection
Just a build of
print('Hello world!')as a onefile was what I used.Legorooj commentedon Feb 27, 2021
@sjackman that's Internet Archive flag is on version 3.6... a little out of date.
srkn commentedon Mar 23, 2021
Would you please give a little more detail regarding this? How should we rebuild bootloader to prevent being flagged as malware?
bwoodsend commentedon Mar 23, 2021
Rebuilding the bootloader only affects people who can't install pyinstaller. If that includes you then the docs are here.
Bug 1647533 - Upgrade to pyinstaller 4.3
Bug 1647533 - Upgrade to pyinstaller 4.3 (#857)