Differences between revisions 19 and 21 (spanning 2 versions)
Revision 19 as of 2004-10-03 20:48:21
Size: 3536
Editor: syr-24-59-54-56
Comment: removed spam links
Revision 21 as of 2005-08-24 21:49:34
Size: 7028
Editor: 208
Comment: Added example py2exe subclass to support automatic UPXing of dll/pyd modules
Deletions are marked like this. Additions are marked like this.
Line 89: Line 89:
Repacking the "library.zip" with [http://www.7-zip.org/ 7-ZIP] saving 357KB (577KB original to 220KB 7ZIP compressed) Repacking the "library.zip" with [http://www.7-zip.org/ 7-ZIP] saving 357KB (577KB original to 220KB 7ZIP compressed).
Note: it requires "import zlib" in your source file to allow unzipping operations.
Line 109: Line 110:

= Automatically UPXing extension modules/dlls =

While converting over from prior use of the Installer package, I missed its ability to automatically UPX binary modules as they were included in the distribution. So this example creates a custom command class based on py2exe that performs the UPX as part of the file copy process into the distribution directory. The one difference compared to Installer is that it doesn't currently keep a cache of UPXed files, although it could easily enough be extended to do that.

Note that as designed I believe it will UPX any dll/pyd file copied over, even if they are part of the data files for the application. But since anything packaged up with py2exe is going to be running independently and unlikely to need to share any DLLs with other applications, there's little harm in UPXing anything in sight.

The class below is a portion of a py2exe-subclass command class that I use that does the following, based on a new "upx" configuration value:

 * Compress (using upx) any dll or pyd file that is copied into the distribution directory. Since it does this at copy time, you only take the hit of upx whenever the file is updated (or you clean out the dist directory).
 * Deals with the fact that you have to postpone UPXing the main python##.dll file until py2exe has a chance to fix it's version identification, and that you shouldn't re-patch the file once it's been UPXed.

This command class was written with py2exe 0.5.4.

{{{
from distutils.core import setup
from py2exe.build_exe import py2exe

#--------------------------------------------------------------------------
#
# Define our own command class based on py2exe so we can perform some
# customizations, and in particular support UPXing the binary files.
#
#--------------------------------------------------------------------------

class Py2exe(py2exe):

    def initialize_options(self):
        # Add a new "upx" option for compression with upx
        py2exe.initialize_options(self)
        self.upx = 0

    def copy_file(self, *args, **kwargs):
        # Override to UPX copied binaries.
        (fname, copied) = result = py2exe.copy_file(self, *args, **kwargs)

        basename = os.path.basename(fname)
        if (copied and self.upx and
            (basename[:6]+basename[-4:]).lower() != 'python.dll' and
            fname[-4:].lower() in ('.pyd', '.dll')):
            os.system('upx --best "%s"' % os.path.normpath(fname))
        return result

    def patch_python_dll_winver(self, dll_name, new_winver=None):
        # Override this to first check if the file is upx'd and skip if so
        if not self.dry_run:
            if not os.system('upx -qt "%s" >nul' % dll_name):
                if self.verbose:
                    print "Skipping setting sys.winver for '%s' (UPX'd)" % \
                          dll_name
            else:
                py2exe.patch_python_dll_winver(self, dll_name, new_winver)
                # We UPX this one file here rather than in copy_file so
                # the version adjustment can be successful
                if self.upx:
                    os.system('upx --best "%s"' % os.path.normpath(dll_name))
}}}

This class would then be used in your main setup() call as:

{{{
setup(...
      cmdclass = {'py2exe': Py2exe}
      ...
      )
}}}

-- David Bolen

Something great

Within samples/extending, Thomas Heller gives a Py2Exe setup.py script which automagically calls [http://www.jrsoftware.org/isinfo.php InnoSetup] to create a great looking (and perfectly working) Windows-Installer. Yes, really, ONE setup.exe is created which contains ALL stuff to have a readily installable Software out of your product.

also large

I program using wxPython. A great library, a large library.

2.240.512 wxc.pyd 
3.416.064 wxmsw24uh.dll

Two modules you cannnot skip. In addition there is as allways

974.915 python23.dll

giving more than 6 Megabytes with no line of code written

In total:

28 Datei(en)      9.549.525 Bytes  (in lib)

and

2 Datei(en)      1.011.779 Bytes   (in prog)

summing up to 10.5 Megabyte ... not much with DSL, but... shocking, isn't it?

get it smaller

In the standard configuration [http://www.jrsoftware.org/isinfo.php InnoSetup] compressed it down to around 6.5 Megabyte.

I changed the setup.py to contain

   1 print >> ofi, r"[Setup]"
   2 print >> ofi, r"AppName=%s" % self.name
   3 print >> ofi, r"AppVerName=%s %s" % (self.name, self.version)
   4 print >> ofi, r"DefaultDirName={pf}\%s" % self.name
   5 print >> ofi, r"DefaultGroupName=%s" % self.name
   6 print >> ofi, r"Compression=bzip" 
   7 print >> ofi

... the special line is

   1 print >> ofi, r"Compression=bzip" 

the result

4.335.283 setup.exe 

Quite an OK download ... and I use ZODB and Elementtree and pythoncom and and and....

Remarks

  • I ([http://videocapture.sourceforge.net/ MGr]) get for my application (which uses wxWindows) even smaller files when using 7-zip instead of bzip. To use 7-zip one has to specify "Compression=none" in the generated installer script (*.iss) and then manually load the script into [http://www.istool.org/ ISTool], which is a front end to Inno Setup. Clicking at the 'Options' button and selecting '7-zip' in the compression combo box does the trick.

    7.208 kB  uncompressed
    2.886 kB  zip
    2.677 kB  bzip
    2.160 kB  7-zip

    The previous comparison shows, that 7-zip results by far in the smallest installer size. An additional benefit of using 7-zip is the fast installation. When using bzip, the files are decompressed in the last step of the installation process, right then when the files are written to the program folder. Since bzip is computationally quite intensive, the progress bar fills not extremely fast. When using 7-zip, there is a short decompression when starting the installer, but the installation process afterwards it fast as lightning, because of "Compression=none" :) .

Addition:

Innosetup 4.2 has 7z "LZMA" compression included. Just use

   1 print >> ofi, r"Compression=lzma/max" 

HAM2005-05-20

7ZIP and UPX

Repacking the "library.zip" with [http://www.7-zip.org/ 7-ZIP] saving 357KB (577KB original to 220KB 7ZIP compressed). Note: it requires "import zlib" in your source file to allow unzipping operations.

Using [http://upx.sourceforge.net/ UPX ] the executable packer to compress *.pyd, *.dll and *.exe Files:

Example: _sre.pyd, _winreg.pyd, python23.dll, unicodedata.pyd, w9xpopen.exe, zlib.pyd 1516,2KB original to 584KB compressed)

Everything together: Original Size: 2093,2 compressed: 804KB !!!

A short Windows Batch-File to automate (see WinBatch):

7z.exe -aoa x library.zip -olibrary\
del library.zip

cd library\
7z.exe a -tzip -mx9 ..\library.zip -r
cd..
rd library /s /q

upx.exe --best *.*

Automatically UPXing extension modules/dlls

While converting over from prior use of the Installer package, I missed its ability to automatically UPX binary modules as they were included in the distribution. So this example creates a custom command class based on py2exe that performs the UPX as part of the file copy process into the distribution directory. The one difference compared to Installer is that it doesn't currently keep a cache of UPXed files, although it could easily enough be extended to do that.

Note that as designed I believe it will UPX any dll/pyd file copied over, even if they are part of the data files for the application. But since anything packaged up with py2exe is going to be running independently and unlikely to need to share any DLLs with other applications, there's little harm in UPXing anything in sight.

The class below is a portion of a py2exe-subclass command class that I use that does the following, based on a new "upx" configuration value:

  • Compress (using upx) any dll or pyd file that is copied into the distribution directory. Since it does this at copy time, you only take the hit of upx whenever the file is updated (or you clean out the dist directory).
  • Deals with the fact that you have to postpone UPXing the main python##.dll file until py2exe has a chance to fix it's version identification, and that you shouldn't re-patch the file once it's been UPXed.

This command class was written with py2exe 0.5.4.

from distutils.core import setup
from py2exe.build_exe import py2exe

#--------------------------------------------------------------------------
#
# Define our own command class based on py2exe so we can perform some
# customizations, and in particular support UPXing the binary files.
#
#--------------------------------------------------------------------------

class Py2exe(py2exe):

    def initialize_options(self):
        # Add a new "upx" option for compression with upx
        py2exe.initialize_options(self)
        self.upx = 0

    def copy_file(self, *args, **kwargs):
        # Override to UPX copied binaries.
        (fname, copied) = result = py2exe.copy_file(self, *args, **kwargs)

        basename = os.path.basename(fname)
        if (copied and self.upx and
            (basename[:6]+basename[-4:]).lower() != 'python.dll' and
            fname[-4:].lower() in ('.pyd', '.dll')):
            os.system('upx --best "%s"' % os.path.normpath(fname))
        return result

    def patch_python_dll_winver(self, dll_name, new_winver=None):
        # Override this to first check if the file is upx'd and skip if so
        if not self.dry_run:
            if not os.system('upx -qt "%s" >nul' % dll_name):
                if self.verbose:
                    print "Skipping setting sys.winver for '%s' (UPX'd)" % \
                          dll_name
            else:
                py2exe.patch_python_dll_winver(self, dll_name, new_winver)
                # We UPX this one file here rather than in copy_file so
                # the version adjustment can be successful
                if self.upx:
                    os.system('upx --best "%s"' % os.path.normpath(dll_name))

This class would then be used in your main setup() call as:

setup(...
      cmdclass = {'py2exe': Py2exe}
      ...
      )

-- David Bolen

BetterCompression (last edited 2008-07-08 11:27:44 by localhost)