[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Condor-devel] Condor disk size
- Date: Fri, 20 May 2011 07:44:05 -0400
- From: Matthew Farrellee <matt@xxxxxxxxxx>
- Subject: Re: [Condor-devel] Condor disk size
On 05/20/2011 07:26 AM, Brian Bockelman wrote:
On May 19, 2011, at 10:08 PM, Matthew Farrellee wrote:
Impressive.
Michael did most of the work :)
I thought he did all of it. 8o)
What do you think about making the lib names less generic, say libcondorutils instead of libutils? Especially since none actually come with headers.
Condor libs - procd_client, daemon_core, daemon_client, procapi, cedar, privsep, sysapi, ccb, utils
Yeah, the names kind of drove me nuts. I was thinking about going through the build files and prefixing everything with condor. But that can be done as a separate ticket.
Another possibility is to drop everything in /usr/{lib64,lib}/condor.
Yet another possibility is to do some engineering and aggregate these into a smaller set of libraries.
/usr/lib{64,}/condor sounds like a good idea. I think of shrinking the
set as an artifact from breaking unnecessary deps.
Very poorly named lib - condorapi (log reader not even used by dagman, iirc)
Uh - I think I might be the last person in the world that actually uses libcondorapi then?
I really hope so. It's an abomination.
Ok to assume elimination of --allow-shlib-undefined is left as an exercise for the hopelessly heroic?
Hopelessly heroic.
Actually, it'd just take an expert with a scalpel, some time, and permission to cause a temporary mess. A lot of the issues looked relatively mild.
If you're bored some Saturday and want to take a stab at it, let me know.
Best,
matt
Brian
Best,
matt
On 05/19/2011 10:05 PM, Brian Bockelman wrote:
Hi,
Michael Hanke's shared libraries work (#2132) got put into a separate branch today, so I've had a lovely evening porting it over to our local RHEL5 RPM (which is basically the Fedora RPM plus Globus support and other goodies enabled). Unfortunately, there were a handful of issues that didn't make the porting clean.
The difference in on-disk size is, as expected, significant. For reference, the "big mother" unstripped tarball at UW is 448MB and the UW RPM is 184MB. Recall these include all externals as static libs, while the rest of the RPMs below link to them (so this isn't an "apples" comparison).
First, the PROPER build with static linking of condor libraries:
[brian@brian-test condor]$ ll ~/rpmbuild/RPMS/x86_64/condor-*7.6.1-0.7.pre.x86_64.rpm -h
-rw-r--r-- 1 brian brian 38M May 17 09:35 /home/brian/rpmbuild/RPMS/x86_64/condor-7.6.1-0.7.pre.x86_64.rpm
-rw-r--r-- 1 brian brian 236K May 17 09:35 /home/brian/rpmbuild/RPMS/x86_64/condor-classads-7.6.1-0.7.pre.x86_64.rpm
-rw-r--r-- 1 brian brian 299K May 17 09:35 /home/brian/rpmbuild/RPMS/x86_64/condor-classads-devel-7.6.1-0.7.pre.x86_64.rpm
-rw-r--r-- 1 brian brian 157M May 17 09:39 /home/brian/rpmbuild/RPMS/x86_64/condor-debuginfo-7.6.1-0.7.pre.x86_64.rpm
-rw-r--r-- 1 brian brian 775K May 17 09:35 /home/brian/rpmbuild/RPMS/x86_64/condor-kbdd-7.6.1-0.7.pre.x86_64.rpm
-rw-r--r-- 1 brian brian 2.0M May 17 09:35 /home/brian/rpmbuild/RPMS/x86_64/condor-qmf-7.6.1-0.7.pre.x86_64.rpm
-rw-r--r-- 1 brian brian 1.6M May 17 09:35 /home/brian/rpmbuild/RPMS/x86_64/condor-vm-gahp-7.6.1-0.7.pre.x86_64.rpm
Now, with the shared library patches:
[brian@brian-test condor]$ ll ~/rpmbuild/RPMS/x86_64/condor-*7.6.1-0.9.pre.x86_64.rpm -h
-rw-r--r-- 1 brian brian 4.7M May 19 20:34 /home/brian/rpmbuild/RPMS/x86_64/condor-7.6.1-0.9.pre.x86_64.rpm
-rw-r--r-- 1 brian brian 236K May 19 20:34 /home/brian/rpmbuild/RPMS/x86_64/condor-classads-7.6.1-0.9.pre.x86_64.rpm
-rw-r--r-- 1 brian brian 77K May 19 20:34 /home/brian/rpmbuild/RPMS/x86_64/condor-classads-devel-7.6.1-0.9.pre.x86_64.rpm
-rw-r--r-- 1 brian brian 22M May 19 20:34 /home/brian/rpmbuild/RPMS/x86_64/condor-debuginfo-7.6.1-0.9.pre.x86_64.rpm
-rw-r--r-- 1 brian brian 28K May 19 20:34 /home/brian/rpmbuild/RPMS/x86_64/condor-kbdd-7.6.1-0.9.pre.x86_64.rpm
-rw-r--r-- 1 brian brian 413K May 19 20:34 /home/brian/rpmbuild/RPMS/x86_64/condor-qmf-7.6.1-0.9.pre.x86_64.rpm
-rw-r--r-- 1 brian brian 143K May 19 20:34 /home/brian/rpmbuild/RPMS/x86_64/condor-vm-gahp-7.6.1-0.9.pre.x86_64.rpm
4.7MB ain't shabby! Shared libraries being shared libraries, you can now peruse all the linked baggage of the svelte 73KB condor_q. See below.
Dynamic libraries have all the advantages of dynamic libraries and all the disadvantages of dynamic libraries. No need to revisit them here ... but I thought the data point about binary size was interesting enough to share!
Brian
[brian@brian-test condor]$ ldd /usr/bin/condor_q
linux-vdso.so.1 => (0x00007fffda7c4000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003feb200000)
libconversion.so => /usr/lib64/libconversion.so (0x00007f722b2ad000)
libanalysis.so => /usr/lib64/libanalysis.so (0x00007f722b06d000)
libqmgmt.so => /usr/lib64/libqmgmt.so (0x00007f722ae62000)
libprocd_client.so => /usr/lib64/libprocd_client.so (0x00007f722ac58000)
libdaemon_client.so => /usr/lib64/libdaemon_client.so (0x00007f722aa15000)
libprocapi.so => /usr/lib64/libprocapi.so (0x00007f722a80b000)
libcedar.so => /usr/lib64/libcedar.so (0x00007f722a59e000)
libprivsep.so => /usr/lib64/libprivsep.so (0x00007f722a399000)
libclassads.so => /usr/lib64/libclassads.so (0x00007f722a111000)
libsysapi.so => /usr/lib64/libsysapi.so (0x00007f7229e60000)
libccb.so => /usr/lib64/libccb.so (0x00007f7229c42000)
libutils.so => /usr/lib64/libutils.so (0x00007f7229880000)
libvomsapi.so.0 => /usr/lib64/libvomsapi.so.0 (0x0000003a2b000000)
libglobus_gss_assist.so.3 => /usr/lib64/libglobus_gss_assist.so.3 (0x000000384ce00000)
libglobus_gssapi_gsi.so.4 => /usr/lib64/libglobus_gssapi_gsi.so.4 (0x000000384d200000)
libglobus_gsi_proxy_core.so.0 => /usr/lib64/libglobus_gsi_proxy_core.so.0 (0x000000384ba00000)
libglobus_gsi_credential.so.1 => /usr/lib64/libglobus_gsi_credential.so.1 (0x000000384de00000)
libglobus_gsi_callback.so.0 => /usr/lib64/libglobus_gsi_callback.so.0 (0x000000384c600000)
libglobus_gsi_sysconfig.so.1 => /usr/lib64/libglobus_gsi_sysconfig.so.1 (0x000000384b600000)
libglobus_oldgaa.so.0 => /usr/lib64/libglobus_oldgaa.so.0 (0x000000384d600000)
libglobus_gsi_cert_utils.so.0 => /usr/lib64/libglobus_gsi_cert_utils.so.0 (0x000000384be00000)
libglobus_openssl.so.0 => /usr/lib64/libglobus_openssl.so.0 (0x000000384ca00000)
libglobus_openssl_error.so.0 => /usr/lib64/libglobus_openssl_error.so.0 (0x000000384c200000)
libglobus_proxy_ssl.so.1 => /usr/lib64/libglobus_proxy_ssl.so.1 (0x0000003ff2e00000)
libglobus_callout.so.0 => /usr/lib64/libglobus_callout.so.0 (0x000000384da00000)
libglobus_common.so.0 => /usr/lib64/libglobus_common.so.0 (0x000000384b200000)
libltdl.so.3 => /usr/lib64/libltdl.so.3 (0x0000003feda00000)
libexpat.so.0 => /lib64/libexpat.so.0 (0x0000003d68200000)
libpcre.so.0 => /lib64/libpcre.so.0 (0x0000003ff0200000)
libssl.so.6 => /lib64/libssl.so.6 (0x00000031f0c00000)
libcrypto.so.6 => /lib64/libcrypto.so.6 (0x0000003fede00000)
libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003feee00000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003fee600000)
libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003fef600000)
libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x0000003feea00000)
libdaemon_core.so => /usr/lib64/libdaemon_core.so (0x00007f7229629000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x0000003a2ac00000)
libm.so.6 => /lib64/libm.so.6 (0x0000003feae00000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000003a2a800000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003feb600000)
libc.so.6 => /lib64/libc.so.6 (0x0000003feaa00000)
/lib64/ld-linux-x86-64.so.2 (0x0000003fea600000)
libz.so.1 => /usr/lib64/libz.so.1 (0x0000003feba00000)
libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00000031ef400000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x0000003fee200000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003fece00000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x0000003fec200000)
libsepol.so.1 => /lib64/libsepol.so.1 (0x0000003fec600000)
[brian@brian-test condor]$ ldd /usr/sbin/condor_shadow
linux-vdso.so.1 => (0x00007fffb8bff000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003feb200000)
libqmgmt.so => /usr/lib64/libqmgmt.so (0x00007f7035a14000)
libprocd_client.so => /usr/lib64/libprocd_client.so (0x00007f7035809000)
libdaemon_core.so => /usr/lib64/libdaemon_core.so (0x00007f70355ba000)
libdaemon_client.so => /usr/lib64/libdaemon_client.so (0x00007f7035378000)
libprocapi.so => /usr/lib64/libprocapi.so (0x00007f703516d000)
libcedar.so => /usr/lib64/libcedar.so (0x00007f7034f00000)
libprivsep.so => /usr/lib64/libprivsep.so (0x00007f7034cfc000)
libclassads.so => /usr/lib64/libclassads.so (0x00007f7034a73000)
libsysapi.so => /usr/lib64/libsysapi.so (0x00007f70347c2000)
libccb.so => /usr/lib64/libccb.so (0x00007f70345a5000)
libutils.so => /usr/lib64/libutils.so (0x00007f70341e2000)
libvomsapi.so.0 => /usr/lib64/libvomsapi.so.0 (0x0000003a2b000000)
libglobus_gss_assist.so.3 => /usr/lib64/libglobus_gss_assist.so.3 (0x000000384ce00000)
libglobus_gssapi_gsi.so.4 => /usr/lib64/libglobus_gssapi_gsi.so.4 (0x000000384d200000)
libglobus_gsi_proxy_core.so.0 => /usr/lib64/libglobus_gsi_proxy_core.so.0 (0x000000384ba00000)
libglobus_gsi_credential.so.1 => /usr/lib64/libglobus_gsi_credential.so.1 (0x000000384de00000)
libglobus_gsi_callback.so.0 => /usr/lib64/libglobus_gsi_callback.so.0 (0x000000384c600000)
libglobus_gsi_sysconfig.so.1 => /usr/lib64/libglobus_gsi_sysconfig.so.1 (0x000000384b600000)
libglobus_oldgaa.so.0 => /usr/lib64/libglobus_oldgaa.so.0 (0x000000384d600000)
libglobus_gsi_cert_utils.so.0 => /usr/lib64/libglobus_gsi_cert_utils.so.0 (0x000000384be00000)
libglobus_openssl.so.0 => /usr/lib64/libglobus_openssl.so.0 (0x000000384ca00000)
libglobus_openssl_error.so.0 => /usr/lib64/libglobus_openssl_error.so.0 (0x000000384c200000)
libglobus_proxy_ssl.so.1 => /usr/lib64/libglobus_proxy_ssl.so.1 (0x0000003ff2e00000)
libglobus_callout.so.0 => /usr/lib64/libglobus_callout.so.0 (0x000000384da00000)
libglobus_common.so.0 => /usr/lib64/libglobus_common.so.0 (0x000000384b200000)
libltdl.so.3 => /usr/lib64/libltdl.so.3 (0x0000003feda00000)
libexpat.so.0 => /lib64/libexpat.so.0 (0x0000003d68200000)
libpcre.so.0 => /lib64/libpcre.so.0 (0x0000003ff0200000)
libssl.so.6 => /lib64/libssl.so.6 (0x00000031f0c00000)
libcrypto.so.6 => /lib64/libcrypto.so.6 (0x0000003fede00000)
libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003feee00000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003fee600000)
libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003fef600000)
libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x0000003feea00000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x0000003a2ac00000)
libm.so.6 => /lib64/libm.so.6 (0x0000003feae00000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000003a2a800000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003feb600000)
libc.so.6 => /lib64/libc.so.6 (0x0000003feaa00000)
/lib64/ld-linux-x86-64.so.2 (0x0000003fea600000)
libz.so.1 => /usr/lib64/libz.so.1 (0x0000003feba00000)
libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00000031ef400000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x0000003fee200000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003fece00000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x0000003fec200000)
libsepol.so.1 => /lib64/libsepol.so.1 (0x0000003fec600000)
_______________________________________________
Condor-devel mailing list
Condor-devel@xxxxxxxxxxx
https://lists.cs.wisc.edu/mailman/listinfo/condor-devel
