[Condor-users] Dynamic and static linking of Condor

[Alan De Smet <adesmet@xxxxxxxxxxx>]

The Condor Team is considering no longer releasing "static"
binaries.  Furthermore, we plan on changing our "dynamic" binary
releases to dynamically link third party libraries, like OpenSSL,
Globus, and Kerberos and providing copies of those libraries as a
convenience.  Today the dynamic releases only dynamically link
libc.

We are interested in any feedback from our users and
contributors.  We don't anticipate this causing problems for
anyone, but we can't predict every user's particular case.  Let
us know if you anticipate problems in your situation and what
your needs are.  If you're eager for us to do this, let us know
that as well!

Feel free to reply to the list, or email your thoughts directly.


DETAILS

Static binaries:

The static binaries that the Condor Team releases today are just
that; static.  They pull in almost no third party libraries at
run time.  This theoretically makes them self contained and very
portable.  In practice they prove problematic and are frequently
less portable than our dynamic releases.  For several years, we
have been advising users to use the dynamic binaries, and to use
the static binaries only in special cases.  For the overwhelming
majority of users, the dynamic binaries are a better solution.
Our proposal would simply eliminate the static binaries.

Dynamic binaries:

The dynamic binaries the Condor Team releases today are
misleadingly named.  These binaries dynamically link in libc, the
C library.  Every other library Condor uses is statically linked
in.  This has proven to work reasonably well.  The binaries are
largely self contained, but using the system's libc improves
portability.  However, there are some problems:

- Newer versions of some libraries Condor uses do not support
  static linking.  As a result, the Condor Team must either patch
  these libraries or use older versions.  Kerberos is one
  example.  Linking these libraries dynamically will simplify
  this process.

- In the event of a security update to a library Condor uses,
  users must download an update from Condor or build Condor
  themselves against the newer library.  If Condor instead linked
  these libraries in dynamically and could use the system
  library, Condor would automatically benefit from an update
  along with other programs when the shared library was updated.

- Because each Condor binary contains a copy of the library,
  additional space is used both on disk and in memory.  A dynamic
  library can be shared between different binaries on disk and in
  memory, reducing space and memory usage.  For users sending
  Condor binaries across the network, perhaps using a network
  file system or glide-in, smaller binaries will reduce network
  traffic and potentially speed starting Condor.

Our proposal is to dynamically link in all of the third party
libraries Condor uses.  However, Condor links in a number of
unusual libraries that many systems do not necessarily have.  To
simplify installation, Condor would include dynamic copies of
these libraries in its install. Condor would default to using
these local copies, but a local administrator could replace them or
simply remove them to have Condor use the system-wide libraries.
Our goal is for Condor to locate libraries automatically using
the RPATH.  It should not be necessary for users to be aware of
the details, Condor will not modify any LD_LIBRARY_PATHs, nor
will Condor use wrapper scripts to set LD_LIBRARY_PATHs.

A small number of "essential" libraries will not be included,
likely starting with OpenSSL.  We will select libraries present
on almost all modern systems that are also core to Condor's
security.  Condor will instead look for a system-wide install of
the library.  These libraries are core to the security of Condor
and many other programs, so keeping up with security updates is
essential.  By using the system-wide versions of the library,
Condor will be able to take advantage of security updates at the
same time the rest of the system is updated, reducing the window
between the library being updated and Condor using the updated
library.

It is our belief that such a change will be transparent to almost
all of our users while bringing them a security improvement.

There will be additional complications:

- Users repackaging or redistributing Condor's binaries will face
  additional complexity.  This would include users making custom
  glide-in distributions.  There will be additional files to
  include the package.

- In the event of problems, shared libraries can complicate
  debugging.  It becomes necessary to identify which libraries
  are being used, which could vary for a variety of reasons.
  When libraries not packaged with Condor are used, it is not
  feasible for the Condor Team to test all possible permutations.

-- 
Alan De Smet                              Condor Project Research
adesmet@xxxxxxxxxxx                http://www.cs.wisc.edu/condor/