Re: [Condor-devel] Wisdom needed: should_transfer_files = IF_NEEDED & when_to_transfer_output = ON_EXIT_OR_EVICT

[Dan Bradley <dan@xxxxxxxxxxxx>]

Hi Matt,

I think the reason that IF_NEEDED + ON_EXIT_OR_EVICT is forbidden is 
that ON_EXIT_OR_EVICT requires sandboxing of the job's working files.  
Each attempt to run the job that is evicted is treated like a 
transaction that atomically updates the spooled files (or fails to 
update and is rolled back).  Only a successful run is allowed to update 
files outside of the sandbox.  Without file transfer, those semantics 
are not really possible in vanilla universe.  Since IF_NEEDED does not 
guarantee that file transfer will happen, it is incompatible with 
ON_EXIT_OR_EVICT.

--Dan

On 6/30/11 8:46 AM, Matthew Farrellee wrote:
> condor_submit is adamant that IF_NEEDED + ON_EXIT_OR_EVICT is bad and 
> must be avoided. I want to know why.
>
> Digging through the repo history I've found only that in Apr 2003 it 
> took only 3 days from the addition of IF_NEEDED to the inclusion of 
> the check.
>
> What do you remember about this issue? I hope the knowledge isn't lost 
> to time.
>
> Best,
>
>
> matt
>
> --
>
> commit 85d38d72aa8703c529d54a52f19fce6c622d6282
> Author: wright <wright>
> Date:   Tue Apr 29 22:47:12 2003 +0000
>
>     added code to check for the inconsistent setting of both
>     when_to_transfer_output == FTO_ON_EXIT_OR_EVICT &&
>     should_transfer_files == STF_IF_NEEDED.  in this case, submit now
>     prints an error message and aborts...
>
> diff --git a/src/condor_submit.V6/submit.C 
> b/src/condor_submit.V6/submit.C
> index 3d51dde..e6bc811 100644
> --- a/src/condor_submit.V6/submit.C
> +++ b/src/condor_submit.V6/submit.C
> @@ -1632,6 +1632,24 @@ SetNewTransferFiles( bool in_files_specified, 
> bool out_files_specified )
>                 }
>         }
>
> +       if( found_it && when_output == FTO_ON_EXIT_OR_EVICT &&
> +                           should_transfer == STF_IF_NEEDED ) {
> +                       // error, these are incompatible!
> +               err_msg = "\nERROR: \"when_to_transfer_output = 
> ON_EXIT_OR_EVICT\" "
> +                       "and \"should_transfer_files = IF_NEEDED\" are 
> incompatible.  "
> +                       "The behavior of these two settings together 
> would produce "
> +                       "incorrect file access in some cases.  Please 
> decide which "
> +                       "one of those two settings you're more 
> interested in. "
> +                       "If you really want \"IF_NEEDED\", set "
> +                       "\"when_to_transfer_output = ON_EXIT\".  If 
> you really want "
> +                       "\"ON_EXIT_OR_EVICT\", please set 
> \"should_transfer_files = "
> +                       "YES\".  After you have corrected this 
> incompatibility, "
> +                       "please try running condor_submit again.\n";
> +               print_wrapped_text( err_msg.Value(), stderr );
> +               DoCleanup(0,0,NULL);
> +               exit( 1 );
> +       }
> +
>                 // if we found the new syntax, we're done, and we 
> should now
>                 // add the appropriate ClassAd attributes to the job.
>         if( found_it ) {
> _______________________________________________
> Condor-devel mailing list
> Condor-devel@xxxxxxxxxxx
> https://lists.cs.wisc.edu/mailman/listinfo/condor-devel