Re: [HTCondor-users] condor_adstash behaviour when dealing with big amounts of jobs

[Maria Alandes Pradillo <maria.alandes.pradillo@xxxxxxx>]

Hi Jason,

 

Yes, I have monthly indexes. Openseach colleagues have told me that If you specify a document ID (_id) that already exists, OpenSearch will overwrite the existing document with the new one. If you donât specify an ID, OpenSearch will automatically generate a new one. This means it will create a new document, even if the content is the same as an existing document. My understanding is that condor_adstash always generates an ID based on {GlobalJobId}#{RecordTime}, so it will overwrite. Itâs not clear to me if this is only an Index level. I will ask.

 

My understanding is that this only applies for a single index. If you send a doc with an _id that matches another doc in the current month's index, it gets overwritten. But if last month's index has a doc with that _id, Opensearch doesn't know about it, and the doc ends up duplicated in this month's index as well. I would like to be proven wrong about this, though!

 

You are right, confirmed by the Opensearch colleagues: itâs per index.

 

Cheers,

Maria