Bug 515329 - Multiple mods in one operation can result in an inconsistent replica

https://bugzilla.redhat.com/show_bug.cgi?id=515329
Resolves: bug 515329
Bug Description: Multiple mods in one operation can result in an inconsistent replica
Reviewed by: nkinder, nhosoi (Thanks!)
Branch: master
Fix Description: When a master is processing a list of mod ops in a single
modify operation, all of which have the same CSN, the server processes them
in the order specified.  When a replica processes this same operation, and
has to merge this operation with other operations, it essentially processes
the operations in CSN order, and loses the original order of the ops within
each modify operation.  The function that initially receives the
replicated op, entry_apply_mods_wsi(), has the original order, and enforces
this order by incrementing the until now unused subsequence number in the
CSN for each mod op.  However, it only does this if the operation has
been replicated, and if the original CSN does not already have a
subsequence number.
One side effect is that the subsequence number is stored with the state
information in the database, which does not appear to affect anything else,
and things like state information purging still work correctly.
I've verfied that the subsequence number does not end up in in the changelog,
nor the RUV or other externally used CSNs.
Platforms tested: RHEL6 x86_64
Flag Day: no
Doc impact: no

ldap/servers/slapd/csn.c

@@ -212,6 +212,11 @@ PRUint16 csn_get_seqnum(const CSN *csn)
     return csn->seqnum;
 }
 
+PRUint16 csn_get_subseqnum(const CSN *csn)
+{
+    return csn->subseqnum;
+}
+
 time_t csn_get_time(const CSN *csn)
 {
 	if(csn==NULL)

ldap/servers/slapd/entrywsi.c

@@ -748,6 +748,11 @@ entry_apply_mods_wsi(Slapi_Entry *e, Slapi_Mods *smods, const CSN *csn, int urp)
 {
 	int retVal= LDAP_SUCCESS;
 	LDAPMod *mod;
+	CSN localcsn;
+
+	if (csn) {
+		localcsn = *csn; /* make a copy */
+	}
 
 	LDAPDebug( LDAP_DEBUG_TRACE, "=> entry_apply_mods_wsi\n", 0, 0, 0 );
 
@@ -756,7 +761,15 @@ entry_apply_mods_wsi(Slapi_Entry *e, Slapi_Mods *smods, const CSN *csn, int urp)
 	{
 		if(csn!=NULL)
 		{
-			retVal= entry_apply_mod_wsi(e, mod, csn, urp);
+			retVal= entry_apply_mod_wsi(e, mod, &localcsn, urp);
+			/* use subsequence to guarantee absolute ordering of all of the
+			   mods in a set of mods, if this is a replicated operation,
+			   and the csn doesn't already have a subsequence
+			   if the csn already has a subsequence, assume it was generated
+			   on another replica in the correct order */
+			if (urp && (csn_get_subseqnum(csn) == 0)) {
+				csn_increment_subsequence(&localcsn);
+			}
 		}
 		else
 		{

ldap/servers/slapd/slapi-private.h

@@ -190,6 +190,7 @@ void csn_set_seqnum(CSN *csn, PRUint16 seqnum);
 ReplicaId csn_get_replicaid(const CSN *csn);
 time_t csn_get_time(const CSN *csn);
 PRUint16 csn_get_seqnum(const CSN *csn);
+PRUint16 csn_get_subseqnum(const CSN *csn);
 char *csn_as_string(const CSN *csn, PRBool replicaIdOrder, char *ss); /* WARNING: ss must be CSN_STRSIZE bytes, or NULL. */
 int csn_compare(const CSN *csn1, const CSN *csn2);
 time_t csn_time_difference(const CSN *csn1, const CSN *csn2);