Managed document not working in MarkLogic 10 after xdmp:document-insert - database-migration

Question

First time manage document using dls:document-insert-and-manage
Update the same document using xdmp:document-insert
Document get lost from the dls latest version collection cts:search(/scopedIntervention/id , dls:documents-query())
First time manage document
<scopedIntervention>
<id>someId12345</id>
<scopedInterventionName>
First Name
</scopedInterventionName>
<forTestOnly>
true
</forTestOnly>
<inactive>
true
</inactive>
</scopedIntervention>)```
**Document inserted with versioning**
Verify document is present in latest documents collection
cts:search(/scopedIntervention/id , dls:documents-query())
Document present in managed latest collection
Update the same document
<scopedIntervention>
<id>someId12345</id>
<scopedInterventionName>
Updated Name
</scopedInterventionName>
<forTestOnly>
true
</forTestOnly>
<inactive>
true
</inactive>
</scopedIntervention>)```
**Update document to same URI using xdmp:document-insert**
Again verify document is present or NOT in latest documents collection
cts:search(/scopedIntervention/id , dls:documents-query())
Document NOT present in managed latest collection (lost from collection)
After applying DLS package using following upgrade step, the same document shows in the list
```xquery version "1.0-ml";
import module namespace dls = "http://marklogic.com/xdmp/dls"
at "/MarkLogic/dls.xqy";
dls:set-upgrade-status(fn:false()),
dls:start-upgrade(),
fn:doc("http://marklogic.com/dls/upgrade-task-status.xml"),
dls:latest-validation-results(),
dls:set-upgrade-status(fn:true())```

Answer 1

Update the same document using xdmp:document-insert
You are most likely removing the DLS Latest collection at this step. Further, version history is not preserved when you do this.
Instead of using xdmp:document-insert you should use dls:document-checkout-update-checkin .

Answer 2

Please read to the end -- if you did NOT do a DLS Upgrade on an upgraded ML version - STOP NOW and follow the upgrade instructions. Not doing so will leave DLS in a unstable state and anything else you do will make things much harder to repair.
+1 Rob. #IAM, reguardless of if it 'worked' or appeared to 'work' in V7, dls was not designed to handle the case you describe. DLS architecture depends on encapsulating all changes to documents within the checkin/checkout semantics. Bypassing that, you might as well bypass DLS entirely because it wont work. The fact that it was 'working' in V7 is a misnomer, it may have not behaved incorrectly in ways that your application cared about, or your code may have coincidentally done sufficiently similar work as the internals. You might get lucky and find a way to do so again, but I encourage you to consider how to work within the define behaviour of the library, or to refactor out those parts of your code that are not 'DLS Friendly' to operate between checkout/checkin windows -- not all updates have to be the checkout-update-checkin -- you can checkout -- do whatever -- then checkin.
As a migration workaround you MIGHT be able to make use of the upgrade functions added to dls on an ongoing basis.
See https://docs.marklogic.com/dls:start-upgrade
In V9 (I believe), significant non-backwards compatible changes were made to DLS internals that require running this code. one time
The assumption was as in-place update from prior DLS to current. However the code may also happen to work on an ongoing basis, depending on the details of exactly what your application code is doing that the DLS code doesn't know about.
The 'new' DLS code adds an internal collection to optimize the common case of searching for 'latest' documents -- if that is dropped then those documents will not show up on DLS searches (for 'latest').
You mention your code is 'migration scripts' --> If these are migrating from V7 to V10 then you could run your code before the V10 update, then run the V10 update then run the dls-upgrade. After that the documents should be in good shape -- as long as you don't do anything else that is not defined behaviour for managed documents.