Tuesday, April 17, 2012

How to find the full path of a file stuck in DFSR backlog

In this post I will describe the method I use to find the full path of a file replicated using Windows DFSR. Usually, I use this method when I have one or more files stuck in the backlog. As you have have probably discovered, issuing the command DFSRDIAG BACKLOG will output the list of the files currently being replicated by the replication engine:
DFSRDIAG BACKLOG /sendingmember:member1 /rgname:dfsr_replica /rfname:data_to_replicate /verbose
[INFO] Computer Name: member1
[INFO] Computer DNS: member1.yourdomain.com
[INFO] Domain Name: yourdomain
[INFO] Domain DNS: yourdomain.com
[INFO] Site Name: site1
[INFO] Computer Name: member1
[INFO] Computer DNS: member1.yourdomain.com
[INFO] Domain Name: yourdomain
[INFO] Domain DNS: yourdomain.com
[INFO] Site Name: site1
[INFO] Connected to WMI services on computer: member1.yourdomain.com
[INFO] Issuing query: SELECT * FROM DfsrReplicationGroupConfig WHERE ReplicationGroupName="dfsr_replica"
[INFO] Found DfsrReplicationGroupConfig object, guid: 494C997F-4E2A-4392-BC69-27E9E6757BA0
[INFO] Issuing query: SELECT * FROM DfsrReplicatedFolderConfig WHERE ReplicationGroupGuid="494C997F-4E2A-4392-BC69-27E9E6757BA0" AND ReplicatedFolderName="data_to_replicate"
[INFO] Found DfsrReplicatedFolderConfig object, guid: 0569B726-2B9H-4DB4-B09F-219217670980
[INFO] Object Path: DfsrReplicatedFolderInfo.ReplicatedFolderGuid="0569B726-2B9H-4DB4-B09F-219217670980"
[INFO] Invoke GetVersionVector() method on member1.yourdomain.com
[INFO] Connected to WMI services on computer: member1.yourdomain.com
[INFO] Invoke GetOutboundBacklogFileCount() method on member1.yourdomain.com
[INFO] Invoke GetOutboundBacklogFileIdRecords() method on member1.yourdomain.com

Member  Backlog File Count: 3
Backlog File Names (first 3 files)
     1. File name: file1.txt
     2. File name: file2.docx
     3. File name: file3.html

[INFO] Execution Time: 0 seconds
Operation Succeeded
The problem here is that if you, like me, have tens or hundreds of files with the same filename but in different locations, you won't be able to tell which is the full path of the file stuck in the backlog. For instance, in the output shown above, file1.txt is stuck there from many weeks.

To find out the path of the parent folder, the following actions must be performed.

First of all increase the DFSR debug level to 5 by issuing the following command:
wmic /namespace:\\root\microsoftdfs path dfsrmachineconfig set debuglogseverity=5
Now browse to c:\windows\debug on your DFSR member and open with notepad the last debug logfile, usually named Dfsr00100.log.

Make a recursive search for file1.txt until you find the problematic file. In my case the error code I have found beside the file name is 'A failure was reported by the remote partner' and 'The file meta data is not synchronized with the file system'.

The fact of having activated a debug level of 5 will make the debug log show a lot of information about the files it is trying to replicate. In particular the following fileds are shown:
  • gvsn, which is the modified file record
  • uid, which is the original file record
  • parent, which is the ID of the folder holding the file
  • csId, which is the GUID of the replicated root folder
  • createTime, which is the creation time of the original file
 A sample entry for file1.txt in Dfsr00100.log could be:
+    gvsn {9EBE0A27-8AA9-4263-B942-DA9A92F30671}-v305329801
+    uid {9EBE0A27-8AA9-4263-B942-DA9A92F30671}-v240880
+    parent {9EBE0A27-8AA9-4263-B942-DA9A92F30671}-v240840
+    fence 16010101 00:00:00.000 
+    clock 20120206 10:43:16.102
+    createTime 20070522 14:44:46.648 GMT
+    csId {0569B726-2B9H-4DB4-B09F-219217670980}
+    hash 00000000-00000000-00000000-00000000
+    similarity 00000000-00000000-00000000-00000000
+    name file1.txt-{9EBE0A27-8AA9-4263-B942-DA9A92F30671}-v28623516
Unfortunately those ID won't help you much if you can't translate them to human readable paths.

Happily enough we can use WMI to query the DFSR ESE for the transaltion of those records.

To find the parent folder path, the following wmic command can be run from the command line:
Wmic.exe /namespace:\\root\microsoftdfs path dfsridrecordinfo.Uid="{9EBE0A27-8AA9-4263-B942-DA9A92F30671}-v240840" call getfullfilepath
The output will tell us the exact folder path:
Executing (\\member1\root\microsoftdfs:DfsrIdRecordInfo.Uid="{9EBE0A27-8AA9-4263-B942-DA9A92F30671}-v240840")->getfullfilepath()
Method execution successful.
Out Parameters:
instance of __PARAMETERS
{
FullPath = "T:\\data_to_replicate\\sufolder1\\subfolder2\\subfolder3\\subfolder4\\Config";
ReturnValue = 0;
}
At this point we can run wmic against the uid of file1.txt
Wmic.exe /namespace:\\root\microsoftdfs path dfsridrecordinfo.Uid="{9EBE0A27-8AA9-4263-B942-DA9A92F30671}-v240880" call getfullfilepath
Here's a sample output:
Executing (\\member1\root\microsoftdfs:DfsrIdRecordInfo.Uid="{9EBE0A27-8AA9-4263-B942-DA9A92F30671}-v240880")->getfullfilepath()
Method execution successful.
Out Parameters:
instance of __PARAMETERS
{
FullPath = "T:\\data_to_replicate\\sufolder1\\subfolder2\\subfolder3\\subfolder4\\Config\\file1.txt-{9EBE0A27-8AA9-4263-B942-DA9A92F30671}-v28623516";
ReturnValue = 0;
}
Here we are! We have found that the path of file we were looking for in our jungle of subfolders is:
T:\data_to_replicate\sufolder1\subfolder2\subfolder3\subfolder4\Config\\file1.txt
Now we can take the appropriate corrective action to remove this file from the backlog, even if no easy solutions exist. You can try to remove the file from both the replicated folders, or restart the 'DFSR Replication' service.

Whatever you do, remember to lower the debug level to 4 once you have finished your troubleshooting.

I hope this post has helped you in understanding how you can interact with the DFSR service by using dfsrdiag.exe and wmic.exe. Do not hesitate to comment and share with the community your personal experience with DFSR backlog troubleshooting.

7 comments:

  1. How in the world did you discover this?

    ReplyDelete
  2. Investigating this kind of issue is my job. I spent many hours on it trying to reverse engineering the process. Then, when I reached the limits of all the variuos dfs*.exe executables on my DFS system, I switched to WMI (which is the best tool for Windows admins when all others solutions have failed).

    ReplyDelete
  3. Do you know how to purge the backlog from a member?

    ReplyDelete
  4. After running the first command to show the backlog line #1 in my case is blank. Does this indicate that the file no longer exists?

    ReplyDelete
  5. Hi, I would like to know how to perform the basic troubleshooting related to backlog and replication issue.

    ReplyDelete
  6. Thank you for writing this! We had an issue where we needed to find the filepath of some accidentally deleted files. They were recovered using Recuva, but not the folder structure. I attempted this, but the circular logging erased the needed debug files. Missed it by 2 days! Doh!

    ReplyDelete
  7. When I run the first Wmic.exe command for with the UID for the parent I get "No instance available." Does this mean the directory doesn't exist? If so, how do i get this file ou of the backlog if it no longer exists?

    ReplyDelete

Related Posts Plugin for WordPress, Blogger...