4 Steps to download Microsoft Sharepoint Document Library recursively

I stumble across this problem when we try to decommissioning Microsoft Sharepoint. We had a huge document library and it is not possible to copy them from explorer view, so the solution is to use PowerShell script to do this automagically.

Step 1. Define the DLL that is required.

This is done through the following code snippets. It is crucial to have the 2 DLL to allow the copy function to work. The script will use the credential of the user login into the machine and executing the script. This removes the complexity having to enter the sharepoint credential into the script.

# Load the SharePoint 2013 .NET Framework Client Object Model libraries. # 
[void][Reflection.Assembly]::LoadFrom("c:\Microsoft.SharePoint.Client.dll")
[void][Reflection.Assembly]::LoadFrom("c:\Microsoft.SharePoint.Client.Runtime.dll")

Step 2. Define the sharepoint site URL and the Document Library repository

You can simply enter the sharepoint URL by replacing the following $serverURL variable. Enter the document library by replacing the $DocumentLibrary variable and don’t forget to define the destination folder.

$serverURL = “http://sharepoint.url/sites/sitename”
$destination = "C:\temp\"
$DocumentLibary = "Document Library Name"

Step 3. Choose whether you only want specific folder to be downloaded from the Document Library

Change the folder name that you are interest in downloading, in the following example we are only interested in downloading folder “Payments” and all the folder underneath it.

function Parse-Lists ($Lists)
{
$clientContext.Load($Lists)
$clientContext.Load($Lists.RootFolder.Folders)
$clientContext.ExecuteQuery()
    
    foreach ($Folder in $Lists.RootFolder.Folders)
        {
            if ($Folder.name -eq "Payments"){   #onlydownload selected folder
                recurse $Folder
            }
        }

}

Step 4. Execute the script via PowerShell window or from Command line.

To execute the script via command line you can execute the following Powershell command, with the assumption the name of the powershell script is “scriptname.ps1”

C:\Powershell.exe scriptname.ps1

Here are the full script to download the Sharepoint Document library, be careful the script will download the entire document library recursively, so please make sure you check Step 3 above. With great power comes great responsibility.

# Load the SharePoint 2013 .NET Framework Client Object Model libraries. # 
[void][Reflection.Assembly]::LoadFrom("c:\Microsoft.SharePoint.Client.dll")
[void][Reflection.Assembly]::LoadFrom("c:\Microsoft.SharePoint.Client.Runtime.dll")
Clear-Host

$serverURL = “http://sharepoint.url/sites/sitename”
#$siteUrl = $serverURL+"/documents”
$destination = "C:\temp\"
$DocumentLibary = "Document Library Name"
$downloadEnabled = $true
$versionEnabled = $false

# Authenticate with the SharePoint Online site. # 
#$username = ""
#$Password = ""
#$securePassword = ConvertTo-SecureString $Password -AsPlainText -Force  

$clientContext = New-Object Microsoft.SharePoint.Client.ClientContext($serverURL) 
#$credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username, $securePassword) 
#$clientContext.Credentials = $credentials 
if (!$clientContext.ServerObjectIsNull.Value) 
{ 
    Write-Output "Connected to SharePoint Online site: '$serverURL'"
} 


function HTTPDownloadFile($ServerFileLocation, $DownloadPath)
{
#Download the file from the version's URL, download to the $DownloadPath location
    $webclient = New-Object System.Net.WebClient
    $webclient.credentials = $credentials
    Write-Output "Download From ->'$ServerFileLocation'"
    Write-Output "Write to->'$DownloadPath'"
    $webclient.Headers.Add("X-FORMS_BASED_AUTH_ACCEPTED", "f")
    $webclient.DownloadFile($ServerFileLocation,$DownloadPath)
}

function DownloadFile($theFile, $DownloadPath)
{
    $fileRef = $theFile.ServerRelativeUrl;
    Write-Host $fileRef;
    $fileInfo = [Microsoft.sharepoint.client.File]::OpenBinaryDirect($clientContext, $fileRef);
    $fileStream = [System.IO.File]::Create($DownloadPath)
    $fileInfo.Stream.CopyTo($fileStream);
    $fileStream.Close()
}

function Get-FileVersions ($file, $destinationFolder)
{
    $clientContext.Load($file.Versions)
    $clientContext.ExecuteQuery()
    foreach($version in $file.Versions)
    {
        #Add version label to file in format: [Filename]_v[version#].[extension]
        $filesplit = $file.Name.split(".") 
        $fullname = $filesplit[0] 
        $fileext = $filesplit[1] 
        $FullFileName = $fullname+"_v"+$version.VersionLabel+"."+$fileext           

        #Can't create an SPFile object from historical versions, but CAN download via HTTP
        #Create the full File URL using the Website URL and version's URL
        $ServerFileLocation = $siteUrl+"/"+$version.Url

        #Full Download path including filename
        $DownloadPath = $destinationfolder+"\"+$FullFileName
        
        if($downloadenabled) {HTTPDownloadFile "$ServerFileLocation" "$DownloadPath"}

    }
}

function Get-FolderFiles ($Folder)
{
    $clientContext.Load($Folder.Files)
    $clientContext.ExecuteQuery()

    foreach ($file in $Folder.Files)
        {

            $folderName = $Folder.ServerRelativeURL
            $folderName = $folderName -replace "/","\"
            $folderName = $destination + $folderName
            $fileName = $file.name
            $fileURL = $file.ServerRelativeUrl
            
                
            if (!(Test-Path -path $folderName))
            {
                $dest = New-Item $folderName -type directory 
            }
                
            Write-Output "Destination -> '$folderName'\'$filename'"

            #Create the full File URL using the Website URL and version's URL
            $ServerFileLocation = $serverUrl+$file.ServerRelativeUrl

            #Full Download path including filename
            $DownloadPath = $folderName + "\" + $file.Name
                    
            #if($downloadEnabled) {HTTPDownloadFile "$ServerFileLocation" "$DownloadPath"}
            if($downloadEnabled) {DownloadFile $file "$DownloadPath"}

            if($versionEnabled) {Get-FileVersions $file $folderName}
            
    }
}


function Recurse($Folder) 
{
       
    $folderName = $Folder.Name
    $folderItemCount = $folder.ItemCount

    Write-Output "List Name ->'$folderName'"
    Write-Output "Number of List Items->'$folderItemCount'"

    if($Folder.name -ne "Forms")
        {
            #Write-Host $Folder.Name
            Get-FolderFiles $Folder
        }
 
    Write-Output $folder.ServerRelativeUrl
 
    $thisFolder = $clientContext.Web.GetFolderByServerRelativeUrl($folder.ServerRelativeUrl)
    $clientContext.Load($thisFolder)
    $clientContext.Load($thisFolder.Folders)
    $clientContext.ExecuteQuery()
            
    foreach($subfolder in $thisFolder.Folders)
        {
            Recurse $subfolder  
        }       
}


function Parse-Lists ($Lists)
{
$clientContext.Load($Lists)
$clientContext.Load($Lists.RootFolder.Folders)
$clientContext.ExecuteQuery()
    
    foreach ($Folder in $Lists.RootFolder.Folders)
        {
            if ($Folder.name -eq "Payments"){   #onlydownload selected folder
                recurse $Folder
            }
        }

}

$rootWeb = $clientContext.Web
$LibLists = $rootWeb.lists.getByTitle($DocumentLibary)
$clientContext.Load($rootWeb)
$clientContext.load($LibLists)
$clientContext.ExecuteQuery()

Parse-Lists $LibLists

 

Please let me know if the above script is useful, feel free to subscribe to my blog, share this script or ask me any questions related to the script.