Download Copy a Sharepoint document library to another site along with the column structure and metadata

Copy a Sharepoint document library to
another site along with the column
structure and metadata
15th June 2021
In this article we take a detailed look at how we can copy one or more sharepoint document libraries to
another site and include all the custom columns and values.
Prerequisites
You may need to configure a couple of things up front.
1: Enable custom scripting
In order to export a list or library as a provisioning template you may need to Enable Custom scripting,
for this use the following powershell command
Set-SPOsite https://YourTenant.sharepoint.com/sites/YOURSITEHERE -DenyAddAndCustomizePages
0
2: Set the script user account to be a Term Store
administrator
To do this navigate to Sharepoint Admin panel, there on the left you will see [Content Services] expand
this and select [Term Store].
On the right choose the [Admins] section and click [edit], from there add the script user account as an
administrator and Save
Export one or more document librarys to a provisioning
template
Use the script below to export the specified library to a Provisioning template. This will include structure
of all columns.
param (
[string]$username ="your user email",
[string]$password ="your password"
)
#Config Variables
$SiteURL = "https://yourtenant.sharepoint.com/sites/youSourceSite"
$TemplateFile = "c:\temp\TemplateList.xml"
#Comma seperated string of your document libraries to export
$ListNames ="Invoices,SitePhotos"
$ListArray =$ListNames.Split(",")
#Connect to PNP Online
$creds2 = (New-Object System.Management.Automation.PSCredential $username,(ConvertTo-Secur
eString $password -AsPlainText -Force))
Connect-PnPOnline -Url $SiteURL
-Credentials $creds2 #-UseWebLogin
#Get the List schema as Template and export to a File
$Templates = Get-PnPProvisioningTemplate -OutputInstance -Handlers Lists
$ListTemplate = $Templates.Lists | Where-Object { $_.Title -in $ListArray }
$Templates.Lists.Clear()
foreach($li in $ListTemplate)
{
$Templates.Lists.Add($li)
}
Save-PnPProvisioningTemplate -InputInstance $Templates -Out $TemplateFile
Import the library template to a different Sharepoint site
This next script imports our new template to the target site resulting in the target library having the
imported column definitions.
param (
[string]$username ="your user email",
[string]$password ="your password"
)
#Config Variables
$SiteURL = "https://yourtennant.sharepoint.com/sites/yourTargetSite"
$TemplateFile = "C:\Temp\TemplateList.xml"
#Connect to PNP Online
$creds2 = (New-Object System.Management.Automation.PSCredential $username,(ConvertTo-Secur
eString $password -AsPlainText -Force))
Connect-PnPOnline -Url $SiteURL
-Credentials $creds2 #-UseWebLogin
Write-Host "Creating List from Template..."
Apply-PnPProvisioningTemplate -Path $TemplateFile
Copy the library document files
So far we have created and exported a provisioning template of the required Document library, then
imported that template into our target site. With this complete we now have a new empty document
library of the required structure in our target site. All that remains for us to do is to import the files from
our source site into the new library, ideally this should also populate the custom field column values.
How to identify the user defined columns in a document
library
Identifying the user defined columns in a list or document library is not straight forward, the solution I
use below appears to work for my needs and indeed correctly identifies my custom columns but I am
sure there is room for improvement.
The method I have used relies on testing a combination of field properties to filter out the unwanted
columns. In the code snippet below we use CanBeDeleted, FromBaseType and InternalName, at the
point we hit the try/catch block we have one of our custom columns and we then copy the value from
the source to the target column.
#Copy the user defined Custom fields from Source to Target as best we can! (mo
dify as required)
foreach ($field in $TargetFieldList.Fields)
{
if( ($field.CanBeDeleted) -and (!$field.FromBaseType) -and (!$field.Intern
alName.StartsWith("_")) -and ($field.InternalName -ne "Combine") -and ($field.InternalName
-ne "RepairDocument") )
{
try{
#Write-Host $field.InternalName
$TargetListItem[$field.InternalName] = $SourceListItem[$field.Inte
rnalName]
}
catch{
Write-Host $field.InternalName
write-host -f Red "Error Setting Custom File properties!" $_.Excep
tion.Message
}
}
}
Finally, here is the complete working powershell script to copy document library files from source site
to target site and include custom colum values.
param (
[string]$username ="your email account",
[string]$password ="your passwrord"
)
#Reference SharePoint CSOM Assemblies
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\IS
API\Microsoft.SharePoint.Client.dll"
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\IS
API\Microsoft.SharePoint.Client.Runtime.dll"
Function CopyFilesWithMetadata
{
param
(
[Parameter(Mandatory=$true)] [Microsoft.SharePoint.Client.Folder] $SourceFolder,
[Parameter(Mandatory=$true)] [Microsoft.SharePoint.Client.Folder] $TargetFolder,
[Parameter(Mandatory=$true)] [System.Object] $TargetFieldList
)
Try {
#Get all Files from the source folder
$SourceFilesColl = $SourceFolder.Files
$SourceFolder.Context.Load($SourceFilesColl)
$SourceFolder.Context.ExecuteQuery()
#Iterate through each file and copy
Foreach($SourceFile in $SourceFilesColl)
{
#Get the source file relative url
$FileInfo = [Microsoft.SharePoint.Client.File]::OpenBinaryDirect($SourceFolder
.Context, $SourceFile.ServerRelativeUrl)
#Get the Target File relative url
$TargetFileURL = $TargetFolder.ServerRelativeUrl+"/"+$SourceFile.Name
[Microsoft.SharePoint.Client.File]::SaveBinaryDirect($TargetFolder.Context, $T
argetFileURL, $FileInfo.Stream,$True)
#Get Source Field collection
$SourceListItem = $SourceFile.ListItemAllFields
$SourceFolder.Context.Load($SourceListItem)
$SourceFolder.Context.ExecuteQuery()
#Get the new Target file created
$TargetFile = $TargetFolder.Context.Web.GetFileByServerRelativeUrl($TargetFile
URL)
#Get the Target field collection
$TargetListItem = $TargetFile.ListItemAllFields
#Set Basic Metadata values from the source to the Target
$Author =$TargetFolder.Context.web.EnsureUser($SourceListItem["Author"].Email)
$TargetListItem["Author"] = $Author
$Editor =$TargetFolder.Context.web.EnsureUser($SourceListItem["Editor"].Email)
$TargetListItem["Editor"] = $Editor
$TargetListItem["Created"] = $SourceListItem["Created"]
$TargetListItem["Modified"] = $SourceListItem["Modified"]
#$TargetListItem["Project"] = $SourceListItem["Project"]
#Copy the user defined Custom fields from Source to Target as best we can! (mo
dify as required)
foreach ($field in $TargetFieldList.Fields)
{
if( ($field.CanBeDeleted) -and (!$field.FromBaseType) -and (!$field.Intern
alName.StartsWith("_")) -and ($field.InternalName -ne "Combine") -and ($field.InternalName
-ne "RepairDocument") )
{
try{
#Write-Host $field.InternalName
$TargetListItem[$field.InternalName] = $SourceListItem[$field.Inte
rnalName]
}
catch{
Write-Host $field.InternalName
write-host -f Red "Error Setting Custom File properties!" $_.Excep
tion.Message
}
}
}
#Save/Commit the updates
try{
$TargetListItem.Update()
$TargetFolder.Context.ExecuteQuery()
}
catch{
write-host -f Red "Error Updating File custom properties!" $_.Exception.Me
ssage
}
Write-host -f Green "Copied File '$($SourceFile.ServerRelativeUrl)' to '$Targe
tFileURL'"
}
#Process Sub Folders
$SubFolders = $SourceFolder.Folders
$SourceFolder.Context.Load($SubFolders)
$SourceFolder.Context.ExecuteQuery()
Foreach($SubFolder in $SubFolders)
{
If($SubFolder.Name -ne "Forms")
{
#Prepare Target Folder
$TargetFolderURL = $SubFolder.ServerRelativeUrl -replace $SourceLibrary.Ro
otFolder.ServerRelativeUrl, $TargetLibrary.RootFolder.ServerRelativeUrl
Try {
$Folder=$TargetFolder.Context.web.GetFolderByServerRelativeUrl($Ta
rgetFolderURL)
$TargetFolder.Context.load($Folder)
$TargetFolder.Context.ExecuteQuery()
}
catch {
#Create Folder
if(!$Folder.Exists)
{
$TargetFolderURL
$Folder=$TargetFolder.Context.web.Folders.Add($TargetFolderURL
)
$TargetFolder.Context.Load($Folder)
$TargetFolder.Context.ExecuteQuery()
Write-host "Folder Added:"$SubFolder.Name -f Yellow
}
}
#Call the function recursively
CopyFilesWithMetadata -SourceFolder $SubFolder -TargetFolder $Folder -Targ
etFieldList $Folder
}
}
}
Catch {
write-host -f Red "Error Copying File!" $_.Exception.Message
}
}
#Set Parameter values
$SourceSiteURL="https://yourTenant.sharepoint.com/sites/YourSourcesite"
$TargetSiteURL="https://yourTenant.sharepoint.com/sites/YourTargetsite"
$SourceLibraryName="SourceLibraryName"
$TargetLibraryName="TargetLibraryName"
$credentials = (New-Object System.Management.Automation.PSCredential $username,(ConvertToSecureString $password -AsPlainText -Force))
$cred2 = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username, $cr
edentials.password)
#Setup the contexts
$SourceCtx = New-Object Microsoft.SharePoint.Client.ClientContext($SourceSiteURL)
$SourceCtx.Credentials = $cred2 #$Credentials
$TargetCtx = New-Object Microsoft.SharePoint.Client.ClientContext($TargetSiteURL)
$TargetCtx.Credentials = $cred2 #$Credentials
#Get the source library and Target Libraries
$SourceLibrary = $SourceCtx.Web.Lists.GetByTitle($SourceLibraryName)
$SourceCtx.Load($SourceLibrary)
$SourceCtx.Load($SourceLibrary.RootFolder)
$TargetLibrary = $TargetCtx.Web.Lists.GetByTitle($TargetLibraryName)
$TargetCtx.Load($TargetLibrary)
$TargetCtx.Load($TargetLibrary.RootFolder)
$TargetCtx.Load($TargetLibrary.Fields)
$TargetCtx.ExecuteQuery()
#Call the copy function
CopyFilesWithMetadata -SourceFolder $SourceLibrary.RootFolder -TargetFo