| T O P I C R E V I E W |
| Phil Marcum |
Posted - 06/03/2011 : 12:48:58 PM Here's an odd request that I'm trying to script: Task: move all files located in spreadsheet to a remote server for possible deletion.
I have a spreadsheet containing thousands of files scattered across multiple servers that need to be moved to a folder located on a single server. The spreadsheet contains the full UNC of the file(s) on the source server. What I'd like is for the script to recreate the same UNC on the destination server and move the file into it. Maintaining the ACL's on the files would alse be nice but isn't necessary as the main goal at this point is to move the files from the designated locations and delete them shortly after. If necessary I can always rollback by manually copying the file back to its original location. My searches are fubar as I'm only getting hits on the import-csv cmdlet. Any responses are direction appreciated. |
| 27 L A T E S T R E P L I E S (Newest First) |
| Phil Marcum |
Posted - 07/19/2011 : 10:55:31 AM Freakin' AWESOME!!!! |
| Xenophane |
Posted - 07/18/2011 : 3:38:07 PM It seems to me that you are trying to pass multiple parameters to remove-item Something like this should be able to do it, remove -whatif when you are sure you really really want to do it :)
$paths = @() $paths = import-csv C:\temp\MySpreadsheetHere.csv Foreach ($path in $paths) { $($path.File) remove-item $($path.File) -force -whatif }
|
| Phil Marcum |
Posted - 07/18/2011 : 1:06:11 PM Error(s) thrown:
Remove-Item : A positional parameter cannot be found that accepts argument '\\myfilesrv3\e$\filemove\myserver\d$\myfiles\samplefile1.txt'. At D:\Projects\data\Delete\dlpdelete.ps1:9 char:31 + Write-Output "f" | remove-item <<<< $($path.File) $NewPath /I /O /Y + CategoryInfo : InvalidArgument: (:) [Remove-Item], ParameterBindingException + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand
Error is thrown on each file. |
| Phil Marcum |
Posted - 07/18/2011 : 10:12:32 AM Now that I've successfully copied the files I need to go in and delete each of them. I could use some help figuring out how this would work with the current script. Meaning there shouldn't be a "newserver" statement:
$newServer = "\\MyServerName\c$\Foldername"
When removed it throws all sorts of errors. It looks as though I need to remove every "NewServer" reference?
Am I missing something? |
| Xenophane |
Posted - 07/07/2011 : 06:40:59 AM BE carefull running the above script.. You are using remove-item instead of move-item |
| Phil Marcum |
Posted - 07/07/2011 : 01:18:53 AM No luck in getting this to copy the files and delete them using a single script and will try using a second script once the files have successfully copied. In looking at the script applied I wonder if it's possible to modify it to delete the same file that was copied earlier.
$paths = @() $paths = import-csv C:\temp\MySpreadsheetHere.csv $newServer = "\\MyServerName\c$\Foldername"
Foreach ($path in $paths) { $NewPath = "$newServer$($path.File.substring(1))" $($path.File) $NewPath Write-Output "f" | remove-item $($path.File) $NewPath /I /O /Y #Catch { Write-Output "Error: remove-item: $_">>c:\temp\Moved.log}
If ($?) {Write-Output "Success: remove-item $($path.File) -> $NewPath ">>c:\temp\Succeeded.log } else {Write-Output "Error: remove-item: $($path.File)">>c:\temp\Failed.log} }
Having it to write to a a separate would be nice for verification purposes. Am I missing something here?
Any responses appreciated. |
| Nobody |
Posted - 06/24/2011 : 4:21:58 PM I used xcopy because that's the only tool I could think of that would replicate your folder structure and keep the ACL as well as the various other parameters that xcopy offers. Oddly, powershell doesn't seem to have a tool on par with xcopy.
"Remove-Item path/to/somefile.txt" will only delete the file specified, not the entire folder structure. Play around with these commands and build some confidence in what they do.
use the -full parameter to get detailed command info and examples
Get-Help Remove-Item -full
|
| Phil Marcum |
Posted - 06/24/2011 : 3:22:35 PM Also looking at the "move-item" cmdlet as it seems to ensure that the entire UNC path and file is moved to the destination server which is what I want.
|
| Phil Marcum |
Posted - 06/24/2011 : 3:18:53 PM Stuck on the remove-item cmdlet as I need to ensure that the entire UNC path and file is copied to the destination server, which is covered. The other side is to remove the file and not the entire path from the source server as there are other files in that folder. |
| Nobody |
Posted - 06/24/2011 : 11:46:16 AM A move is nothing more than a copy and a delete. So you can just add in a Remove-Item line immediately after the xcopy, or after all files have copied, run through the foreach loop again and delete. Just be careful, if don't do any error handling, and xcopy goes sideways, it could delete all your files before they are copied.
Using Xenophane's script you can delete right after the success log entry, the log then delete if successful.
|
| Phil Marcum |
Posted - 06/24/2011 : 11:14:05 AM Got it and able to confirm that the complete folder and file structure is copied to the destination server. What I need to do now is modify the script to move the files to the destination server. Swapping to xcopy command with move isn't working. Am I missing something?
Tia |
| Phil Marcum |
Posted - 06/24/2011 : 11:08:09 AM Got it! Tested and confirmed that the script will copy the complete URL path and file from the source server(s) to the destination server. What I need to do now is modify it to "move" the files from the source to the destination. Tried unsuccessfully at switching the Xcopy commands with the Move command.
Any pointers? |
| Nobody |
Posted - 06/23/2011 : 11:57:34 AM As mentioned, my method requires a text file with one path per line, no column headers or any extra content. You can just "save as" .txt from excel, and make sure any column headers are removed.
Xenophane's method requires a csv file with a column header named 'file'.
|
| Xenophane |
Posted - 06/23/2011 : 02:16:00 AM Phil, The line containing the file paths in the csv file, has to contain the word "file" (Columnheader without the quotes) in the first line.
$paths = @()
$paths = import-csv C:\temp\MySpreadsheetHere.csv
$newServer = "\\MyServerName\c$\Foldername"
Foreach ($path in $paths) {
$NewPath = "$newServer$($path.File.substring(1))"
$($path.File)
$NewPath
}
You can just try to copy and paste the above into the console to see if you get anything (It just prints the path to the screen) |
| Phil Marcum |
Posted - 06/22/2011 : 3:50:31 PM Almost a month later and I'm just getting the green light to move forward with this project. Being that I need to move the files from one location to another server creating the same path I'm going to give "Nobody's" script a try. Here's my dilema: 1. I won't say that the script doesn't work, I just can't get it to work for me. Nobody's original powershell script called for the spreadsheet to be saved as a text file. From what I can tell Xenophane made some changes and now appears to import the .csv, spreadsheet, file instead of the text file. I've modified the following:
$paths = @() $paths = import-csv C:\temp\MySpreadsheetHere.csv $newServer = "\\MyServerName\c$\Foldername"
Foreach ($path in $paths) { $NewPath = "$newServer$($path.File.substring(1))" $($path.File) $NewPath Write-Output "f" | xcopy $($path.File) $NewPath /I /O /Y #Catch { Write-Output "Error: Xcopy: $_">>c:\temp\Failed.log}
If ($?) {Write-Output "Success: Xcopy $($path.File) -> $NewPath ">>c:\temp\Succeeded.log } else {Write-Output "Error: Xcopy: $($path.File)">>c:\temp\Failed.log} }
Save my script as filemove.ps Launch Powershell and navigate to my script:
Now I attempt to run the script: ps> movefile.ps
I get nothing. Am I doing something wrong?
tia
|
| Nobody |
Posted - 06/07/2011 : 4:26:33 PM Thanks for clearing that up. That was the solution to a problem I had recently. I had to work around it like this...
$note = Get-Process notepad|foreach-object {$_.handles}
write-host "Notepad uses $note handles"
heh...I've met my learning quota for the week, and it's only Tuesday! Thanks. |
| Xenophane |
Posted - 06/07/2011 : 3:38:11 PM Sorry about that I wanted to post it, but my 7 month old woke up, so after lulling him back to sleep I completely forgot :)
Below is my .csv file quote:
File \\localhost\c$\temp\123.txt \\localhost\c$\temp\1234.txt \\apt410-ctn\c$\temp\12345.txt
$($path.File) vs $path.File It should not be necessary in this example I more or less do it as a habit.. When you wrap you variable in $() it gets treated like an expression and evaluated. Example: (Red is what is output to the console)
PS C:\> notepad
PS C:\> Get-Process notepad
Handles NPM(K) PM(K) WS(K) VM(M) CPU(s) Id ProcessName
------- ------ ----- ----- ----- ------ -- -----------
70 7 1944 7128 79 0,05 8740 notepad
PS C:\> $note = Get-Process notepad
PS C:\> Write-host "Calc Uses $note.handles handles"
Calc Uses System.Diagnostics.Process (notepad).handles handles
PS C:\> write-host "Calc uses $($note.handles) handles"
Calc uses 70 handles
PS C:\>
|
| Nobody |
Posted - 06/07/2011 : 1:07:20 PM Xenophane, I was really struggling with your version for a bit... I didn't have the column header in my csv file 
I'm still a powershell noob, Can you explain:
$($path.File) vs $path.File |
| Xenophane |
Posted - 06/06/2011 : 6:06:00 PM Just made a little update to Nobody's script:
$paths = @()
$paths = import-csv C:\temp\UNCPaths.csv
$newServer = "\\apt410-ctn\c$\Scripts"
Foreach ($path in $paths) {
$NewPath = "$newServer$($path.File.substring(1))"
$($path.File)
$NewPath
Write-Output "f" | xcopy $($path.File) $NewPath /I /O /Y
#Catch { Write-Output "Error: Xcopy: $_">>c:\temp\Failed.log}
If ($?) {Write-Output "Success: Xcopy $($path.File) -> $NewPath ">>c:\temp\Succeeded.log }
else {Write-Output "Error: Xcopy: $($path.File)">>c:\temp\Failed.log}
}
It will write to files, one with errors and the other with successfull copies.
In this script I use the old path, to create the folder structure on the new server, to make sure, I do not get any problems with duplicate filenames... |
| Nobody |
Posted - 06/06/2011 : 11:30:06 AM This is what I came up with. I didn't do any error handling or logging, but the basics seem to work. Requires that you save your spreadsheet as a .txt file.
Powershell
$NewServer = '\\NewServer\Share' # Your New Server and Share
$Files = 'files.txt' # Save Excel file to text file
##################################################################
$arryUNC = @(get-content $files) # Loading each UNC path to an array
Foreach ($i in $arryUNC){ # Loop through the array
[string]$leaf = [regex]::split($i,"^\\\\\w+?(?:\\[\w\s$]+)") # Split file path from server\share
$leaf = $leaf -replace " ","" # trim that pesky leading space
$destination = $NewServer + $leaf # Set destination path variable
echo "f"|xcopy $i $destination /I /O /Y # Using xcopy to copy file. Preserves ACL
}
|
| cj_berlin |
Posted - 06/03/2011 : 5:06:43 PM OK, so here's a rough but working draft of an Excel macro. You'd have to reference the Windows Scripting Host library in your project to make this work.
Sub MoveThemAll()
Dim strSrcBase As String
Dim strDestBase As String
Dim fsoX As New FileSystemObject
Dim i, intLastLine As Integer
Dim strDestPath, strFile, strXPath As String
strSrcBase = "\\OLDSERVER\COMMONSHARE_IF_ANY\" 'with a trailing backslash
strDestBase = "\\NEWSERVER\COMMONSHARE_IF_ANY\" 'with a trailing backslash
intLastLine = 3 'number of significant line. This macro assumes that the file paths are in column A starting with row 1 and will write status output into columns B (for copy and/or create dest folder) and C (for delete)
For i = 1 To intLastLine
strXPath = Range("A" & Trim(Str(i))).Formula
If fsoX.FileExists(strXPath) Then
If Left(UCase(strXPath), Len(strSrcBase)) = UCase(strSrcBase) Then
strFile = fsoX.GetFileName(strXPath)
strDestPath = fsoX.GetParentFolderName(strXPath) & "\"
strDestPath = Replace(UCase(strDestPath), UCase(strSrcBase), UCase(strDestBase))
If Not fsoX.FolderExists(strDestPath) Then
MakeDestFolder (strDestPath)
End If
If fsoX.FolderExists(strDestPath) Then
fsoX.CopyFile strXPath, strDestPath & strFile
If fsoX.FileExists(strDestPath & strFile) Then
Range("B" & Trim(Str(i))).Formula = "copied successfuly"
fsoX.DeleteFile strXPath
If fsoX.FileExists(strXPath) Then
Range("C" & Trim(Str(i))).Formula = "delete failed"
Else
Range("C" & Trim(Str(i))).Formula = "deleted successfuly"
End If
Else
Range("B" & Trim(Str(i))).Formula = "copy failed"
End If
Else
Range("B" & Trim(Str(i))).Formula = "Don't seem to be able to create " & strDestPath
End If
End If
End If
Next i
End Sub
Sub MakeDestFolder(ByVal xPath As String)
Dim fsoX As New FileSystemObject
Dim xPF As String
xPF = fsoX.GetParentFolderName(xPath)
If fsoX.FolderExists(xPF) Then
fsoX.CreateFolder xPath
Else
MakeDestFolder (xPF)
If fsoX.FolderExists(xPF) Then fsoX.CreateFolder xPath
End If
End Sub
You'll have to rerun it for each source server.
EDIT: If ACLs are of any value for you, I'd have the VBA invoke a robocopy command. |
| Phil Marcum |
Posted - 06/03/2011 : 5:03:10 PM Basically This is the format of the paths throughout the spreadsheet: \\ServernameA\d$\folder\filename.txt \\ServernameZ\e$\folder\folder\folder\filename1.xls \\ServernameZ\e$\folder\folder\folder\filename2.xls \\ServernameD\g$\folder\folder\folder\folder\filename2.xls
1. Yes I'm dealing with an Excel spreadsheet which has all of the paths listed in one row / column.
2. The paths are to different servers and different shares where some go as far as 4 or 5 folders deep. I'd need to confirm if any extend beyond 255 characters.
3. There are multiple files in separate locations on the same server. So one server may have 5 files in different folders and another may have 2 files, etc. So I may have to sort the spreadsheet accordingly and go from there. The less intervention the better. Hopefully I didn't miss anything.
tia |
| cj_berlin |
Posted - 06/03/2011 : 4:10:56 PM quote: Originally posted by Nobody
How deep might the folder structure go?
This is an important one. You would be dealing with functions that expect the path as string so if those strings can potentially grow longer than 255 character, you'd need to use unicode functions which is a (minor) PITA... |
| Nobody |
Posted - 06/03/2011 : 4:01:55 PM Really need to know a couple specifics before we can help script anything. As asked above, is the spreadsheet just a one column excel spreadsheet with a unc path in each row? Are the files on the D$ share of each server? Are the folder/file names following some nameing convention? Are there going to be duplicate named folders/files on each server? How deep might the folder structure go? etc etc etc. So basically a couple examples would clear up most of the questions I have. |
| cj_berlin |
Posted - 06/03/2011 : 3:09:17 PM Phil,
by 'spreadsheet', do you mean an Excel spreadsheet? |
| Phil Marcum |
Posted - 06/03/2011 : 2:20:53 PM Standard UNC path: \\Servername\d$\folder\filename.txt
I'd need to recreate the UNC path on the destination server instead of dumping everything into a folder.
Tia |
| Nobody |
Posted - 06/03/2011 : 1:22:23 PM Can you post some examples of the spreadsheet format, and the source/destination paths? |