Mark Minasi's Reader Forum
Mark Minasi's Reader Forum
Home | Profile | Register | Active Topics | Active Polls | Members | Search | FAQ | Minasi Forum RSS Feed
Username:
Password:
Save Password
Forgot your Password?

 All Forums
 HALP! Questions on Windows and Windows Server
 Scripting
 Move files from spreadsheet to another server
 New Topic  Reply to Topic
 Printer Friendly
Author Previous Topic Topic Next Topic  

Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline

Posted - 06/03/2011 :  12:48:58 PM  Show Profile  Reply with Quote
Here's an odd request that I'm trying to script:
Task: move all files located in spreadsheet to a remote server for possible deletion.

I have a spreadsheet containing thousands of files scattered across multiple servers that need to be moved to a folder located on a single server. The spreadsheet contains the full UNC of the file(s) on the source server. What I'd like is for the script to recreate the same UNC on the destination server and move the file into it. Maintaining the ACL's on the files would alse be nice but isn't necessary as the main goal at this point is to move the files from the designated locations and delete them shortly after. If necessary I can always rollback by manually copying the file back to its original location.
My searches are fubar as I'm only getting hits on the import-csv cmdlet. Any responses are direction appreciated.

Nobody
Here To Stay

USA
184 Posts
Status: offline

Posted - 06/03/2011 :  1:22:23 PM  Show Profile  Visit Nobody's Homepage  Reply with Quote
Can you post some examples of the spreadsheet format, and the source/destination paths?

aka - Matt
www.SnowTrek.org
Go to Top of Page

Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline

Posted - 06/03/2011 :  2:20:53 PM  Show Profile  Reply with Quote
Standard UNC path:
\\Servername\d$\folder\filename.txt

I'd need to recreate the UNC path on the destination server instead of dumping everything into a folder.

Tia

Edited by - Phil Marcum on 06/03/2011 2:24:52 PM
Go to Top of Page

cj_berlin
Honorable But Hopeless Addict

Germany
3964 Posts
Status: offline

Posted - 06/03/2011 :  3:09:17 PM  Show Profile  Visit cj_berlin's Homepage  Look at the Skype address for cj_berlin  Reply with Quote
Phil,

by 'spreadsheet', do you mean an Excel spreadsheet?

Evgenij Smirnov
Go to Top of Page

Nobody
Here To Stay

USA
184 Posts
Status: offline

Posted - 06/03/2011 :  4:01:55 PM  Show Profile  Visit Nobody's Homepage  Reply with Quote
Really need to know a couple specifics before we can help script anything. As asked above, is the spreadsheet just a one column excel spreadsheet with a unc path in each row? Are the files on the D$ share of each server? Are the folder/file names following some nameing convention? Are there going to be duplicate named folders/files on each server? How deep might the folder structure go? etc etc etc. So basically a couple examples would clear up most of the questions I have.

aka - Matt
www.SnowTrek.org
Go to Top of Page

cj_berlin
Honorable But Hopeless Addict

Germany
3964 Posts
Status: offline

Posted - 06/03/2011 :  4:10:56 PM  Show Profile  Visit cj_berlin's Homepage  Look at the Skype address for cj_berlin  Reply with Quote
quote:
Originally posted by Nobody

How deep might the folder structure go?



This is an important one. You would be dealing with functions that expect the path as string so if those strings can potentially grow longer than 255 character, you'd need to use unicode functions which is a (minor) PITA...

Evgenij Smirnov
Go to Top of Page

Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline

Posted - 06/03/2011 :  5:03:10 PM  Show Profile  Reply with Quote
Basically This is the format of the paths throughout the spreadsheet:
\\ServernameA\d$\folder\filename.txt
\\ServernameZ\e$\folder\folder\folder\filename1.xls
\\ServernameZ\e$\folder\folder\folder\filename2.xls
\\ServernameD\g$\folder\folder\folder\folder\filename2.xls

1. Yes I'm dealing with an Excel spreadsheet which has all of the paths listed in one row / column.

2. The paths are to different servers and different shares where some go as far as 4 or 5 folders deep. I'd need to confirm if any extend beyond 255 characters.

3. There are multiple files in separate locations on the same server. So one server may have 5 files in different folders and another may have 2 files, etc. So I may have to sort the spreadsheet accordingly and go from there. The less intervention the better. Hopefully I didn't miss anything.

tia
Go to Top of Page

cj_berlin
Honorable But Hopeless Addict

Germany
3964 Posts
Status: offline

Posted - 06/03/2011 :  5:06:43 PM  Show Profile  Visit cj_berlin's Homepage  Look at the Skype address for cj_berlin  Reply with Quote
OK, so here's a rough but working draft of an Excel macro. You'd have to reference the Windows Scripting Host library in your project to make this work.


Sub MoveThemAll()
    Dim strSrcBase As String
    Dim strDestBase As String
    Dim fsoX As New FileSystemObject
    Dim i, intLastLine As Integer
    Dim strDestPath, strFile, strXPath As String
    
    
    strSrcBase = "\\OLDSERVER\COMMONSHARE_IF_ANY\" 'with a trailing backslash
    strDestBase = "\\NEWSERVER\COMMONSHARE_IF_ANY\" 'with a trailing backslash
    intLastLine = 3 'number of significant line. This macro assumes that the file paths are in column A starting with row 1 and will write status output into columns B (for copy and/or create dest folder) and C (for delete)


    


    For i = 1 To intLastLine
        strXPath = Range("A" & Trim(Str(i))).Formula
        If fsoX.FileExists(strXPath) Then
            If Left(UCase(strXPath), Len(strSrcBase)) = UCase(strSrcBase) Then
                strFile = fsoX.GetFileName(strXPath)
                strDestPath = fsoX.GetParentFolderName(strXPath) & "\"
                strDestPath = Replace(UCase(strDestPath), UCase(strSrcBase), UCase(strDestBase))
                If Not fsoX.FolderExists(strDestPath) Then
                    MakeDestFolder (strDestPath)
                End If
                If fsoX.FolderExists(strDestPath) Then
                    fsoX.CopyFile strXPath, strDestPath & strFile
                    If fsoX.FileExists(strDestPath & strFile) Then
                        Range("B" & Trim(Str(i))).Formula = "copied successfuly"
                        fsoX.DeleteFile strXPath
                        If fsoX.FileExists(strXPath) Then
                            Range("C" & Trim(Str(i))).Formula = "delete failed"
                        Else
                            Range("C" & Trim(Str(i))).Formula = "deleted successfuly"
                        End If
                    Else
                        Range("B" & Trim(Str(i))).Formula = "copy failed"
                    End If
                Else
                     Range("B" & Trim(Str(i))).Formula = "Don't seem to be able to create " & strDestPath
                End If
            End If
        End If
    Next i
End Sub


Sub MakeDestFolder(ByVal xPath As String)
    Dim fsoX As New FileSystemObject
    Dim xPF As String
    xPF = fsoX.GetParentFolderName(xPath)
    If fsoX.FolderExists(xPF) Then
        fsoX.CreateFolder xPath
    Else
        MakeDestFolder (xPF)
        If fsoX.FolderExists(xPF) Then fsoX.CreateFolder xPath
    End If
End Sub



You'll have to rerun it for each source server.

EDIT: If ACLs are of any value for you, I'd have the VBA invoke a robocopy command.

Evgenij Smirnov

Edited by - cj_berlin on 06/03/2011 5:10:39 PM
Go to Top of Page

Nobody
Here To Stay

USA
184 Posts
Status: offline

Posted - 06/06/2011 :  11:30:06 AM  Show Profile  Visit Nobody's Homepage  Reply with Quote
This is what I came up with. I didn't do any error handling or logging, but the basics seem to work. Requires that you save your spreadsheet as a .txt file.

Powershell

$NewServer = '\\NewServer\Share'                                 # Your New Server and Share
$Files = 'files.txt'                                             # Save Excel file to text file
##################################################################
$arryUNC = @(get-content $files)                                 # Loading each UNC path to an array
Foreach ($i in $arryUNC){                                        # Loop through the array
    [string]$leaf = [regex]::split($i,"^\\\\\w+?(?:\\[\w\s$]+)") # Split file path from server\share
    $leaf = $leaf -replace " ",""                                # trim that pesky leading space
    $destination = $NewServer + $leaf                            # Set destination path variable
    echo "f"|xcopy $i $destination /I /O /Y                      # Using xcopy to copy file. Preserves ACL
    
}

aka - Matt
www.SnowTrek.org

Edited by - Nobody on 06/06/2011 11:42:43 AM
Go to Top of Page

Xenophane
Honorable But Hopeless Addict

Denmark
3070 Posts
Status: offline

Posted - 06/06/2011 :  6:06:00 PM  Show Profile  Visit Xenophane's Homepage  Send Xenophane an ICQ Message  Reply with Quote
Just made a little update to Nobody's script:


$paths = @()
$paths = import-csv C:\temp\UNCPaths.csv
$newServer = "\\apt410-ctn\c$\Scripts"

Foreach ($path in $paths) {
$NewPath = "$newServer$($path.File.substring(1))"
$($path.File)
$NewPath
 Write-Output "f" | xcopy $($path.File) $NewPath /I /O /Y 
#Catch { Write-Output "Error: Xcopy: $_">>c:\temp\Failed.log}

If ($?) {Write-Output "Success: Xcopy $($path.File) -> $NewPath ">>c:\temp\Succeeded.log }
else {Write-Output "Error: Xcopy: $($path.File)">>c:\temp\Failed.log}
}



It will write to files, one with errors and the other with successfull copies.

In this script I use the old path, to create the folder structure on the new server, to make sure, I do not get any problems with duplicate filenames...

Microsoft Powershell MVP

SIG> George Bernard Shaw : The power of accurate observation is commonly called cynicism by those who have not got it. </SIG>

You can read my blog at www.xipher.dk
Go to Top of Page

Nobody
Here To Stay

USA
184 Posts
Status: offline

Posted - 06/07/2011 :  1:07:20 PM  Show Profile  Visit Nobody's Homepage  Reply with Quote
Xenophane, I was really struggling with your version for a bit... I didn't have the column header in my csv file

I'm still a powershell noob, Can you explain:

$($path.File) vs $path.File

aka - Matt
www.SnowTrek.org
Go to Top of Page

Xenophane
Honorable But Hopeless Addict

Denmark
3070 Posts
Status: offline

Posted - 06/07/2011 :  3:38:11 PM  Show Profile  Visit Xenophane's Homepage  Send Xenophane an ICQ Message  Reply with Quote
Sorry about that I wanted to post it, but my 7 month old woke up, so after lulling him back to sleep I completely forgot :)

Below is my .csv file
quote:

File
\\localhost\c$\temp\123.txt
\\localhost\c$\temp\1234.txt
\\apt410-ctn\c$\temp\12345.txt



$($path.File) vs $path.File
It should not be necessary in this example I more or less do it as a habit.. When you wrap you variable in $() it gets treated like an expression and evaluated.
Example: (Red is what is output to the console)



PS C:\> notepad
PS C:\> Get-Process notepad

Handles  NPM(K)    PM(K)      WS(K) VM(M)   CPU(s)     Id ProcessName
-------  ------    -----      ----- -----   ------     -- -----------
     70       7     1944       7128    79     0,05   8740 notepad


PS C:\> $note = Get-Process notepad
PS C:\> Write-host "Calc Uses $note.handles handles"
Calc Uses System.Diagnostics.Process (notepad).handles handles
PS C:\> write-host "Calc uses $($note.handles) handles"
Calc uses 70 handles
PS C:\>

Microsoft Powershell MVP

SIG> George Bernard Shaw : The power of accurate observation is commonly called cynicism by those who have not got it. </SIG>

You can read my blog at www.xipher.dk
Go to Top of Page

Nobody
Here To Stay

USA
184 Posts
Status: offline

Posted - 06/07/2011 :  4:26:33 PM  Show Profile  Visit Nobody's Homepage  Reply with Quote
Thanks for clearing that up. That was the solution to a problem I had recently. I had to work around it like this...


$note = Get-Process notepad|foreach-object {$_.handles}
write-host "Notepad uses $note handles"


heh...I've met my learning quota for the week, and it's only Tuesday! Thanks.

aka - Matt
www.SnowTrek.org
Go to Top of Page

Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline

Posted - 06/22/2011 :  3:50:31 PM  Show Profile  Reply with Quote
Almost a month later and I'm just getting the green light to move forward with this project. Being that I need to move the files from one location to another server creating the same path I'm going to give "Nobody's" script a try. Here's my dilema:
1. I won't say that the script doesn't work, I just can't get it to work for me. Nobody's original powershell script called for the spreadsheet to be saved as a text file. From what I can tell Xenophane made some changes and now appears to import the .csv, spreadsheet, file instead of the text file.
I've modified the following:

$paths = @()
$paths = import-csv C:\temp\MySpreadsheetHere.csv
$newServer = "\\MyServerName\c$\Foldername"

Foreach ($path in $paths) {
$NewPath = "$newServer$($path.File.substring(1))"
$($path.File)
$NewPath
Write-Output "f" | xcopy $($path.File) $NewPath /I /O /Y
#Catch { Write-Output "Error: Xcopy: $_">>c:\temp\Failed.log}

If ($?) {Write-Output "Success: Xcopy $($path.File) -> $NewPath ">>c:\temp\Succeeded.log }
else {Write-Output "Error: Xcopy: $($path.File)">>c:\temp\Failed.log}
}

Save my script as filemove.ps
Launch Powershell and navigate to my script:

Now I attempt to run the script:
ps> movefile.ps

I get nothing. Am I doing something wrong?

tia



Go to Top of Page

Xenophane
Honorable But Hopeless Addict

Denmark
3070 Posts
Status: offline

Posted - 06/23/2011 :  02:16:00 AM  Show Profile  Visit Xenophane's Homepage  Send Xenophane an ICQ Message  Reply with Quote
Phil, The line containing the file paths in the csv file, has to contain the word "file" (Columnheader without the quotes) in the first line.


$paths = @()
$paths = import-csv C:\temp\MySpreadsheetHere.csv
$newServer = "\\MyServerName\c$\Foldername"

Foreach ($path in $paths) {
$NewPath = "$newServer$($path.File.substring(1))"
$($path.File)
$NewPath
}


You can just try to copy and paste the above into the console to see if you get anything (It just prints the path to the screen)

Microsoft Powershell MVP

SIG> George Bernard Shaw : The power of accurate observation is commonly called cynicism by those who have not got it. </SIG>

You can read my blog at www.xipher.dk
Go to Top of Page

Nobody
Here To Stay

USA
184 Posts
Status: offline

Posted - 06/23/2011 :  11:57:34 AM  Show Profile  Visit Nobody's Homepage  Reply with Quote
As mentioned, my method requires a text file with one path per line, no column headers or any extra content. You can just "save as" .txt from excel, and make sure any column headers are removed.

Xenophane's method requires a csv file with a column header named 'file'.


aka - Matt
www.SnowTrek.org
Go to Top of Page

Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline

Posted - 06/24/2011 :  11:08:09 AM  Show Profile  Reply with Quote
Got it! Tested and confirmed that the script will copy the complete URL path and file from the source server(s) to the destination server. What I need to do now is modify it to "move" the files from the source to the destination. Tried unsuccessfully at switching the Xcopy commands with the Move command.

Any pointers?
Go to Top of Page

Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline

Posted - 06/24/2011 :  11:14:05 AM  Show Profile  Reply with Quote
Got it and able to confirm that the complete folder and file structure is copied to the destination server. What I need to do now is modify the script to move the files to the destination server. Swapping to xcopy command with move isn't working. Am I missing something?

Tia
Go to Top of Page

Nobody
Here To Stay

USA
184 Posts
Status: offline

Posted - 06/24/2011 :  11:46:16 AM  Show Profile  Visit Nobody's Homepage  Reply with Quote
A move is nothing more than a copy and a delete. So you can just add in a Remove-Item line immediately after the xcopy, or after all files have copied, run through the foreach loop again and delete. Just be careful, if don't do any error handling, and xcopy goes sideways, it could delete all your files before they are copied.

Using Xenophane's script you can delete right after the success log entry, the log then delete if successful.


aka - Matt
www.SnowTrek.org

Edited by - Nobody on 06/24/2011 11:47:15 AM
Go to Top of Page

Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline

Posted - 06/24/2011 :  3:18:53 PM  Show Profile  Reply with Quote
Stuck on the remove-item cmdlet as I need to ensure that the entire UNC path and file is copied to the destination server, which is covered. The other side is to remove the file and not the entire path from the source server as there are other files in that folder.
Go to Top of Page

Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline

Posted - 06/24/2011 :  3:22:35 PM  Show Profile  Reply with Quote
Also looking at the "move-item" cmdlet as it seems to ensure that the entire UNC path and file is moved to the destination server which is what I want.

Edited by - Phil Marcum on 06/24/2011 3:51:02 PM
Go to Top of Page

Nobody
Here To Stay

USA
184 Posts
Status: offline

Posted - 06/24/2011 :  4:21:58 PM  Show Profile  Visit Nobody's Homepage  Reply with Quote
I used xcopy because that's the only tool I could think of that would replicate your folder structure and keep the ACL as well as the various other parameters that xcopy offers. Oddly, powershell doesn't seem to have a tool on par with xcopy.

"Remove-Item path/to/somefile.txt" will only delete the file specified, not the entire folder structure. Play around with these commands and build some confidence in what they do.

use the -full parameter to get detailed command info and examples

Get-Help Remove-Item -full


aka - Matt
www.SnowTrek.org
Go to Top of Page

Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline

Posted - 07/07/2011 :  01:18:53 AM  Show Profile  Reply with Quote
No luck in getting this to copy the files and delete them using a single script and will try using a second script once the files have successfully copied. In looking at the script applied I wonder if it's possible to modify it to delete the same file that was copied earlier.

$paths = @()
$paths = import-csv C:\temp\MySpreadsheetHere.csv
$newServer = "\\MyServerName\c$\Foldername"

Foreach ($path in $paths) {
$NewPath = "$newServer$($path.File.substring(1))"
$($path.File)
$NewPath
Write-Output "f" | remove-item $($path.File) $NewPath /I /O /Y
#Catch { Write-Output "Error: remove-item: $_">>c:\temp\Moved.log}

If ($?) {Write-Output "Success: remove-item $($path.File) -> $NewPath ">>c:\temp\Succeeded.log }
else {Write-Output "Error: remove-item: $($path.File)">>c:\temp\Failed.log}
}

Having it to write to a a separate would be nice for verification purposes. Am I missing something here?

Any responses appreciated.
Go to Top of Page

Xenophane
Honorable But Hopeless Addict

Denmark
3070 Posts
Status: offline

Posted - 07/07/2011 :  06:40:59 AM  Show Profile  Visit Xenophane's Homepage  Send Xenophane an ICQ Message  Reply with Quote
BE carefull running the above script.. You are using remove-item instead of move-item

Microsoft Powershell MVP

SIG> George Bernard Shaw : The power of accurate observation is commonly called cynicism by those who have not got it. </SIG>

You can read my blog at www.xipher.dk
Go to Top of Page

Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline

Posted - 07/18/2011 :  10:12:32 AM  Show Profile  Reply with Quote
Now that I've successfully copied the files I need to go in and delete each of them. I could use some help figuring out how this would work with the current script. Meaning there shouldn't be a "newserver" statement:

$newServer = "\\MyServerName\c$\Foldername"

When removed it throws all sorts of errors. It looks as though I need to remove every "NewServer" reference?

Am I missing something?

Edited by - Phil Marcum on 07/18/2011 10:30:05 AM
Go to Top of Page

Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline

Posted - 07/18/2011 :  1:06:11 PM  Show Profile  Reply with Quote
Error(s) thrown:

Remove-Item : A positional parameter cannot be found that accepts argument '\\myfilesrv3\e$\filemove\myserver\d$\myfiles\samplefile1.txt'.
At D:\Projects\data\Delete\dlpdelete.ps1:9 char:31
+ Write-Output "f" | remove-item <<<< $($path.File) $NewPath /I /O /Y
+ CategoryInfo : InvalidArgument: (:) [Remove-Item], ParameterBindingException
+ FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand

Error is thrown on each file.
Go to Top of Page

Xenophane
Honorable But Hopeless Addict

Denmark
3070 Posts
Status: offline

Posted - 07/18/2011 :  3:38:07 PM  Show Profile  Visit Xenophane's Homepage  Send Xenophane an ICQ Message  Reply with Quote
It seems to me that you are trying to pass multiple parameters to remove-item
Something like this should be able to do it, remove -whatif when you are sure you really really want to do it :)


$paths = @()
$paths = import-csv C:\temp\MySpreadsheetHere.csv
Foreach ($path in $paths) {
$($path.File)
remove-item $($path.File) -force -whatif
}

Microsoft Powershell MVP

SIG> George Bernard Shaw : The power of accurate observation is commonly called cynicism by those who have not got it. </SIG>

You can read my blog at www.xipher.dk
Go to Top of Page

Phil Marcum
Seasoned But Casual Onlooker

44 Posts
Status: offline

Posted - 07/19/2011 :  10:55:31 AM  Show Profile  Reply with Quote
Freakin' AWESOME!!!!
Go to Top of Page
  Previous Topic Topic Next Topic  
 New Topic  Reply to Topic
 Printer Friendly
Jump To:
Mark Minasi's Reader Forum © 2002-2011 Mark Minasi Go To Top Of Page
This page was generated in 0.3 seconds. Snitz Forums 2000