Robocopy
At one time, maybe a year ago, I was trying to find a way to copy big files around a small network without choking the network and / or the sending or receiving computers.
Well, I've finally found it. The utterly useful Robocopy utility from the Windows Resource kit has a
/IPG:n : Inter-Packet Gap (ms)
flag that will delay each subsequent packet.
IPG - will send a 64 KB packet followed by a delay of n Milliseconds. If my math is correct, a 1ms delay every 64KB would generate a maximum of 6 MBps - but it's actually much less on a 100Mbps LAN because it would take a significant portion of the time to transmit the packet.
See: http://www.ss64.com/nt/robocopy.html
Example:
- note the tricky 3rd parameter specifying the filter, this is different from the regular Windows / DOS copy or xcopy syntax.
robocopy "d:\Company Shared Folders\IT" \receiving2\c$\backups *.bkf /ipg:10 /zb
UPDATE 2021: ipg doesn't seem to work the way I expect, something may have changed (maybe multi core related?). I used /IPG:100 on a 100mbit LAN to throttle a backup to about 75mbit.
- for a single file from current folder (".") to w: drive using ipg to give other users a chance to access files: robocopy . w: really-big-file.img w: /ipg:4
Syntax:
ROBOCOPY source destination [file [file]...] [options]
Robocopy does not do authentication. To copy to network shares on different domains... something like this can work:
net use \\server1\g$ /user:domain1\user1 *
net use \\server2\g$ /user:domain2\user2 *
robocopy \\server1\G$\testdir\%3 \\server2\g$\uploads
Teracopy
2017-10 - I've seen a couple references to this. Claims to be able to retry on errors. I'm hoping it will work on my WiFi that keeps cutting out when the neighbours start filling up the 2.4GHz bandwidth... http://www.codesector.com/teracopy
SuperCopier
2007-06-23: Another tool, supercopier, this one with a GUI. Haven't personally tested it yet: http://sourceforge.net/projects/supercopier/
Update 2008-02-20:
I had a need to try this today. Only the GPL source code is on Sourceforge. Visit the homepage for binary install for Windows:
http://supercopier.sfxteam.org/modules/news/
Update 2014: This seems to be infected by malware these days. :(
This program works great! There's a simple drop down to limit the transfer to 1, 2, 4 or custom MB/sec.
Copying all files that have been changed recently
Robocopy changed files - http://ss64.com/nt/robocopy.html
ROBOCOPY source_folder destination_folder [file(s)_to_copy] [options]
/E = empty -- don't use... seems to occur before MAXAGE, so, a LOT of folders get created
/S = subfolders
/L = list only
/Z = network restartable
ROBOCOPY "C:\TEMP" "C:\TEMPTEST" *.* /L /Z /S /MAXAGE:7
ROBOCOPY "D:\Company Shared Folders" "D:\ToMove" *.* /Z /S /MAXAGE:10
ROBOCOPY "D:\Users Shared Folders - Delete in 2013" "D:\ToMoveUsers" *.* /Z /S /MAXAGE:10
ignore errors while copying: robocopy c:\ d:\ /MIR /R:0 /W:0
The Reg List
A fellow tried to move 60 million files. Mentions "richcopy" and ultimately uses Linux as a middle man to handle really deep folder trees.
http://www.theregister.co.uk/2010/09/24/sysadmin_file_tools/
xcopy
Unfortunately, it's time to avoid xcopy. Even though I used it for decades, xcopy on XP will bomb if the file path hits a 254 character limit, thereby baffling everyone. Try robocopy.
xcopy - continue if errors found (e.g. bum spots on disk): xcopy /C/H/R/S/Y c:\ d:\
relevant notes
Windows Vista added i/o priority. There are tools that will let you adjust programs to have low disk priority.
e.g. http://processhacker.sourceforge.net/ (not tested)
Tags: Win, w2k, winxp, xcopy, robo copy, slow, file, backup, mirror, speed, robocopy, supercopier,