Throttle File Copy and other Advanced Copy Situations in Windows

Robocopy

At one time, maybe a year ago, I was trying to find a way to copy big files around a small network without choking the network and / or the sending or receiving computers.

Well, I've finally found it. The utterly useful Robocopy utility from the Windows Resource kit has a

/IPG:n : Inter-Packet Gap (ms)

flag that will delay each subsequent packet.

IPG - will send a 64 KB packet followed by a delay of n Milliseconds. If my math is correct, a 1ms delay every 64KB would generate a maximum of 6 MBps - but it's actually much less on a 100Mbps LAN because it would take a significant portion of the time to transmit the packet.

See: http://www.ss64.com/nt/robocopy.html

Example:
- note the tricky 3rd parameter specifying the filter, this is different from the regular Windows / DOS copy or xcopy syntax.

robocopy "d:\Company Shared Folders\IT" \receiving2\c$\backups *.bkf /ipg:10 /zb

- for a single file from current folder (".") to w: drive using ipg to give other users a chance to access files: robocopy . w: really-big-file.img w: /ipg:4

 

Syntax:

ROBOCOPY source destination [file [file]...] [options]

 

Robocopy does not do authentication. To copy to network shares on different domains... something like this can work:

net use \\server1\g$ /user:domain1\user1 * 

net use \\server2\g$ /user:domain2\user2 *

robocopy \\server1\G$\testdir\%3 \\server2\g$\uploads

 

Teracopy

2017-10 - I've seen a couple references to this. Claims to be able to retry on errors. I'm hoping it will work on my WiFi that keeps cutting out when the neighbours start filling up the 2.4GHz bandwidth... http://www.codesector.com/teracopy



SuperCopier


2007-06-23:  Another tool, supercopier, this one with a GUI. Haven't personally tested it yet: http://sourceforge.net/projects/supercopier/

Update 2008-02-20:
I had a need to try this today.  Only the GPL source code is on Sourceforge. Visit the homepage for binary install for Windows:
http://supercopier.sfxteam.org/modules/news/

Update 2014: This seems to be infected by malware these days. :(


This program works great! There's a simple drop down to limit the transfer to 1, 2, 4 or custom MB/sec.

 

 

Copying all files that have been changed recently

 

Robocopy changed files - http://ss64.com/nt/robocopy.html

ROBOCOPY source_folder destination_folder [file(s)_to_copy] [options]

/E = empty -- don't use... seems to occur before MAXAGE, so, a LOT of folders get created

/S = subfolders

/L = list only

/Z = network restartable

ROBOCOPY "C:\TEMP" "C:\TEMPTEST" *.* /L /Z /S /MAXAGE:7 

ROBOCOPY "D:\Company Shared Folders" "D:\ToMove" *.*  /Z /S /MAXAGE:10 

ROBOCOPY "D:\Users Shared Folders - Delete in 2013" "D:\ToMoveUsers" *.*  /Z /S /MAXAGE:10 

 

ignore errors while copying: robocopy c:\ d:\ /MIR /R:0 /W:0

 

The Reg List

A fellow tried to move 60 million files.  Mentions "richcopy" and ultimately uses Linux as a middle man to handle really deep folder trees.

http://www.theregister.co.uk/2010/09/24/sysadmin_file_tools/

 

 

xcopy

Unfortunately, it's time to avoid xcopy. Even though I used it for decades, xcopy on XP will bomb if the file path hits a 254 character limit, thereby baffling everyone. Try robocopy.

xcopy - continue if errors found (e.g. bum spots on disk): xcopy /C/H/R/S/Y c:\ d:\

 

relevant notes

Windows Vista added i/o priority. There are tools that will let you adjust programs to have low disk priority.

e.g. http://processhacker.sourceforge.net/ (not tested)

Tags: Win, w2k, winxp, xcopy, robo copy, slow, file, backup, mirror, speed, robocopy, supercopier, 
 

Related Scribbles:
  • System Administration
  • Throttle Processes Windows and Start Syntax
  • Utilities
  • Utilities for Microsoft Windows
  • Windows


  • ID: 779
    Author:
    leonard
    Date Updated:
    2017-10-30 22:47:02
    Date Created:
    2006-08-02 16:10:58

    Edit

    Comments?
     >> Leonard Chan's Homepage  >> Scribble Web  >> Throttle File Copy and other Advanced Copy Situations in Windows
    leonard.lotus-land.ca is hosted by Perceptus Solutions Inc.