Throttle File Copy and other Advanced Copy Situations in Windows

Robocopy

At one time, maybe a year ago, I was trying to find a way to copy big files around a small network without choking the network and / or the sending or receiving computers.

Well, I've finally found it. The utterly useful Robocopy utility from the Windows Resource kit has a

/IPG:n : Inter-Packet Gap (ms)

flag that will delay each subsequent packet.

IPG - will send a 64 KB packet followed by a delay of n Milliseconds. If my math is correct, a 1ms delay every 64KB would generate a maximum of 6 MBps - but it's actually much less on a 100Mbps LAN because it would take a significant portion of the time to transmit the packet.

See: http://www.ss64.com/nt/robocopy.html

Example:
- note the tricky 3rd parameter specifying the filter, this is different from the regular Windows / DOS copy or xcopy syntax.

robocopy "d:\Company Shared Folders\IT" \receiving2\c$\backups *.bkf /ipg:10 /zb

- for a single file from current folder (".") to w: drive using ipg to give other users a chance to access files: robocopy . w: really-big-file.img w: /ipg:4

 

Syntax:

ROBOCOPY source destination [file [file]...] [options]

 

 




SuperCopier


2007-06-23:  Another tool, supercopier, this one with a GUI. Haven't personally tested it yet: http://sourceforge.net/projects/supercopier/

Update 2008-02-20:
I had a need to try this today.  Only the GPL source code is on Sourceforge. Visit the homepage for binary install for Windows:
http://supercopier.sfxteam.org/modules/news/

Update 2014: This seems to be infected by malware these days. :(


This program works great! There's a simple drop down to limit the transfer to 1, 2, 4 or custom MB/sec.

 

 

Copying all files that have been changed recently

 

Robocopy changed files - http://ss64.com/nt/robocopy.html

ROBOCOPY source_folder destination_folder [file(s)_to_copy] [options]

/E = empty -- don't use... seems to occur before MAXAGE, so, a LOT of folders get created

/S = subfolders

/L = list only

/Z = network restartable

ROBOCOPY "C:\TEMP" "C:\TEMPTEST" *.* /L /Z /S /MAXAGE:7 

ROBOCOPY "D:\Company Shared Folders" "D:\ToMove" *.*  /Z /S /MAXAGE:10 

ROBOCOPY "D:\Users Shared Folders - Delete in 2013" "D:\ToMoveUsers" *.*  /Z /S /MAXAGE:10 

 

ignore errors while copying: robocopy c:\ d:\ /MIR /R:0 /W:0

 

The Reg List

A fellow tried to move 60 million files.  Mentions "richcopy" and ultimately uses Linux as a middle man to handle really deep folder trees.

http://www.theregister.co.uk/2010/09/24/sysadmin_file_tools/

 

 

xcopy

Unfortunately, it's time to avoid xcopy. Even though I used it for decades, xcopy on XP will bomb if the file path hits a 254 character limit, thereby baffling everyone. Try robocopy.

xcopy - continue if errors found (e.g. bum spots on disk): xcopy /C/H/R/S/Y c:\ d:\

 

relevant notes

Windows Vista added i/o priority. There are tools that will let you adjust programs to have low disk priority.

e.g. http://processhacker.sourceforge.net/ (not tested)

Tags: Win, w2k, winxp, xcopy, robo copy, slow, file, backup, mirror, speed, robocopy, supercopier, 
 

Related Scribbles:
  • System Administration
  • Throttle Processes Windows and Start Syntax
  • Utilities
  • Utilities for Microsoft Windows
  • Windows


  • ID: 779
    Author:
    leonard
    Date Updated:
    2015-04-01 13:34:43
    Date Created:
    2006-08-02 16:10:58

    Edit

    Comments?
     >> Leonard Chan's Homepage  >> Scribble Web  >> Throttle File Copy and other Advanced Copy Situations in Windows
    leonard.lotus-land.ca is hosted by Perceptus Solutions Inc.