Have you tried copying large files on Linux, sometimes it just takes ages? I was doing that when I thought there should be a faster and better way to copy files in Linux. So I started searching and came across these commands which can offer better copying speed.
As simple cp command is very useful but sometimes it can slow down the process. These commands should help you get your copying done in the fastest way.
For copying sometimes the tar command can be a better alternative. Sometimes providing a faster and safer alternative. Here is how to use tar.
How to copy files faster using tar command in Linux
To copy files open a terminal, it can generally be opened by Ctrl + Alt + T. Now in terminal change the current directory to the folder from which you want to copy files.
cd Downloads/test
Now just run the command below to copy files.
tar cf - . | (cd /output/directory/ && tar xvf -)
While executing the command just replace /output/directory with the directory in which you want to copy files. All the files and subfolders are copied from the current directory to the /output/directory.
Now if you want, you can also use pv to help you monitor the progress of copying files. For example:
tar cf - . | pv | (cd /output/directory && tar xvf -)
Cp vs tar why is the speed difference?
In cp vs tar, tar sometimes has much higher copy speed than cp. The reason behind that is cp does open-read-close-open-write-close in a loop. And while tar does reading and writing in a separate process. Tar also uses multiple threads to read and write and can even several files at once.
This makes tar clearly win in comparison of cp vs tar. As tar works in a more speedy and efficient way.
Here are other alternatives that you can use to copy files in fastest way on Linux.
Another alternative command
The other command that is fast and very versatile for copying files between two locations is rsync . It can be used to copy between local as well as remote locations.
To copy files using rsync you need to enter the command below.
rsync -a Downloads/songs/abc.zip Downloads/music/
To view the progress while copying large size files you can use the command below.
rsync --info=progress2 -auvz Downloads/songs/abc.zip Downloads/music/
If you are wondering, here is what -auvz stands for.
- a: archive files and directory while synchronizing.
- u: Don’t copy files from source to destination, if destination already has newer files.
- v: Verbose output.
- z: Compress data during the transfer.
In the above example, copying is being done locally, but you can use rsync for copying over remote locations also.
You can also use like n for the dry run (to perform a trial run without synchronization) and r for recursive (sync files and directories recursively). If you are transferring from a remote location you can also use -e ssh to secure communications. Here are some other commands that you can use if you want.
If your system doesn’t come preinstalled with rsync, then you can install using the commands below.
cp vs rsync which one is better?
Although rsync is not generally faster than cp, but as it only syncs files that are modified or new. It can offer better speed when synchronizing files. The rsync also has multiple advanced options that are not available in cp.
How to install rsync
rsyn comes pre-installed on most Linux distros. But if it’s not preinstalled you can install with the command below.
On Debian and Ubuntu-based systems use the command below.
sudo apt-get install rsync
On CentOS/RHEL based systems use the command below.
yum install rsync
For SUSE/Open based systems.
zypper in rsync
These commands will install rsync on your system. Now you can try copying files with a better speed.
SCP for copying
Secure copy or also known as SCP, can also be used for copying. Although it is not for fast copying, it can be used for the secure transmission of files between a local host and a remote host. Or between two remote hosts. So when you are looking for secure transmission of files you can use this method.
Here is how you can use SCP for file transferring from a local to a remote host.
scp file_name.zip username@to_host:/remote/directory/
For transferring of file from a remote host to local host you can use the command below.
scp username@from_host:file_name.zip /local/directory/
For transferring of a file from remote host to remote host use the command below.
scp username@from_host:/remote/directory/file_name.zip username@to_host:/remote/directory/
I hope these commands should help you. Here are some other articles that I think you might like How to add a user to a group in Linux and how can Linux NTFS file.
Thanks for reading.