Why do transfer speeds drop so drastically when moving/copying a very large number of very small files compared to just moving/copying a very small number of large files?
File transfer speeds are generally faster for larger files than smaller files because it takes longer to transmit more data. However, transferring many small files can be even slower than transferring a few large files that are the same total size. This is because the operating system (OS) spends more time performing operations on each small file, like finding, opening, and closing.
Imagine you need to move a single large box.
You only need to bend down once, pick it up, carry it to your destination, and put it down, completing the task in one motion.
Now, if you have multiple smaller boxes to move, even if the total weight is the same as the large box, you have to repeat the bending down, picking up, and putting down actions multiple times.
These repeated actions add extra time, which slows down the overall process and results in a longer total time to complete the task.
2 Likes
This is typical behaviour across all operating systems since they were invented. There is a fair bit of overhead in any operating system to manage the structure, permissions, attributes etc of files as well as opening and closing data streams and other functions.
When dealing with small files, the “management” functions are easily more resource intensive than reading/writing the actual data.