Experience tells us that large files, such as a 5TB Hyper V disk, will transfer at approx. twice the speed of 5TB of regular files and small files (less than 200K in size). The reason is that a lot of backup and restore overhead occurs during the first exchange. Once this handshaking is complete, larger files will transfer as fast as the connection allows.
Online backup software expects to run on low bandwidth connections, so all providers will have grown up with Kbps speeds where the software is written specifically to handle these situations.
Online backup software will check every file encountered during the pre-flight checks, and it will look something like this:
1. Does the file need backing up (i.e. has it been modified since the last backup ‘delta’)?
2. What part of the file has been modified? There is no obvious way to ascertain this, and very often, the backup software will dump out the file locally and stream out the delta. This will increase the backup time but not the transfer time.
3. Does the delta file need chopping ‘chunking’ up into smaller manageable chunks so the transfer can be retried if a block fails in transmission? Our software will chunk the file into blocks of 32Mb or smaller.
4. Does the delta need encrypting? The answer should always be yes.
When this process is run on the 5TB Hyper V disk we mentioned earlier, steps 1 and 2 are done once.
If we repeat this on 5TB of small files, the overhead in steps 1 and 2 is greater because it is repeated approximately. 25,000 times.
These factors all have an impact on the backup and restore speeds.
There is no way to estimate backup times besides rigid benchmarking on customer environments.
Thanks for posting. We have used S3 and EC2 for a few years and it is good to see how BB and Wasabi are offering a better product.
has anyone use soft delete on idrive?
I tried idrive storage and it worked as fast as I need. Glad to find
Is the AWS pricing right? We use S3, but haven’t experienced these costs
Wasabi baby!
We have used IDrive’s object storage for a few months and it benchmarks just as well as Backblaze the same as you tested. We are in Ireland.
Azure ist nicht S3
A good report. Azure isn’t S3 compatible though
Azure isn’t S3. It is easy to script though and follows a very similar convention.
A good report. Azure isn’t S3 compatible though
We are using idrive object storage and it is as quick as Bacblaze b2 we are replacing.
Someone is making a lot of money from storage
FYI Backblaze is the fastest for us. We are in NY
IDrive must be losing money to get that price.
idrive are uber slow in my test!
we are using iDrive and B2 from Backblaze and they are as good as one another. Will you be running another speed test in 6 months time to see if the response times are the same?
Good move by iDrive
Maybe you can run an article on how their support teams compare for first answers
BTW, I find Wasabi console very hard to navigate. Azure is very easy and more expensive though.
backblaze is a good one and we use since moving from their hosted backup