After i’ve posted my experience with Cloud.net two weeks ago, Terry Myers, a rep from Cloud.net contacted me to see if i’d be interested in trying out some more of their locations. I instantly said yes, of course, since who can reject such sweet, sweet chance of spinning up some free VMs from multiple providers in different locations for free.
So this is a disclaimer. I’ve got $50 credits from Cloud.net to test out their products, and that’s it. Those credits can’t be converted into money, and all of the things that i’ll be writing in this review is purely my own experience and opinion, without anybody from Cloud.net having a pre-published approval.
Here We Go, Then
I got the credits. I decided to spin up some of their more exotic locations. As i’m located in Asia, i decided to deploy a server in Hongkong, since that’s the only asia location they have up on their marketplace.
And this turned out to be a really bad experience. From the start, i knew there’s something wrong with the server. It’s very slow in responding to SSH command, and after struggling to do a basic OS update before i do any testing, i decided to just get a basic bench.sh benchmark and what do you know:
System Info ----------- Processor : QEMU Virtual CPU version (cpu64-rhel6) CPU Cores : 1 Frequency : 1999.999 MHz Memory : 486 MB Swap : MB Uptime : 23 min, Disk Speed ---------- I/O (1st run) : 3 MB/s I/O (2nd run) : 6 MB/s I/O (3rd run) : 4 MB/s Average I/O : 4.33333 MB/s
That I/O might explain a few things. But then it got worse, about a few minutes later after i’ve finished updating the VM and reboot, it won’t come back up again.
So i raised a ticket, and at first Terry replies to say that they’re looking into it, might just be a firewall issue. But after a few minutes and some interactions later, a staff replied to the tickets saying that it’s a network-wide issue, and as a result of this, the provider (they call them “zone”) will be hidden from Cloud.net. I haven’t seen them since.
Okay. To be completely honest, although the support is great, that whole experience wasn’t exactly pleasant, and to me illustrates typical friction points when dealing with a “marketplace” (some people might call middleman), as opposed to a provider who manage their own equipment directly. Lack of control by the platform owners, and fragmentation. On fragmentation especially, i have a slight issue with how Cloud.net tries to reduce this friction point, mainly by using a measurement figures called Cloud Index.
Cloud Index
Like Google’s Android platform, Cloud.net, or their parent company Onapp, probably just sets up a specific rules & requirements for provider who wants to join. From there, the providers will get listed on their marketplace, and Cloud.net will monitor their performance. The metrics that they use to monitor the provider, also known as “zones”, is called ‘Cloud Index’.
Now, cloud index, as a term of measurement, is an invention of Cloud.net. And unfortunately, their marketplace doesn’t really explain in full details of what that term means. To find out, one have to click on their “FAQ“, and then you’ll find the explanation of what it means, in their own words:
We have three individual parameters (index scores) for calculating the total Cloud Index score:
CPU Performance via Unixbench
Disk Performance via IOPS
Average Bandwidth/Throughput Performance via CDN tests on various global edge servers.
We take the raw numbers and put them in to our system. For each individual parameter (CPU/Disk/Bandwidth), we assign a score between 1 – 100. The following formula is used for each parameter:Providers are scored as follows: (Provider Raw Score / Max Raw Score) * 100
In my own personal opinion, this is a recipe for user’s confusion. As an internal metrics used by Cloud.net to measure the performance of each providers in their platform, it’s completely fine. But as a public measurement units for users looking to find a quick way to compare providers, it’s very opaque.
For example, here’s what it looks like in practice:
Out of three locations, you have two that doesn’t have any cloud index. One doesn’t even have uptime average listed for some reason. For a completely new user to the platform, without looking at what “cloud index” means, you have to make a decision basically based on nothing, unless you want to do further research or test one by one. This arbitrary “cloud index” number, for these new users literally means nothing since it doesn’t translate to any typical real world measurements for servers, such as specifications, performance, price, or real testimonials from users in that zone.
Saying that, I can see what they’re trying to achieve here. They predict the confusion that might arise from having so many providers listed in the same locations, and they try to simplify the decision making process to one single number. But in order for that number to be meaningful, it has to be explained comprehensively in a single page filled with measurement formulas. Unless the target market is CIOs who loves having to go through hoops to make a single decision, it’s not gonna work. Average users will most probably just go for the cheapest providers, since they don’t really know what they’re actually losing, especially if both the cheapest and more expensive options have similar uptime, of which the figures are the biggest UI element being displayed.
Lack of control , and some other Weird Quirks
The other odd fragmentation issue that i experience from trying out multiple providers on Cloud.net is big differences of OS templates offered by each providers. Some for example, only offers Debian 8 variety, some have Debian 7 too. One locations for some reason did not offer Ubuntu LTS, but regular release of Ubuntu. Some offer Cloudlinux, some don’t. If cloud.net, or Onapp have direct control of the providers, i’m pretty sure they’re gonna streamline all of this and make OS templates similar in every single locations. But they didn’t seem to have control over this issue.
This further highlights the need for more explanations in their marketplace page, especially because templates being offered can actually be a meaningful differentiation and value proposition by each providers, and further incentivize their active participations in the platform by competing with one another.
Another criticism and suggestions i have about their platform user experience is it’s not really that snappy. Rebooting, shutting down and starting up doesn’t feel instant. There’s also no snapshot capability yet. You have to take backup manually, and the backup can’t be easily migrated to another zone. But they’re working on this, according to Terry.
Benchmarks
All of that out of the way, it’s benchmark time. Since i’ve tried so many locations in a very short period of time, this benchmark doesn’t in any way try to illustrate the reliability of provider’s offerings. In total, this benchmark will cover 9 providers.:
| Provider Name | Location | RAM | Disk Capacity | Bandwidth | Price/Month |
|---|---|---|---|---|---|
| Quadranet | Dallas, US | 1GB | 40GB | 1TB | $7.6 |
| Togglebox | Dallas, US | 512MB | 20GB | 500GB | $3.80 |
| Clouvider | London, UK | 768MB | 20GB | 500GB | $7.60 |
| Jolt | Nottingham, UK | 512MB | 20GB | 500GB | $3.80 |
| Qweb | Blasserdam, NL | 512MB | 20GB | 500GB | $3.80 |
| CL8 | Limassol, CY | 512MB | 20GB | 50GB | $3.80 |
| Advantage | Palmerston North, NZ | 512MB | 20GB | 500GB | $13.70 |
| Vultr | Paris, FR | 768MB | 15GB | 1TB | $5 |
| BuyVM | New York, US | 1GB | 20GB | Unlimited | $3.50 |
BuyVM’s New York KVM Slice (1GB Ram) and Vultr Dallas (768MB Ram) will be used as comparison. Click on the provider’s name to get the full bench.sh result.
I’ve selected some figures from the benchmark result of each of them to compare in a chart. Here they are:
Overall, performance-wise, Cloud.net’s offerings are really good. But really, these providers are pretty known good quality, so it’s not really a worry in the first place. To me, at this point, the advantage of using Cloud.net instead of buying directly, is price. If you buy directly from Jolt, for example, their ‘Cloud VPS’ line starts at 9.95 pounds per month, while if you’re buying from Cloud.net, you can get them starting at $3.80, with more bandwidth allocation. Some of these providers, like Softlayer, for example, don’t even offer public cloud service directly. One special note i got though, is for provider in exotic location like CL8 in Cyprus. 50GB bandwidth, really?
In short, there is a place for Cloud.net in the market. People might hate middlemen, but if the middlemen can act as a good buffer for average end users to get access to these big enterpris-level providers, they definitely have values. The issue that remains is will Cloud.net able to truly improve their user experience, level of support and integration between providers. Because Linode, DO and Vultr, already have a bunch of locations under their belt, and if Cloud.net is unable to control, monitor and support customers in each different zones, their value proposition will just gone out the window. I have good hopes, though, that they can improve.
Thank you for Cloud.net for giving me the chance to try out their platform further.