Portal Home > Knowledgebase > Articles Database > VPS.net are incompetent
Posted by perholmes, 09-08-2011, 03:51 AM Hi, I want to describe three months of tech support hell with VPS.net, so someone else can use it to inform their decision. It started with me building a CRM server on the Atlanta cloud as a test. It was a successful test, and we started using it. The problem was that it's primarily used from Italy, and we had major latency issues to Atlanta, so we asked to have the server moved to the London cloud, where our other servers have perfect traffic to Italy. We've moved servers between 100tb.com/VPS.net datacenters before, it never was a problem. Initially, we tried to have it moved to 100tb.com in London, since VPS.net and 100tb.com is the same infrastructure. One of the lead techs said no problem, but we had to wait for some backup snapshot thing to be reenabled so they could move the snapshot. For weeks, he said it would happen in the next two days, but after 5-10 updated tickets we stopped believing. In the meanwhile he also went on vacation for 3 weeks. OK, so we contacted VPS.net to just move it to the VPS.net London cloud. Over the next many tickets, we got responses ranging from "no, it can't be done" even though we've done it before, to "I'm one of the lead engineers, I'll personally do it tomorrow". I have at least 10 times sat ready to manage downtime for a supposed server move. But nothing has ever happened. It's now been nearly 3 months of repeated commitments and bizarrely contradictory responses. I guess that eventually, I'll have to build a new server and rsync our data over, which is a major task. But our latency to Atlanta is unbearable, so we have to do something. Anyway, if you're considering VPS.net, then here's the story for the record. Best, Per
Posted by relichost, 09-08-2011, 04:33 AM Hi I can see how the responses have upset you, but if you knew about your traffic requirements, why didnt you start in the London DC ? do you still use VPS.net ? Thanks
Posted by perholmes, 09-08-2011, 05:00 AM Hi, Because I honestly didn't expect it to be a big deal. But the CRM system we use makes a lot of small requests, so having 40 ms of latency performs a lot better than 200 ms. Atlanta was picked because the East Coast typically has the same latency to Europe as to the West Coast, so it would supposedly be the sweet spot in the middle. But Italy is terrible over the Atlantic, bad enough that it's better to put the server in London and double the latency for the West Coast users. Still, why promise 10-20 times that it'll be done the day after, interspersed with comments from other techs that it's not possible despite the fact that we've easily done it before? The lack of knowledge of their own systems is incredible. Yes, it's still with VPS.net, but I'm less and less confident about it. At present, I'd be looking at probably a full week of work to build the server again from scratch and migrate the data. It's a cost/benefit thing at this point. But no, not totally a fan of VPS.net. Per
Posted by relichost, 09-08-2011, 05:07 AM Its a shame you seem to have to stick with VPS.net if you have lost confidence, but I know what you mean about having to build servers from scratch, it can be a pain
Posted by Martin-D, 09-08-2011, 05:36 AM How about getting in touch with one of the techs who moved it before? Maybe they can offer you a sane reply and get the work done?
Posted by 6PS-Chris, 09-08-2011, 09:53 AM VPS.NET really should be avoided. Wen have had nothing but problems from downtime (17-21 hours not uncommon) to techs who are completely incompetent. I would reccomend any other provider for someone considering VPS.net. DO NOT work with these guys, you will regret it. -Chris
Posted by perholmes, 09-08-2011, 03:37 PM Hi, Just to be salt to the wound, we have a possible issue with a log file that has gone berzerk, maxing out the file system, taxing the machine so much that we can't get a solid root login. I've created a PAID support ticket 12 hours ago, and have yet to hear anything. Best, Per
Posted by Techy, 09-08-2011, 03:45 PM Hrm seems odd. I'll let Terry know about this thread.
Posted by bergholt, 09-08-2011, 03:45 PM Doesn't sound good. During today and yesterday, I've heard a fair amount of people having problem with their VPS's at VPS.net. Would paying an external sysadm to do the VPS migration be an option perhaps?
Posted by Jacob Wall, 09-08-2011, 04:11 PM Terry or Rus will surely make it right.
Posted by dediserve, 09-08-2011, 05:06 PM Some providers will offer you a free migration service if you are having troubles...
Posted by Ronald_Craft, 09-08-2011, 09:30 PM @the OP: If you have any ticket numbers you might want to post them here for reference when the VPS.net guys take a look. Also, I am interested in hearing the issue from their side as well. It's not that I'm doubting your story, but I like to have both sides perspective on things before I jump to any conclusions. If what's been said about this process going on for 3 months - well, it's pretty unacceptable. If techs are giving you different answers then that means that there's a lack on internal documentation as there should only be one answer that you're being given on whether you can move your VPS or not. They shouldn't be saying yes, no, yes, no, etc. Good luck and hopefully this gets sorted out soon for you.
Posted by inwebdev, 09-09-2011, 08:18 AM My VPS has been down for the last 48 hours and I'm also on the Atlanta cloud. This has been by far the worst hosting experience I have ever had. worst 48 hours ago I received this message: We are having issues on this cloud. Server admins are working to resolve this as soon as possible and start your vps. Please check our status page status.vps.net for further updates. 46 hours ago: Your VPS is queued to start. But we can't check where your VPS is in this queue. My response 7 hours after that: Well it's still down 7 hours later. How long is that queue? 23 hours ago: Your VPS is starting now and will be available soon. We apologise for the inconvenience. 17 hours ago: SAN synchronization should be completed in 5-6 hours and cloud will start work stable. Please check status.vps.net for the final updates. 15 hours ago: 84% of synchronization already completed and it should be completed in terms i mentioned before. 3 hours ago: I am sorry for delay. Your VPS is starting now and will be available soon. I'm actually still down at the moment with no response from VPS.net for the last three hours. I have had nothing but problems with this company. Last month alone I have had probably 5 days of downtime. On the 15th 16th and 17th of August I woke up with my server offline . This was what I was told then: your VPS is still booting up. But it's very-very slow. The reason is that we have SAN synchronization in ATL-D keep going for a few days already. Unfortunately we can't do nothing with this but only wait when it will be completed. I'm done with this company. I'm just hoping I can get access to my data at some point today. I guess I should have kept local backups.
Posted by JohnnyUtah, 09-09-2011, 08:30 AM vps.net is a real nightmare. Horrible downtimes. Worst support.
Posted by durbz, 09-09-2011, 10:04 AM I echo many of the other thoughts in this thread. I work for a company that utilises VPS.net and we've had nothing but trouble with them - horrendous I/O speeds, spontaneous crashes, very slow support response times and the inability to reboot via their control panel 95% of the time. We usually have to wait 5-8 hours for the ticket to be escalated to an 'L3 Tech' who can barely string 1 line of English together. They'll either say the VPS overloaded (which is never really the case - we've tested this), an issue with their hypervisors or there is SAN maintenance going on and we have to wait 2-3 days for that to be completed. I honestly can't count how many times they've told me they're working on the SAN now. For these very reasons, we intend to look for an alternative provider/solution.
Posted by sailor, 09-09-2011, 10:36 AM i think some will also give a free first month if you ask for it so they dont double pay. I am sure you will be able to get it handled by working with the provider if you want to move or stay. good luck.
Posted by inwebdev, 09-09-2011, 02:50 PM This is of course ongoing. I'm still down and this is going on over two days now. Here is their latest message which implies that the VPS is online but of course it is not. btw, sorry for jacking your thread perholmes. We apologize for the inconveniences. VPS is now booting. SAN is not fully working unfortunately. We are about to perform emergency upgrade on ATL-D to resolve the performance issues. The maintenance window will be from Friday 9th September 2011 10pm PST -> 12am PST (2 hours). --
Posted by MannDude, 09-09-2011, 04:30 PM You can usually spot current issues with their service by looking at what is being said on Twitter: https://twitter.com/#!/search/vpsnet https://twitter.com/#!/search/vps.net Or here: http://status.vps.net/2011/09 Sorry about your experience, but it seems that their SANs are often to blame. (From what i've gathered from complaints and threads here, have never been a customer of them) Maybe their new Japan cloud will perform better if built taking into considerations issues with their current clouds? Last edited by MannDude; 09-09-2011 at 04:33 PM.
Posted by perholmes, 09-09-2011, 05:32 PM Hi, Their SANs are apparently fried. Right now, our VPS has been down for 48 hours. We've intermittently been able to gain root access, although we have to wait almost 60 seconds for a response to any SSH command, so the machine is really struggling. The irony is that VPS.net had done what they had promised a dozen times, I'd be on the London cloud and not know about it. They even had me purchase extra nodes temporarily to build the London machine before killing the Atlanta machine. Right now, we can't even move, because all our data is on the machine. But yes, once the dust settles, I'm going to build a new machine with another provider, and Rsync everything over. It's a major task, it's an email/CRM system with hundreds of thousand of files on it, 15 years of company email + attachments, and custom Apache modules to server the CRM system. It's no picnic. I hope that VPS.net doesn't think that this is somehow a small negligible issue. For us, this is a walkout issue. After this kind of outage, and a total information blackout as well, it's essentially impossible to trust VPS.net again. It doesn't make it better that I get different responses from the techs that do respond. So I still have no idea what the heck actual problem is. Per
Posted by perholmes, 09-09-2011, 05:36 PM Hi, I should add that thank God, all our email funnels through a couple of Google Apps accounts on its way to the CRM system, which gives us a mechanism to access our email if the CRM system is down. Google Apps has never failed us, they have a different idea of what "mission critical" means. The few times Google Apps has had an outage, the apology email has been 10 pages. Best, Per
Posted by IGobyTerry, 09-09-2011, 06:05 PM I think this post is a little dramatic and clearly intended to stir up some unnecessary drama. No SAN is "fried" infact, they're all online, as you know from your ticket reply. To the OP: Can you contact me at terry@vps.net? I believe I worked your ticket at one point in time, but it's been lost in the shuffle. We can probably do what you need us to, I just need to assign an L3 staff member to it.
Posted by perholmes, 09-09-2011, 06:15 PM Hi Terry, I'm the OP, and I don't think it's dramatic at all. I'm f***ing furious about this. I have endless tickets for the last 3 months, some of them paid tickets, with repeated promises on your side to move the server, you've made me buy extra nodes to contain the server move, you've made we sit and wait up nights to handle downtime. Of course the SAN is not fried in the sense of losing data. It's most definitely fried in terms of being available. A typical root login will result in a broken connection. If I get a password prompt at all, it takes 60 seconds, and then the password entry times out. Only once in the last 48 hours have I been able to actually do a login, to verify disk resources and running services, which dispelled my fear that there was anything wrong with the resource usage like a log file gone berzerk and making the system inoperable. Since then, I've gotten the most varied ticket responses from you, including "the server is running fine now", which it isn't, I can barely ping it, and it doesn't have enough SAN bandwidth for a friggin' root login. I mean, come on! I don't know what you can do additionally, so I don't know what the point is of emailing. How about just updating status.vps.net and not leave us hanging? It hasn't been updated in 12 hours. I'm sitting here counting the seconds. I did get a ticket update that some nondescript SAN update would be done to increase performance. I don't need "increased" performance. I just need any "performance" at all. If I can just be allowed to read 500 bytes from the SAN, I can root login without the root login process crashing. I got "broken pipe" a couple of times as well. This is not a good day. This system is our email, task and CRM system. We can only respond to customers when they write to us and we intercept the emails in Google Apps. But we have tasks in tickets in the CRM system we can't deal with, so we now have tech support requests with out customers that are almost 3 days overdue. It would be almost impossible to dig them out of Google Apps, this hack only works for recently arrived email. Per
Posted by Visbits, 09-09-2011, 07:47 PM Yep sounds about right, there is 2 thing I've seen about VPS.net. 1. Its flexible and fast. 2. The san storage is down 99.9% of the time! *giggle* (clouds) lol
Posted by Cloudstra, 09-09-2011, 08:23 PM Let me guess...you guys offer a free migration service?
Posted by inwebdev, 09-09-2011, 08:41 PM A little dramatic? I have about 50 clients sites which have been down for 3 days. You have to be kidding me. This is all you can say? I'm guessing you want to help the OP out so they will delete this post. Good luck with that one. Direct quote from my support ticket: "SAN is not fully working unfortunately. We are about to perform emergency upgrade on ATL-D to resolve the performance issues. The maintenance window will be from Friday 9th September 2011 10pm PST -> 12am PST (2 hours). " This is definitely a walkout issue for me as well. I should have left a long time ago. This sealed the deal for me.
Posted by JohnnyUtah, 09-09-2011, 08:55 PM Dramatic... Lol. I would be way beyond furious. Come On Terry and spare us with those useless posts. The op was on par and right on. This is 100% justified. Last edited by JohnnyUtah; 09-09-2011 at 08:59 PM. Reason: Add details
Posted by Text12489, 09-09-2011, 09:22 PM Hi, I totally agree, their support is based out of an small European country. (I can get logs, the VPN ran through this country). They don't speak English very well. But hey. if they can sell servers without offering a English speaking support team, they are doing a great job keeping clients. We were also affected in the SAN downtime, just a question, isn't the cloud about redundancy and uptime? - Nic
Posted by Visbits, 09-09-2011, 11:40 PM Clouds are proven to have nearly 2x the downtime other setups are. Lots of overhead with SAN and all that BS. The cloud is a hyped ******** story. You can't have a *cloud* VM, you can deploy applications as a cloud where all systems are redundant, a VM is just that a VM. Same **** as a VPS or Dedicated Server. Sigh. Some never learn.
Posted by Cloudstra, 09-10-2011, 12:05 AM lol...funniest post ever, especially if you seriously believe what you just typed!
Posted by Kusai, 09-10-2011, 12:53 AM +1 Cloud is still not matured yet. We have seen more cloud failure posts in the past 2 years than VPS's going down.
Posted by Amphibulus, 09-10-2011, 01:27 AM I was with VPS.net 2 yrs ago or so, and they was as bad as people describe it. Since then I use cloud server are Rackspace, and it's never down, 100% uptime all the time. That's the place to go if you are looking for something really reliable.
Posted by perholmes, 09-10-2011, 03:12 AM We're now on day 4 of server unreachable. WTF? Per
Posted by perholmes, 09-10-2011, 03:26 AM Hi, Shame on you VPS.net. 4 f'ing days of downtime, and only a generic message posted on your blog about what sounds like a multi-phase infrastructure improvement over the next month. Please don't hold our data hostage. At least allow us to access some kind of backup or something so that we can start rebuilding our lives with another hosting provider. In all my years of hosting, this is the worst incompetence I've ever seen. You've already lost us as a customer, this is now a matter of which memory you want to leave. Per
Posted by perholmes, 09-10-2011, 03:30 AM Hi, And I'm going to request a credit card chargeback with the bank for the nodes you asked us to purchase a month ago for the server migration that you never did. F-you VPS.net Per
Posted by perholmes, 09-10-2011, 03:34 AM Just received some generic ticket response that they're performing scheduled maintenance. For 4 friggin' days? Do these people understand what "scheduled maintenance" even is? Scheduled maintenance is that you plan for the server to go down. It doesn't just randomly disappear from the internet for 4 days without an understandable word from support that even begins to explain or set some kind of expectation for when the "scheduled maintenance" will be complete. Scheduled maintenance my butt. Per
Posted by DMEHosting, 09-10-2011, 03:51 AM I think there are far more SAN cloud outages than Grid cloud outages. I just don't see how those SAN's can withstand all the pressure all these random VM's put on them. Cloud providers want to maximize revenue and want to throw as many VM's on each SAN as possible. Eventually they are going to give. I wish OnApp had a way to do a Grid type cloud instead, use each servers' local storage and replicate it across like AppLogic's method. That way, instead of bringing down 100's of VM's that are off a single SAN, the grid will only affect minimal VM's if nodes go offline...if any VM's in that fact.
Posted by perholmes, 09-10-2011, 03:53 AM Hi, As an end-client, I have to echo that. The cloud failures we've had have all been SAN failures. The rest of the hardware seems to work fine. I'm becoming quickly sceptical of this way of hosting, although all our servers are in some way running from some SAN somewhere. Best, Per
Posted by perholmes, 09-10-2011, 05:48 AM Hi, Since VPS.net are having a news blackout and it seems many are affected, I'll post our updates here. Our server has just booted, but it's loading extremely slowly, something like 1-2 minutes for a full page load. This is not a public server, so it's not hit by public traffic. Best, Per
Posted by Chris - Whitesystem, 09-10-2011, 10:27 AM outch..., I highly suggest you reading and learning much much more about Cloud computing (related to web hosting as well)...
Posted by Keiro, 09-10-2011, 10:36 AM ... And to think I was going to resell VPS.net cloudy VPSes in addition to our SolusVM VPSes. I'm not sure if I want to be tied to a provider that has so many SAN issues. o_O;
Posted by perholmes, 09-10-2011, 10:57 AM Hi, I'm the original poster. Our VPS has somewhat come back, in the sense that the site will load now, but very slowly -- in fact so slowly that the email polling script in the CRM system crashes sometimes and pulls certain emails several times. But this issue is making me start to wonder about the feasibility of SAN based VPS, or at least the way it's done by VPS.net. Our original reason for wanting to move the server to London would be to get better latency, but seeing how the application behaves under this kind of SAN load, I'm starting to wonder if the real problem is that the SAN is oversold. Then it doesn't matter if we have 8 GB of ram and 6 GHz of CPU -- database lookups are still going to be tied to the SAN speed, which I'm increasingly feeling is abysmal at VPS.net. So, now that we know that our data is safe, we need to learn a lesson about picking VPS providers. I never realized that the SAN is probably the bottleneck, and that no amount of CPU or RAM can make up for a slow SAN. I'm going to look for providers we can move this server to, and the number of servers/users per SAN is suddenly my #1 question after uptime and redundancy. Best, Per
Posted by Visbits, 09-10-2011, 10:58 AM You can load balance a VM on multiple servers at once? COOL STORY BRO. If that HN fails then the VM is down till it boots somewhere else. With a *grid* or true CLOUD as others have mentioned its fault tolerant for failures. VPS + Cloud = ******** story. Your application has be to be designed for it. Go spam your signatures elseware. WhiteSystems, Mr hetzner reseller. You can talk once you have your own facility, own network and own equipment. Last edited by Visbits; 09-10-2011 at 11:03 AM.
Posted by perholmes, 09-10-2011, 10:59 AM Hi, On that note, does anyone know how to test SAN performance from the command-line? Does anyone know any good commands or small scripts that write and read a large file and measure the MB/sec? Or is there a command-line benchmarking utility? Then you could also test with a lot of small files. I'm going to do this for all our servers. God knows if we're getting any of the performance we're paying for. Some of our CPS servers have stupid amounts of CPU and RAM and are still a bit slow. I want to test the SAN for each of our servers. Best, Per
Posted by Visbits, 09-10-2011, 11:02 AM Most funny post ever! NO IDEA WHAT ABOUT THE TOPIC!
Posted by perholmes, 09-10-2011, 11:13 AM Hi, Our server seems to have normal performance now. What ever that means! Per
Posted by Keiro, 09-10-2011, 11:20 AM Aggressive much? We don't have our own facility, nor have our own network. But we do own our equipment. However - I wouldn't tear someone else down for that... I mean, we all have to start somewhere, don't we? Ontopic: VPS + Cloud = not ********. You just need to design your hardware + network + SAN correctly. Are you saying that you cannot have load-balanced VPSes/cloudy VPSes? We do it all the time with other servers, for web servers, databases servers... what makes it impossible for HA to work on VPSes? Seriously? The lesson learned from this, I'd say, is always check for how many users per SAN for a given provider. ... And to correctly design YOUR SAN to account for this!
Posted by Chris - Whitesystem, 09-10-2011, 11:28 AM @Visbits, Why are you so aggressive my friend? calm down..., so one provider fails and the concept of Cloud is all a joke because of them? I don't want to start an unnecessary discussion here, as these threads have already been discussed a thousand times here... """You can load balance a VM on multiple servers at once? COOL STORY BRO.""" actually yes sir, you can... edit: Last edited by Chris - Whitesystem; 09-10-2011 at 11:35 AM.
Posted by Visbits, 09-10-2011, 11:33 AM Don't talk like you do something magical and you wont get singled out. Your APPLICATION that you are *HOSTING* has to be configured for fault tolerance. A cloud is VPS + SAN storage, nothing more! Some peoples *cloulds* use local storage, cough storm cough. The VM on a cloud design was for people who couldn't utilize a full dedicated server, no reason to let those cpu cycles go to waste right? Sure you get features like failover ect but how many times does a server just explode? I'll put money on a multi server load balanced setup being more reliable than a SAN powered cloud, you have to spend thousands on network alone to have a reliable san setup, and iSCSI MP is finicky performance wise in comparison to FC, so be prepared to shell out top dollar there to gain reliability at that level + all of your san + HBA in servers. So yeah, the whole cloud idea for virtual machines is rather stupid. It makes more sense to use a VPS and the performance will almost always be better in the *web hosting* industry. So you were saying?
Posted by Keiro, 09-10-2011, 11:41 AM I'm not. There's no reason a cloudy setup can't be done with HA. You're more or less doing the same thing like you do with dedicated servers in a cluster setup. That's not hard to do, is it? Besides that... And? You're doing the same thing anyway for dedicated servers. I don't get your hate on cloudy VPSes. As for how many times a server explodes... we haven't had one explode. We did, however, have a hard drive explode within 3 months of ordering a server. First hardware death we've had. You're still doing the same for dedicated servers. Admittedly, at a lower cost and with different setups... but again, problem? Really, those VPSes're just miniature dedicated servers, cloudy or no. There's no real technological reason why you can't have HA. So long as you correctly design for high availability... you shouldn't have a problem. You seem to be intent on ignoring the fact that so long as you correctly design the cloud platform, you won't have that issue. Why?
Posted by Visbits, 09-10-2011, 11:46 AM Cloud VM are pointless, that's what I'm saying. There is nothing it offers over a standard VPS or Dedicated server. The true *CLOUD* is a redundant configuration of web servers and database servers designed to resist a failure. Not a VM that can be resized just like a VPS and moved around.. just like a VPS.
Posted by Amphibulus, 09-10-2011, 11:57 AM Move to Linode.com or RAckspace cloud server offer.
Posted by JasonD10, 09-10-2011, 03:03 PM This thread has derailed quite a bit, so I'm just going to simply comment on Cloud, what it is, why it IS better, and leave it at that. Everyone is welcome to their opinions that is what good discussion is all about, but incorrect information is also rather misleading to the others reading this thread. So, here we go... First just to clarify something for all the readers virtualization is a predecessor, and building block of what Cloud is. Virtualization in and of itself is not Cloud, just as a frame is not a car. It is however a very important component of it whether Xen, VMWare, etc. The hypervisor plays an important role to Cloud. When I think of VPS in comparison to Cloud I think of a few very important missing components that do make it sub-par. Here's the most important things: 1) Location independence 2) Scalability 3) Reliability Let's break it down further and explain why each thing is so important, and why central storage (whether supplied by a SAN or any other method) is absolutely critical for this. Location independence is the single most important factor of Cloud. Location independence is what provides a truly scalable, and reliable platform (points 2 & 3!). Location independence allows the provider to move around that environment with complete freedom. It's not tied to a server, a rack, or even the same physical Data Center. Without location independence, it is not Cloud. So providers that don't have this (Ie: Rackspace Cloud Servers) don't qualify. Scalability. You say VPS is scalable? Sure it *technically* is scalable. However, it's not practical. Why? Because a VPS isn't location independent. Those environments are tied to that server so as a provider, are left with the choice of not actually utilizing the server to allow for scalability. What happens when your customer asks for an upgrade in a VPS and that server is full? You can't provide the customer with the resources or you have to do a time consuming and extended outage to provide those resources. Depending on the amount of data, this could easily take many hours of downtime. Cloud however due to location independence can scale up on the fly with just a simple reboot to that individual Cloud Server. A customer can go from a fraction of a CPU core and 256MB/RAM to 16 CPU cores and 24GB/RAM or whatever the Cloud is built for on that provider. Sorry but you're not going to see that happen with VPS providers or a Dedicated server. If it does happen with a VPS then the provider is vastly underutilized their resources and it's either by sheer luck that they had it available, or expect them to go bankrupt because they can't operate their business profitable. Reliability is increased as well for again.. location independence. The physical server is having an issue? No problem! the Cloud Server is restarted on another available physical server and the customer will probably never know it happens unless they look at their uptime and see it rebooted because that's all it was.. a 2 minute reboot. No lingering issues, and as a provider we don't even need to rush to fix it as the Cloud can handle several complete server losses without a hitch. Now, I'm really going to blow your mind because you seem insistent upon the idea of Clustering and Cloud. Clustering has nothing to do with Cloud by itself, however it's something I've commented on many times here as I believe Cloud Clusters are the "truest of true" Cloud platforms as they provide exactly what you are talking about, which is a multi-tiered application designed to replace that of a traditional, physical, server cluster that is now entirely on the Cloud. I always heard a picture is worth a thousand words, so to avoid writing a thousand words let me show you one. That's 7 servers, a NAS, a layer 7 switch, two routers, and a fully load balanced and redundant LAMP stack. It's all privately connected, directly routed private tunnels and dedicated resources.. oh, and there's another one exactly like it in another Data Center for complete redundancy of the entire application. Guess how long it takes to build one of those? About 7 minutes from start to finish including creating the volume's and IP'ing everything. I just did it to check and see. It's physically impossible to provision a Cluster of this size, power, and magnitude to reach everything that can be done in the Cloud outside of the Cloud for the same cost, and in the same time.
Posted by user_204207, 09-10-2011, 08:15 PM I was a vps.net customer for 1 year, I've been a Linode customer for 1 year. We've used Softlayer cloud for work. There is nothing wrong with cloud hosting, the problem that the people here have is entirely down to vps.net, not cloud hosting. Many companies manage cloud hosting successfully; vps.net don't.
Posted by Kalriath, 09-10-2011, 08:52 PM I'm impressed at your ability to redefine "cloud" so that no solution but yours qualifies. Even Amazon EC2 and Microsoft Azure don't let you shift physical regions. I prefer Wikipedia's definition of Cloud computing. Especially how location independence is described as the location independence of the client, not the server.
Posted by tchen, 09-10-2011, 09:59 PM AppLogic is definately a very nice template system for cluster deployment. The fact that you deploy it on the underlying replicated grid doesn't really mean much (except in a few cases), as if you had the same HA templates, you could have done the same on any collection of VPS/dedicated machines you happen to have. We can nitpick though and shift about the procurement of the machines from you, the IaaS provider, or me, the client and draw down or inflate the deployment times either way. In a way, I'd agree with Visbit in that if you really want 100% HA, just having a image backed VM, whether it be a SAN or a DRBD like grid is not sufficient. BTW, the reference scalable LAMP cluster you have will suffer some availability issues when the Dbase appliance moves or goes down. Likewise for the NFS storage. On the flip side, its one of the few templates that DO rely on the underlying grid storage to be redundant and failsafe. Move up the appliance stack and use the replicated versions. At that point, it's moot that the underlying blocks are replicated. If anything, I start worrying more about flutter and network split issues, considering you have a single monitoring service. Cheers.
Posted by Cloudstra, 09-10-2011, 10:13 PM Nicely put.
Posted by tchen, 09-10-2011, 10:15 PM 1. From a users' standpoint, quick provisioning. Especially when you get to the point of allowing user loaded images. 2. The fact that you don't have to reinstall everything from backup is a plus to some when dealing with image backed cloud VMs. The 'downtime' from that can be higher than SAN availability issues for a lot of people. 3. The resizing is an interesting one. Although any provider can tweak the CPU/IO schedulers and move VMs around, I haven't seen it as pervasive in the standard VPS world as you make it sound to be. The lack of built in control panel support and the relative homogeneity of the market means that the only people providing that service are the cloud providers, who have already tweaked a lot of things.
Posted by Cloudstra, 09-10-2011, 10:20 PM Hmmm...someone has taken some angry pills. Do you really believe this tripe you are typing? If so, skip back over to the VPS board. A cloud VM has a number of benefits over a traditional VM.
Posted by reloaded, 09-11-2011, 05:51 AM NONSENSE! Their homepage says "100% uptime" so they must be doing it right!
Posted by Text12489, 09-11-2011, 06:26 AM Don't fall into that trap, I was just told by their managing director Rus Foster that they have no SLA, so hmm. Cheers, Nic
Posted by dediserve, 09-11-2011, 10:04 AM Lot's of clouds can and do give 100% uptime and deliver on the 'promise' that is true cloud Infrastructure as a service. Lots of re-badged vps local server/disk solutions, and poorly managed 'server' SANs are just giving cloud a bad name.
Posted by tchen, 09-11-2011, 06:16 PM Looking at the uptime list from CloudHarmony, the only ones that have managed 100% at least over the recent reporting one year (as of Sept 2011) interval are: dynomesh (launched Jan 2011)eApps (launched Apr 2011)force.com (a PaaS)stratogen (launched Jan 2011)Storm Cloud (MI DC only)ElasticHosts (both DCs)Google Apps Engine (partial service degradation) Even some on that list don't really belong there. - A whole bunch are youngins, not even reaching a year of uptime (as noted). - GAE service degradations (in July 2011) have impacted a segment of their users. The automated monitoring only goes so far to check whether someone responds, so actually checking against facts takes this off the list. - Storm Cloud's single DC report is just lucky IMHO. Really, the only ones that technically belongs there are force.com and elastichosts. The former, had an double failure incident in July last year that took out the service and the backups. ElasticHosts is barely more than a year old. In any case, if we move down the scale of 9's you start bringing in a few datacenters of the majors. Most of which aren't even complete at about three 9's. Dediserve's on that list too - I'll leave you to find it
Posted by dediserve, 09-12-2011, 03:40 AM Hi tchen, If you are going to use cloudharmony as a benchmark, you should note that it monitors a single instance on a cloud - not the full platform. For example, our Dublin platform isn't listed, our Dallas platform shows one outage - which was the impact of the colo4 power issue - and London shows local issues to the monitored node. Otherwise we've delivered 100% uptime for clients for over 27 months and counting... You can easily verify that with a quick google or twitter search
Posted by tchen, 09-12-2011, 08:35 AM Okay, so 100% as a collective service/platform - as in the provisioning API is up and running, and at least some capacity is available for people to spool up replacement instances. I'm all for getting people into the frame of mind of thinking that individual DC's or nodes are should never be considered 100% reliable and should at least make sure they have some application/appliance level HA. But let's not pull too much of the marketing cart ahead of ourselves here. Because unless you're entire service is actually fronted by anycast, a HA load balancer, or at least a switch that can reroute packets to a new instance automatically, then the cloudharmony instance/DC monitoring stats are still quite a valid and informational thing to look at. P.S. by our 'service-level' definition above, don't all multi-DC providers by corollary have 100% uptime? VPS, dedi, cloud, or otherwise?
Posted by dediserve, 09-12-2011, 08:45 AM 100% uptime is physically achievable, and cloud technologies make great strides in that direction at an amazing price point, but nothing is ever 100% guaranteed. The vast majority of our customers have enjoyed 100% uptime across all our platforms, even with single instances. Communication and response are as important as well architected and maintained infrastructure and platforms. I'm not sure how you can extend 100% uptime claims beyond single sites? If three hypervisors of a 100 server grid have an issue, I don't conside that a 'cloud outage', rather a localised even that the platform works around. We're getting rather OT here in terms of thread hijack also
Posted by Scott.Mc, 09-12-2011, 09:17 AM It's funny how VPS.net's poor SAN implementations ends up in a discussion over cloud and has some rather silly opinions that can't differentiate between cloud and clustering. I mean why would you even use VPS.net, they've been having continued SAN problems since they first launched and it still continues to this day. The very fact that when you encounter an outage it's several hours to days alone should make the deciding factor on weather they are even considered worthwhile or not. Just go through their status page, there's 36 outages/problems since Aug alone and apart from one they have all been SAN/storage related. Naturally with so many different SAN and storage solutions in production they are going to dominate the status pages but the frequency and length of some of them is outrageous. It's been this way since they started.
Posted by perholmes, 09-12-2011, 12:04 PM Hi, I'm the original poster. Now that the server is somewhat back online, I wanted to post a follow-up, as we have experienced data-loss due to the outage. This is a serious concern that goes far beyond performance or downtime concerns. The server runs a CRM system called Cerberus Helpdesk from WebGroupMedia. We have 15 years of email and attachments in there, so this is a large database, with additional off-site storage at Amazon. During the outage, the backend bandwidth dropped to near zero, and that was such an unexpected environment for the application, so email-polls kept timing out halfway through, resulting in thousands of duplicated emails. However, today we're discovered that there's also database corruption. I don't blame Cerberus Helpdesk at all, this is a very well-coded application, but I blame the outage. The symptoms we've discovered are that the Search Page produces an error about database corruption, and Drafts are no longer saving. We're discussing this with WebGroupMedia, and we may need to bring in one of their techs to help repair the database. We're obviously testing all aspects of this to see how wide-spread the corruption. Our only alternative is to start the CRM system over, which is a *massive* task, but possible. Our entire email archive is hosted in a few Google Apps accounts where new email also funnels through, so we could ingest all our historical mail again. But because of load-constraints Google places on email download, we have to download the mail over a period of 2-3 weeks, or the download suddenly breaks. We tried faster when we originally built the CRM system, but it failed. Then we also have to configure the system again from scratch, set up all users, and set everybody's UIs again. Clearly, I'd like to avoid this. But I don't see how we can stay with VPS.net. Data-loss is a different kind of sin than downtime or performance. So the shortest path to safety right now is probably to get the database repaired, pick a new provider with caution, and then migrate and Rsync the entire CRM system over. Needless to say, I'm very unhappy about this, and somewhat stunned that we had data-loss due to VPS.net. I honestly do not think that they are competent, as the thread title says. I don't see others having the same problems. Mysteriously, we also have a couple of servers with 100tb.com on the SLC cloud, and besides the fire in their datacenter a year ago, we've never had an outage. Best, Per
Posted by bergholt, 09-12-2011, 12:24 PM Is VPS.net generally considered enterprise class?
Posted by Keiro, 09-12-2011, 12:26 PM Given the responses in here? Probably not.
Posted by DMEHosting, 09-12-2011, 12:42 PM I believe their backup server(s) are separate from their SAN. Can't you just roll back your VM from their backup server(s)? Or are the backups to old for you?
Posted by reloaded, 09-12-2011, 12:51 PM Here is a screenshot from my vps.net dashboard. This is for the PAID backup service. Not working too well, is it? Attached Thumbnails
Posted by Scott.Mc, 09-12-2011, 01:55 PM Are you sure you are not just speaking about a simple database and/or file system repair job? Assuming it is actual data loss (IE files missing from their SAN, which I doubt is the case) where are your backups? You can't pin the blame solely on them when you hold large portions of responsibility also, for starters you should have good backups, you should have plans to migrate providers comfortably so you don't need to struggle to migrate. Assuming you have lots of files/large files the only real problem should be waiting time, not how complicated the application is which going on what you've given and the software mentioned then your application isn't anywhere near as complicated to manage and migrate as you may believe.
Posted by perholmes, 09-12-2011, 02:51 PM Hi Scott, Yes, we do have backups, but they used snapshot image problems as the excuse for not moving the server to the other datacenter. Secondly, the images are too old now. I can't rewind the email data feed a little bit. Since we go through Google Apps, I either have to start over (massive task), or miss email lost in the restore. I don't know if it's a simple repair job. I'm going to follow the instructions given to me by WebGroupMedia, and then I'll know. But it's difficult for an application when it can't trust the stability of its own hardware. You always code for failure for stuff that goes over a network. You don't typically code to prevent against failure of the machine you're running on, beyond a sudden power off. This was suddenly a situation of scripts timing out internally that would never have timed out otherwise. It's not just one table in the database that has problems. Finally, we're less then 3 months into using this CRM, and now that the migration is complete and a success, my next step was to make some more data replication beyond simple snapshots, to move the data off-site. Obviously, you only want to do that with a machine you intend to keep. Per
Posted by mcianfarani, 09-12-2011, 03:57 PM According to VPS.NET my one VM going down twice do to a SAN issue is "Pure Unluckiness"
Posted by tchen, 09-12-2011, 05:33 PM Sorry to hear the problems you're having with the databases. One major thing to note when setting up a VM on a image-backed system is to make sure your boot sequence has automated database error checking/backup/repair before even starting the application. Because these clouds rely more on automated failover than typical systems, and thus can be a bit more lax on the hardware, you can't rely on it being a manual process. Heck, check it daily as well since stealth disk-failure, like in your case, can cause corruption too. Multi-component systems are a pain to backup and recover correctly from. Your email problem sounds new, and personally, I wouldn't have considered it either.
Posted by WireNine, 09-14-2011, 03:25 AM I wouldn't call it being unlucky if it's happening this often. As Scott.Mc said above, looks like their SAN troubles are pretty common since their launch.
Posted by inwebdev, 10-15-2011, 10:20 AM Total disrespect. I cancelled my account with VPS.NET after this fiasco. Not one apology, or offer of a discount, nothing. Good riddance. I'm in the market for a decent VPS now. I refuse to use 1and1 though. They sound even worse than VPS.NET.
Posted by perholmes, 10-15-2011, 12:58 PM Hi, I'm the original poster. I'm also looking to move our servers away from VPS.net. I simply haven't had time to evaluate other hosts, and there's a bit of calm after a major outage. But I can't in good conscience stay with VPS.net. They turned out to be real A-holes about this. The previous poster is correct. Per
Posted by sailor, 10-15-2011, 01:06 PM No - not at all but that is not their target market either. I think they fill a need out there but you need to make sure your needs are matched with the providers offerings. I am sure they will figure things out on the san issues- they seem to have a pretty good crew over there. If you cant tolerate some transient san issues from time to time though you will want to look for something a little more towards the enterprise end. Last edited by SoftWareRevue; 10-16-2011 at 08:09 AM. Reason: corrected
Posted by Sparrow-Sean, 10-15-2011, 06:56 PM VPS.NET was quite good, Terry was even better or at least to did make the offerings to ensure everything was alright and was running well - even touched base on multiple occasions, though I think with the overall demand for the service in itself they probably are pushing to fast and too hard and are running in to more issues than what they are worth. Downtime related to SAN issues were a common problem when I had used them a couple of years ago, it never really bothered me that much in the short period but they continually went down which was quite annoying and it was always seemingly SAN related. Last edited by SoftWareRevue; 10-16-2011 at 08:09 AM. Reason: quoted text removed
Add to Favourites 
Print this Article
