Cloudmin supports clusters or just a pool of nodes?

4 posts / 0 new
Last post
#1 Mon, 08/03/2009 - 21:18
getnetworks

Cloudmin supports clusters or just a pool of nodes?

Due to the extreme lack of documentation, screenshots, demo, etc. we have very little to go on with respect to what Cloudmin can really do. What we are most interested in is whether it really just supports a pool of Xen (etc.) servers, simply checking to see which servers can support the requested virtualized container or if it truly is a *cloud" management application that can support an actual Xen server cluster (one that has has a master node and many slaves), where it can provide for container management across a distributed computing environment (a true cloud), such as ones utilizing tools like OpenNebula, Haizea, DRBD, Heartbeat, etc. that allow for high-availability and very fast container migrations and/or failover. Given the sparse information, it appears it is really just a VPS manager rather than a cloud manager (essentially just providing functionality similar to what PIM does for Virtuozzo Containers). To truly be a cloud management tool it certainly needs to support a scalable cluster, and integrating technologies that allow for distributed computational cycles across containers would easily make it a premier infrastructure management tool.

Looking forward to the technical specs and/or plans for Cloudmin to know if it intends to compete with providing for deploying an EC2 like infrastructure platform or if it is just a Parallels Infrastructure Manager for "other" virtualization technologies such as Xen.

gN

Wed, 08/05/2009 - 14:19
MACscr

gN,

This link might help explain things a bit further. I highly recommend you also browse through the forums a bit.

http://www.virtualmin.com/node/10363

Mon, 08/17/2009 - 06:30
Joe
Joe's picture

To truly be a cloud management tool it certainly needs to support a scalable cluster

This seems to be a popular theory about what "cloud computing" means, but none of the cloud computing service providers offer such capabilities. ;-)

Looking forward to the technical specs and/or plans for Cloudmin to know if it intends to compete with providing for deploying an EC2 like infrastructure platform or if it is just a Parallels Infrastructure Manager for "other" virtualization technologies such as Xen.

I'm not sure we're on the same page about what EC2 is capable of, if you believe it provides any of the features you've described. EC2 can't even resize an instance in place! (Cloudmin can, if the underlying virtualization layer supports it. EC2 requires you to shutdown the instance and bring up a new, bigger one.)

There are some elements of Amazon Web Services that add capabilities like you describe (though nowhere near all of them); S3 provides "unlimited" storage, if you're willing to rewrite your app to store in S3 buckets (and Virtualmin can back up to S3 buckets), EBS provides a reasonably complete standard filesystem for file storage with respectable (but not mind-blowing) performance and good fault tolerance. But EC2 is actually pretty minimal in its level of magic; it is Xen with a new API on top. The only really magical aspect of it is that you can spin up as many as you want via the API and turn them back off when you don't want them; and Cloudmin can already do that (in addition to live resizing).

Anyway, here's what's on our timeline for the next 3-6 months:

memcached will be added to the Virtualmin standard stack in the near future, and a Webmin and Virtualmin module will arrive for management of it. This is actually pretty low-hanging fruit, for everyone (you as the application developer, and us as the folks that have to design the UI, API, and integration), since memcached is magically simple to deploy in most cases. This isn't sexy technology, but memcached is a part of nearly every large Web 2.0 deployment I know (and I know quite a few of them; Silicon Valley is a small place, and we're regularly bumping into folks with top 50 ranked websites).

More MySQL cluster features in Virtualmin, and possibly a cluster/replication "big honkin' shared" MySQL module for Cloudmin. Virtualmin already has management of users and tables across multiple databases, and management of ndb_cluster tables. Replication is now reasonably well-documented and reasonably well-stabilized in recent versions of MySQL, as I understand it, so we can begin to add those features to Webmin, Virtualmin, and possibly a Cloudmin module (based on how it is used and where it fits in the stack...a lot of this stuff is better handled at a lower level than Cloudmin). This has no corollary in Amazon Web Services, though there are several hosted MySQL options running on AWS. Some friends of ours run one such service. We happen to think MySQL is too damned important to the vast majority of our customers infrastructure to think one can just make up a new database like SimpleDB and expect people to rewrite all of their software to use it. So we'll be working on solutions that allow MySQL to scale in a generic fashion, and be better utilized in a shared resources environment. This won't excuse your applications from being designed to scale, but it'll at least provide a standard set of guidelines for what the backend database looks like in every Cloudmin environment.

Addition of GFS support, and/or some other cluster filesystems, to Webmin, and tools for managing reporting of usage in a Cloudmin context for those filesystems. This would roughly correlate to EBS.

Addition of Hadoop+HBase or Cassandra or Tokyo Cabinet support to Webmin and Cloudmin. This would roughly correlate to SimpleDB (after I just explained why SimpleDB is a crappy substitute for MySQL), as there's also an awful lot of problems for which widely distributed, eventually consistent, key-value store databases make more sense than traditional relational SQL databases. We'll leave the decisions about when to use which up to our customers.

Addition of at least one queueing service. This would be corollary to the simple queueing service at AWS. Accounting features will need to be added to whichever product we choose, so this one is rather more involved. No one (literally not a single person) has asked for message queue features, so it's further down the todo list. But, for actual large scale applications (ones that require marshaling data and resources from multiple sources with thousands or millions of concurrent requests), a message queue is pretty damned important.

Given the sparse information, it appears it is really just a VPS manager rather than a cloud manager (essentially just providing functionality similar to what PIM does for Virtuozzo Containers).

If your definition involves capabilities that AWS doesn't offer (which it seems to), and that no other cloud computing provider offers, then yes, Cloudmin is really just a VPS manager. But, it's a VPS manager that has capabilities that we consider important for calling something cloud computing (though we think quite a few of the features you want are interesting and useful, and some of them do fit within our goals for Cloudmin), and our customers have told us they need to begin offering interesting cloud computing services.

The single most important feature of cloud computing in a hosting context, I think, is that you use as much as you need, when you need it, and you only pay for what you use. It requires an API to make it possible for software to make the decisions about this usage. Cloudmin has had this from very early on during its private beta. This is the EC2 scaling model, as well, by the way. When you need more horsepower on EC2, you spin up more EC2 instances via the API. EC2 instances don't share computation units, but they do allow you to get more cores working on whatever problem you give them.

Anyway, as we can all note, the terms are still pretty vague (it's one reason "cloud computing" is such a popular term; everybody thinks it means stuff that they think is really cool, even though no single product or service in the space even begins to offer even a fraction of all of the things people are calling cloud computing). It's very early yet, and we're all (and I mean everybody working in the cloud computing space, including our competitors, our partners, and related services and products) hashing out the details of how the pieces fit together.

If what you want is to build out Amazon Web Services style products, then Cloudmin will be among the products that can help you do that. That is our target in the short term. We actually think things can be quite a bit better than AWS, particularly in the areas of UI and API. AWS is needlessly complex in many regards. But, we do think there needs to be a standardized deployment model for scalable storage, replicated MySQL and memcached, and some sort of replicated key value store; and equally importantly for service provides (and something that nothing else is attempting to address, as far as I know), we think there needs to be standard ways to account for their usage in a fine grained manner so the user can be charged appropriately.

--

Check out the forum guidelines!

Thu, 12/03/2009 - 01:00
SoftDux

Hi Joe,

I'm also interested to know more about Cloudmin, like the OP.

Does, or can, Cloudmin scale over many cluster nodes to give the end user "unlimited" resources? And with unlimited, I mean the resources aint limited to the hardware platform's limits. For example, let's say I have 4x Dual XEON Servers with 24GB RAM each, and 4x 500GB HDD's in RAID 10 (thus giving me 1TB space) - in total, I have 16 XEON CPU's, 96GB RAM, and 4TB storage - can a single VPS actually utilize all of these resources?

And, if the client wants more space, can I deploy another server like this, and he'll have another 1TB space extra?

Topic locked