[Discuss] easy clustering of applications

[tmetro+blu at gmail.com (Tom Metro)]

Edward Ned Harvey (blu) wrote:
>>Tom Metro wrote:
>> It does seem like every application has its own unique approach to 
>> clustering. There is no generalized solution.
> 
> This is one of the reasons why I'm much more strongly inclined toward
> HA at the hypervisor. ...if you do HA at
> the hypervisor, then it doesn't matter what your application is; you
> have HA of your application, no matter what it is, and the setup is
> identical for any and all types of application.

I don't think what you are describing is possible, except for a certain
class of applications, unless you are glossing over some of the
application-specific glue that gets wrapped around your VM.

At minimum, you need some application-specific monitoring and control
mechanism to detect when your app has failed and to send inbound
requests to your remaining working servers.

Then there is the storage layer, and cleaning up incomplete transactions.

As Bill Bogstad put it, "Once the servers have to start maintaining
session state, it gets more complicated."


> *By necessity* if you want HA at the application layer, then the
> application must be designed for it, and therefore setup is unique to
> your application.

Yes, but that setup doesn't have to be something the admin has to
architect and create from scratch. When you install Postfix on a Debian
server, you don't build the config from scratch. You don't architect the
chroot environment it uses from scratch. You install the package, fill
in a few blanks, and you have a working instance. All the dependencies
have been abstracted away by the package installer.

You want these HA implementation decisions to be made by the application
developer or the package maintainer. They know the inner details of the
app best. They've gotten feedback from thousands of deployments.

You want this stuff to be figured out once, shared with the community,
and iteratively perfected. Not built from scratch by each admin.

I assume the variability of HA environments has made doing this
impractical. An HA solution for AWS won't be turn-key on OpenStack.
Despite Richard Pieri's claim that HA tech has "been a mature product in
various forms from various vendors for something like 4 decades," the
space clearly shows plenty of signs of immaturity.


Richard Pieri wrote:
> I see no problem with tailoring a custom solution to a specific problem.
> To the contrary, I see trying to shoe-horn every problem into the 
> cloud's one size fits all mentality as a terrible solution.

It's a matter of perspective. We've come to view x86 hardware as a
commodity platform, and built operating systems to run on top of it that
are a turn key install. You don't build an OS custom for every server
variation. Similarly you shouldn't be custom building an HA solution
over and over again.

But right now, you have to, as there isn't an off-the-shelf solution.

That you view this as ok is just another indicator of the lack of market
maturity. The points of commonality that span across most applications
haven't become obvious.


> Who designs applications for HA environments? Hardly anyone.

Right, but that should change. Postfix, and other apps that Rich Braun
listed, should be designed and packaged with that in mind, because that
is where computing is heading.

 -Tom

-- 
Tom Metro
The Perl Shop, Newton, MA, USA
"Predictable On-demand Perl Consulting."
http://www.theperlshop.com/