Technical Happenings

Transforming Software Architecture with Web as Platform

Focus on the power of people and today’s network opportunity.  Your customers, competitors and all the ideas and innovation is on the network.  There is very few things the browser cannot do, but its still not a great place to develop software.  however, its much bigger that any operating system.

Now we are seeing that no small system can withstand sustained contact with a larger system without being changed. This has been amplified through the move to SOA, but these principles do have complexity problems – there are many frameworks to follow.  The complexity has been addressed in part by REST which is at the heart of the new acronym WOA (Web Orientated Architecture), but there is still a way to go. In effect software architecture needs to be updated to mirror where we are now; everything is connected in the age of network singularity.  And its not the enterprise that houses the biggest systems:

‘ my website is bigger than your enterprise’  the biggest applications are no longer in the enterprise i.e Google, MySpace, Facebook, Amazon …

These are highly distributed and federated systems which often have a social architecture driving there popularity – high network effect.  They move quickly utilizing cutting edge platforms, productivity oriented development tools – Ruby -  and are experts in there ability to scale globally. An example you may not know  clickatell sms and messaging gateway up and running for you in 3-4 minutes.

So what are the design elements of Next generation web 2.0 applications:

• 3rd party sourcing (Cloud)
• Application
• Distribution (Programmable web)
• Social (Network Effect) – the market leaders have this, it’s the key part

What are the production elements around web application development 2.0

• Peer production and crowdsourcing (open source model!)- Moving to peer production provides unprecedented  variety
• open supply chains (programmable web) – build on the shoulders of giants
• Social Systems
• Assembly and Integration
• Perpetual Beta – best products are never finished, by design
• Users are your testers – American Express use there own customers to test.
• Users are your support network

Also Innovations primarily coming from the on-line world:

• cloud computing (cost and agility benefits are unparalleled, amazon 10-15 X cheaper), only one is open source Eucalyptus
• non-relational databases
• productivity-orientated development tools RIA Flex/AIR/Silverlight and stacks like Ruby, CakePHP, Grails and IPhone.

Tags: architecture, cloud computing, Qcon

Situation normal, everything must change – How IT is changing with the advent of cloud computing

How IT is changing with the advent of cloud computing.

Why IT is changing

Not all IT is the same, not all IT has value as some had commoditised. This occurred through the warfare of technological advantage and copycatting.  Anything new is a chance of competitive advantage anything common is a service. Follows the same path as many other industries i.e electricity innovation -> national grid.  If this happens to IT, does it still have strategic value?  Yes as not all IT is commoditised it is always moving along the innovation -> commodity curve, when they get there the are ready to turn into a service.

How this impacts IT

• Disruption – through the growth of the SaaS industry
• Innovation – commoditisation actually promotes more innovation through the stability of the infrastructure i.e componentisation
• Relationships – the differing service model *aaS changes the way an enterprise interacts with it suppliers.  More items can be outsourced, its not common to have a single supplier model.
• No Choice – when  you view you product as an  innovation but the market views it as a commodity – the red queen hypothesis (risk of doing nothing)

Benefits and risk of the cloud

If we view it as a term to show the disruptive force in moving from and innovation to a service based world (commodity).

see many of my other posts for the benefits.  some of the main risks:

• transitional risks
• trusts
• transparency
• failed services on which you have built disappear i.e the company goes out of business or the platform fails
• inability to switch between providers.

Managing the risks

Assuming the platform you are based on disappears.  You need to ensure your providers support the same standards ,this is not the case at this time. It is almost like the commodities (services offered) are in effect the innovation and we find ourselves basing our decisions on trust.  This  will no doubt change with time as the providers increase and the customers demand more from them in an increasing competitive market place.

Why open matters

encourages commoditisation by removing the barriers. Competitive markets depend eventually on open standards, but it is worth noting that vendors push the boundaries and extend these.

What we need

• Ability to run the service on your own machines – build your own cloud
• Ability to switch between own machines and a provider
• Ability to easily switch between providers

Tags: cloud computing, Qcon

Cloud Data Persistence

is this a renaissance for the database?

today we have new problems and new challenges, but the computing power we now have provides alternatives.  This in turn is changing the approaches that are being taken. the following provides examples of the alternatives in the cloud database area.

Physical limitations and computation complexity are driving implementations into new area that we are not use to.  Similar to when multi-cores first came out, many thought lots of software would take account of the multi-cores, but this did not happen that quickly, it’s still happening now.

A regular solution to a social network with large database use is to Shard (divide into chunks across multiple servers).  But remember this has changes for the developer, who now needs to look-up over multiple shards – you cannot use an SQL JOIN across a shard and you cannot guarantee uniqueness and integrity. We are also seeing developers resorting to unnatural acts to deal with the issues of large data sets i.e turning MYSQL into a keyless store (Friendfeed).  good solution for them, but now the DB is no longer a DB.

The Cloud can be views:

• Hiding complexity
• Scalable – elastic resource availability
• Pay as you go
• no need to worry about tuning
• geographical diversity

And can be broken into loose types = the *aaS Model

• Saas – software as a service
• PaaS – platfrom as a service
• TaaS – tools as a service
• IaaS – infrastructure as a service
• ?aaS

All sounds great, so what’s the catch? Safety, geo-graphical availability and commodity hardware.  Also believe it or not the speed of light, which in data terms is still slow when transferring over geographical locations.

Two alternatives to the relational model that can cope with massive datasets in the cloud

Google BigTable

data tables are sharded into tablets and served via a single server, each tablet server can have 1000 tablets.  these table servers have a master and this can be removed and the system will still work for a limited period.

• Distributed store
• hundreds of terabytes
• effectively a big sorted map
• row keys grouped into column families
• data is versioned
• fast, scalable and transnational
• meta data also stored in the same way in the tablets via a route metadata tablet.

You cannot use BigTable yourself but there are some open source alternatives Hypertable Apche HBase.  Also Big Table via Google App Engine, you need to use Python and there is something in-between (although the speaker had not worked out what it is).  But you are getting the benefits of Big Table in a round about way.

Amazon Dynamo

projects Voldermort and Cassandra use this idea.

• Distributed key value store
• Designed for high availability – tolerate network partitions and server failures without effect
• decentralized – no master
• data replicated via consistant hashing
• multi-node reads and writes for redundancy
• objects versioned for consistency
• uses a Vector clock to disambiguate between server version of the same object

And for the lighter touch the smaller alternatives

Amazon Simple DB

• tabular store
• domains which are like tables and contain items
• schemaless
• auto-indexing
• eventually consistent
• no cross domain joins
• query limit to 250 items
• everything is a string

MSFT’s Azure SQL Services – in test

• non-relational – really an XML document store
• Containers which have entities
• Queries through LINQ
• REST and SOAP interfaces

Apache CouchDB – looks pretty good for JavaScript apps.

• Document store in Json
• REST API get,put, post

Other things to watch:

• MongoDB – the speakers own project in this area
• memcachedb – distributed key/value store
• Drizzle – a fork of MYSQL for the cloud.
• Hadoop – distributted file system
• Scalaris – google
  

Tags: amazon, architecture, cloud computing, google, Qcon