Surviving growth- Scale your architecture.

The modern web ecosystem provides a platform for market penetration that can lead to unimaginably fast growth. There are two key factors that allow this; pervasiveness of the web and social networks. I'm not trying to say that achieving growth is easy; competition is fierce; bad impressions are quicker to form and more resistant to disconfirmation than good ones; and when information is cheap, attention becomes expensive.

However if your service is positively endorsed, social networks can precipitate a snowball of traffic that will put any infrastructure through its paces, which often leads to an overnight success or disgruntled users who will think five times before returning.

For startups especially, it is imperative these opportunities are not missed and that there are no kinks in the armour. For instance even if your online store could handle the traffic, can your payment gateway?

At QuayPay we designed scalable infrastructure from the get go.

After the recent purchase of Tumblr by Yahoo for a billion dollars I was rather surprised by their architecture progression, currently they vaguely use similar technologies to Yahoos Flickr, their story is typical for millenial start ups and I doubt they would be able to achieve the same level of success if they started today.

Firstly, for any high traffic service, LAMP stacks are a dead end. Every component should scale out with little or no direct administration and provide monitoring.

We use AngularJS for all our applications, this avoids using a scripting language such as PHP or Ruby on the server side for rendering the view, offloading that responsibility to the client and is great for prototyping.

Rubinius running Rails Metal serves our API requests, everything is paginated to ensure consistent response sizes and this helps maintain constant response times. Lastly our database and search engine are CouchBase with ElasticSearch.

One of the more interesting aspects to Tumblrs architecture is their ID generation, a side effect of using a sharded model, forcing them to build an independant service for generating unique identifiers. We make use of CouchBases low to zero contention atomic increment operations, with independent counters for each model, to implement similar scalable functionality whilst reducing complexity.

The rise of cloud platforms and the products that emerged in the last 10 years have radically changed the landscape. In 2006 I swore on Ruby on Rails, it breathed life into web development for me and started a revolution around tooling. Today anyone can build a web scale application and there has never been a better time to start doing so.

 

Signup to our mailing list to receive more articles like this one. 

Feature image from http://www.vi.net/


Stephen Von Takach Co-founder and CIO at QuayPay. Stephen has designed and delivered scalable architecture solutions for clients such as commonwealth bank and the University of Sydney.