I missed this one last week and it’s the kind of post that I think most Social Media wonks will gloss over – “Social Media Scalability, the New ROI Question” from Shel Israel at Global Neighborhood.

Scalability is an issue for anyone who introduces something new online and wants to grow. It was true for the pioneers who predate social media like ICQ [LINK], for the Stanford start up team for a search engine called Google. It was true for the college project that is now Facebook and so on. The technology that you start with may not be right for large scale. The people that begin it may not be the right people to run it on a large scale. The costs you start with may not fill the petty cash box a year hence.

Wikipedia defines scalability:

Scalability, as a property of systems, is generally difficult to define ^[2] and in any particular case it is necessary to define the specific requirements for scalability on those dimensions which are deemed important. It is a highly significant issue in electronics systems, database, routers, and networking. A system whose performance improves after adding hardware, proportionally to the capacity added, is said to be a scalable system. An algorithm, design, networking protocol, program, or other system is said to scale if it is suitably efficient and practical when applied to large situations (e.g. a large input data set or large number of participating nodes in the case of a distributed system). If the design fails when the quantity increases then it does not scale.

Shel’s only touching the tip of the iceberg.  Scalability is a hamstring issue for social media for a couple of reasons.

• Dynamic interactivity is network/database/processor intensive
• Dynamic interactivity isn’t cache-able like images, css files, etc.
• Traditional solutions to the scalability, ie. throw lots of processors and disks at the problem eventually lead to issues with how to maintain the dynamic experience across a see of boxes and discs
• Remedies to poor architecture issues often outweigh in cost the savings you got in the first place by not doing things the right way.

My system administrators remind me frequently that more machines, more disk space, and more memory isn’t a valid solution to a bad programming. Twitter proved this over the past 6 months, as they found their email centric architecture creaked, moaned and then in June virtually ceased to function, providing an opening for competitors which luckily for them, no one capitalized on.

Unfortunately, the problem is only going to get worse.  Today most social networking apps are being developed by very small development teams, with little funding, and a “build it and they will come” business model that provides no scenario for making an easy transition from “pet project” to “enterprise application.”

The hairy 800 pound gorilla here is this: social media and social networking are highly dynamic and highly interactive mediums.  Instead of the traditional one way flow of content to the user, it’s going all over the place, with our servers as the hub.  Hence our needs for infrastructure are orders of magnitude larger than that which was required just 2 or 3 years ago.

But wait, that’s not all.  This problem is getting significantly worse, and will only grow in momentum.  We’re only just beginning to explore the bounds of video, and for laughs, let’s apply that model of content flow to video: currently it’s predominantly a one way medium on the web.  As it’s developed and fleshed out, the tenets of social media will be applied to it, and it, too will become a omnidirectional medium.  And that’s the point where the issues of scalability start to move out of the data centers, and become an issue for the ISPs and for the Internet as a whole.

That’s, deep down,  the reason that the ISPs are trying to find ways to cap bandwidth on users, and it could become a limiting factor on Internet growth. If they don’t put a cork in the bottle now, they’ll end up with an unsustainable model very soon.

So what are your thoughts: is scalability a problem for Social Media?